--- spec_doc_type: spec spec_id: runtime/edge-node-execution status: 구현됨 source_evidence: - type: contract path: agent-contract/inner/execution-runtime.md notes: Host-neutral provider execution primitives - type: contract path: agent-contract/inner/edge-node-runtime-wire.md notes: Edge-Node registration, execution, tunnel, request-owned workspace artifact, cancellation, command, and refresh wire - type: code path: packages/go/execution/types.go notes: Provider execution and event types - type: code path: packages/go/execution/liveness.go notes: Response-stall timeout default, validation, and RuntimeEvent/ProviderTunnelFrame activity classifiers - type: code path: apps/node/internal/node/runtime_bridge.go notes: Protobuf-to-execution translation with raw stall timeout validation before router/provider invocation - type: code path: apps/edge/internal/transport/server.go notes: Edge-side tunnel-tolerant heartbeat and disconnect supervision - type: code path: apps/node/internal/transport/client.go notes: Node-side tunnel-tolerant heartbeat and reconnect transport - type: code path: apps/edge/internal/service/provider_tunnel.go notes: Provider selection, credential binding validation, reception-aware terminal handoff, lease acquisition, and pre-send fencing - type: code path: apps/edge/internal/service/model_queue_release.go notes: Immutable lease validation, generation/sequence-fenced runtime health overlay, recovery handoff annotation, and exactly-once release - type: code path: apps/edge/internal/service/node_command.go notes: CAPABILITIES dispatch identity retention and exact available recovery evidence application - type: code path: apps/node/internal/node/tunnel_handler.go notes: Provider tunnel handling and recipient-sealed credential lease consumption - type: code path: apps/node/internal/node/liveness_watchdog.go notes: Shared normalized/tunnel stall coordination, close-grace ownership, serialized emission fencing, bounded probe/fence join, and connection-scoped observation sequencing - type: code path: apps/node/internal/node/health_probe.go notes: Bounded independent exact-target health probe coordinator consumed by the stall terminal join - type: code path: apps/node/internal/transport/session.go notes: Connection-scoped monotonic health-observation sequence source - type: code path: packages/go/credentiallease/envelope.go notes: Signed scope validation, recipient sealing, expiry, replay, and exact binding verification - type: test path: apps/node/internal/node/command_test.go notes: Closed provider commands plus fail-closed exact CAPABILITIES health and Session sequence regressions - type: test path: apps/edge/internal/service/provider_health_overlay_test.go notes: S04 binding, stale evidence, normalized/tunnel release races, overlay projection, and CAPABILITIES recovery evidence - type: test path: apps/edge/internal/openai/stream_gate_stall_recovery_test.go notes: S05 always-owned OpenAI recovery, new attempt/provider selection, shared budget, old-transport close, and guard terminals - type: test path: apps/edge/internal/transport/heartbeat_test.go notes: Edge heartbeat liveness profile regression - type: test path: apps/node/internal/transport/heartbeat_test.go notes: Node heartbeat liveness and idle-connection regressions - type: test path: apps/node/internal/node/liveness_watchdog_test.go notes: Manual-clock S01/S02 threshold, progress, terminal, close-grace, ownership, metadata, and late-output evidence - type: test path: apps/node/internal/node/provider_tunnel_test.go notes: Credential preflight admission release regression - type: test path: apps/node/internal/transport/session_test.go notes: Run and tunnel handler lifetime cancellation on disconnect - type: code path: apps/edge/internal/service/single_request_workspace.go notes: Exact configured workspace owner and ready-generation admission projection - type: code path: apps/edge/internal/service/workspace_wire.go notes: Exact-generation dispatch, frozen request-authority construction, closed artifact response validation, bounds, and stable failure translation - type: code path: apps/edge/internal/openai/single_request_plan_stage.go notes: Private fixed Plan stage runner, strict result decoding, and PLAN artifact write - type: test path: apps/edge/internal/openai/single_request_plan_stage_test.go notes: Deterministic Plan request/options/envelope/artifact evidence - type: code path: apps/edge/internal/openai/single_request_work_stage.go notes: Private ornith-fast Work provider/tool loop, request-safe continuation bridge, admitted tool projection, and strict completion evidence - type: test path: apps/edge/internal/openai/single_request_work_stage_test.go notes: Deterministic Work tool continuation, correlation, high-option absence, and bounded completion evidence - type: code path: apps/edge/internal/service/single_request_tool_types.go notes: Closed internal workspace schemas, strict decoding, defensive copies, and raw-free typed result projection - type: code path: apps/edge/internal/service/single_request_tool_loop.go notes: Request-local ordered tool continuation, saved-stage correlation, immutable budgets, and cancellation ownership - type: code path: apps/edge/internal/service/single_request.go notes: Validated closed terminal disposition, frozen terminal ownership, cleanup conversion, and acknowledgement stability - type: code path: apps/edge/internal/openai/single_request_quality_gate.go notes: Request-local S11 stage classification and fixed-hash repetition/no-progress detection - type: test path: apps/edge/internal/openai/single_request_quality_gate_test.go notes: S11 provider, timeout, budget, malformed, context, length, cancel, tool, and no-progress terminal evidence - type: test path: apps/edge/internal/openai/single_request_handler_test.go notes: Buffered Anthropic error/cancel/length mapping, one-ingress evidence, and private-partial exclusion - type: test path: apps/edge/internal/openai/single_request_anthropic_stream_test.go notes: Streaming terminal-disposition mapping, exactly-one terminal, disconnect silence, and private-partial exclusion - type: code path: apps/edge/internal/service/single_request_artifact.go notes: Closed PLAN/REVIEW controller API, shared lazy workspace open, bounded artifact operations, and in-flight cleanup ownership - type: test path: apps/edge/internal/service/single_request_tool_loop_test.go notes: Ordered multi-tool wire evidence plus identity, capability, stale result, budget, deadline, and cancel failures - type: test path: apps/edge/internal/service/single_request_artifact_test.go notes: Artifact-first open sharing, tool-after-artifact reuse, terminal/cancel wait, exactly-once cleanup, and pre-dispatch bounds - type: code path: apps/node/internal/transport/session.go notes: Optional workspace handler registration that preserves legacy provider Handler compatibility - type: code path: apps/node/internal/workspace/runtime.go notes: Closed Darwin/Linux host-exact immutable catalog, opened root authority, operation-aware limits, immutable request-authority copy, and lifecycle ownership - type: test path: apps/node/internal/workspace/runtime_test.go notes: Darwin/Linux positive admission, exact cross-platform mismatch, unsupported-host, empty-catalog compatibility, root identity, and redaction regressions - type: code path: apps/node/internal/workspace/file_executor.go notes: Capability-gated bounded batch listing, descriptor-relative structured write, and non-recursive delete - type: test path: apps/node/internal/workspace/file_executor_test.go notes: Reserved namespace, no-effect symlink/parent/device rejection, bounded listing, atomicity, special-file, and concurrency regressions - type: code path: apps/node/internal/workspace/command_executor.go notes: Exact command-template lookup, minimal allowlisted environment, shared output cap, active-command identity, and terminal result ownership - type: code path: apps/node/internal/workspace/command_process_unix.go notes: Darwin/Linux inherited-root fchdir/exec shim and process-group termination - type: code path: apps/node/internal/workspace/cleanup.go notes: Exactly-once request cleanup ownership, process cancellation and wait, bounded result cache, and locked internal artifact inventory read/write - type: code path: apps/node/internal/workspace/cleanup_path_unix.go notes: No-follow request namespace creation, descriptor enumeration, inventoried file reads, identity validation, and deepest-first non-recursive removal - type: code path: apps/node/internal/node/workspace_handler.go notes: Closed artifact selector mapping plus stable typed open/tool/artifact/cancel/cleanup terminals - type: test path: apps/node/internal/workspace/cleanup_test.go notes: Cleanup races, process groups, timeout, artifact read/write isolation, unsafe entry refusal, identity and device mismatch, user result preservation, and request isolation - type: test path: apps/node/internal/workspace/command_executor_test.go notes: Success, non-zero exit, timeout, context/explicit cancel, child process group, shared output, environment, request isolation, and renamed-root identity evidence - type: test path: apps/node/internal/node/workspace_handler_test.go notes: Typed command/cancel and plan/review artifact mapping, duplicate cancel, not-found, and raw-free stable error evidence - type: test path: apps/edge/internal/service/single_request_workspace_test.go notes: Workspace admission rejection, effective-limit, refresh, and generation-fence regressions - type: test path: apps/edge/internal/service/workspace_wire_test.go notes: Frozen open authority, typed workspace and artifact round trips, malformed response rejection, bounds, cancellation, and stale-generation no-reselection regressions - type: test path: apps/edge/internal/service/single_request_cleanup_test.go notes: Cleanup-before-terminal ordering, success failure conversion, cancellation category preservation, write failure, unopened workspace, and exactly-once terminal races - type: test path: apps/node/internal/bootstrap/workspace_runtime_test.go notes: Path-free startup failure, handler-before-ready composition, and registry/workspace/session/store close-order regressions - type: code path: apps/node/internal/node/liveness_observability.go notes: Node stall counter/histogram and dedicated structured log with closed label values and raw-payload exclusion - type: test path: apps/node/internal/node/liveness_observability_test.go notes: Deterministic S06 Node stall observation regression with closed label values - type: code path: apps/edge/internal/service/provider_health_observability.go notes: Edge overlay evidence/transition counters and dedicated structured log with closed label values and identity exclusion - type: test path: apps/edge/internal/service/provider_health_observability_test.go notes: Deterministic S06 Edge overlay observation regression including sentinel exclusion via TestProviderHealthObservabilityDoesNotExposeSentinels - type: code path: apps/edge/internal/openai/liveness_recovery_observability.go notes: Edge OpenAI eligibility/results counters and dedicated structured log with closed label values and identifier exclusion - type: test path: apps/edge/internal/openai/liveness_recovery_observability_test.go notes: Deterministic S06 OpenAI recovery observation regression with closed label values --- # Edge-Node Provider Execution ## 목적 Edge owns provider selection, queue admission, leases, and connection-generation fencing. Node owns local provider adapters and executes normalized runs or provider HTTP tunnels after a ready handshake. The shared `packages/go/execution` package contains provider lifecycle, registry, execution events, failures, cancellation, usage, and optional tunnel/command primitives. It does not manage host programs or durable conversation state. ## 기능 목록 | 기능 | 설명 | |------|------| | register/readiness | 등록된 Node의 현재 connection이 readiness를 완료한 뒤에만 dispatch한다. | | normalized execution | `adapter + target`으로 provider 실행을 선택하고 ordered `RunEvent` stream을 반환한다. | | single-request coordinator | Immutable admission과 closed stage envelope을 service-owned state graph (`accepted`, `planning`, `working`, `reviewing`, `repairing`, `internal_tool`, `finalizing`, `completed`, `failed`, `cancelled`)로 처리한다. An internal tool result can resume only its saved stage. After a successful workspace open, every terminal path waits for one cleanup before the finalizing candidate can reach surface acknowledgement. | | single-request S11 terminal policy | One validated, copy-safe terminal disposition is frozen across envelope/result/progress with kinds `end_turn`, `length`, `error`, and `cancelled`. Error classes are `provider`, `validation`, `timeout`, `budget`, `repetition`, `malformed`, `context`, `internal_tool`, and `workspace_cleanup`. Cleanup can replace a pending success/length before publication; no acknowledgement race can publish a second terminal. | | single-request observation evidence | Stage-pure timing, tool/cleanup/total counts, cardinality-bounded labels, Node logs, and raw-free correlation are documented for the single-request path. `iop_anthropic_single_request_ingress_total` is strictly unlabeled. A separate `edge_single_request_terminal_rejection` event projects only the fixed terminal kind/error class and HTTP status, so `malformed` and `validation` remain distinguishable without raw model output. The private stage decoder accepts and discards only bounded standard Chat Completions `usage` bookkeeping and optional string `message.reasoning_content`; neither enters stage results or artifacts, while non-string reasoning and unknown envelope members fail closed. Gemini Plan/Review also admit only exact non-empty `extra_content.google.thought_signature`; terminal signatures are discarded and a Review tool-call signature is replayed only in the matching request-local Gemini continuation. Work, artifacts, results, and observations never retain it. The Claude qualification child disables SDK retry and session-title generation so only the actual task can consume ingress. Internal tool names, raw arguments, and private results are absent from public output and log projections. Actual Claude timing evidence on an approved IOP Node is explicitly deferred to `claude-smoke` (SDD S12); deterministic coordinator/tool-loop tests do not imply external qualification. | | workspace admission | An opaque `workspace_ref` resolves only through the configured Node catalog. Edge freezes the exact configured owner, dispatch-ready connection generation, closed operation/command/environment-name capabilities, and effective limits before executor startup; unavailable, foreign, pending, malformed, and stale candidates fail closed without fallback or reselection. | | workspace runtime wire | The dedicated `WorkspaceOpen`/`Tool`/`Artifact`/`Cancel`/`Cleanup` request-response families carry immutable coordinator identities and closed status/error codes. `WorkspaceArtifact` admits only enum-selected `PLAN`/`REVIEW` and `READ`/`WRITE`; it carries no relative path. Edge overwrites open capabilities with frozen request authority; Node copies only catalog-subset operations/command ids and equal-or-lower effective limits. | | workspace tool executor | A validated `darwin|linux` Node catalog owns opened root and directory handles only when every entry platform matches the host exactly. Windows, unknown hosts, and cross-platform catalogs fail before root open; empty catalogs remain compatible. Go 1.24-compatible no-follow file primitives provide bounded read, bounded list, structured write, and non-recursive delete. Exact operator-owned command templates run through an inherited-root `fchdir`/`exec` shim with minimal allowlisted environment, shared stdout/stderr bounds, process-group timeout/cancel, and stable typed results. OS is runtime evidence rather than a caller-visible selector. | | internal workspace tool loop | The service decodes only `workspace_read`, `workspace_list`, `workspace_write`, `workspace_delete`, and `workspace_command`, opens the admitted workspace once, dispatches one call at a time on the frozen generation, and delivers one deep-copied typed result to the emitting executor continuation. Unique request/stage/tool correlation, per-stage iteration/output/deadline limits, request wall-clock budget, and typed cancel fail closed without external continuation or reselection. | | request-owned internal artifacts | `SingleRequestController` exposes closed plan/review read/write operations. Artifact calls and model workspace tools share one serialized lazy `WorkspaceOpen`, the exact admitted Node generation, the active stage deadline, the immutable output bound, in-flight work accounting, and one terminal cleanup. Node alone maps selectors to `plan.md` and `review.md`, and inventoried descriptor-relative reads fail closed on identity replacement. | | Plan stage | The Plan runner emits the `planning` envelope, sends the immutable task through the frozen Gemini Chat binding with `reasoning_effort=high` and an Edge-owned OpenAI `json_schema` response format, requires one strict small `plan`/`verification` JSON result, and writes deterministic bounded Markdown through `SingleRequestArtifactPlan`. Stage options cannot replace the schema, and the strict parser still enforces exact nonempty canonical fields. | | Work stage | The `ornith-fast` Work runner reads the closed PLAN artifact, projects only the admitted workspace tools, and resumes the same frozen provider route after exactly correlated Node results. It rejects any Work `reasoning_effort`, malformed or multiple tool calls, and empty completion or verification evidence. | | request-owned cleanup | Node creates and inventories only `.iop/job/` internal state, cancels and waits for all active command groups, validates the exact tree without following entries, and removes matching artifacts deepest-first with non-recursive descriptor operations. Symlinks, special files, foreign devices, identity replacements, and unowned entries fail closed. User results and sibling request state are preserved. Concurrent cleanup callers receive one bounded cached typed result. | | provider raw tunnel | 선택된 provider의 HTTP/SSE를 `ProviderTunnelRequest`/`ProviderTunnelFrame`으로 relay하며 순서와 단일 terminal outcome을 보장한다. | | response-stall activity contract | 선택된 provider의 response-stall timeout을 normalized/tunnel request에 보존한다. Node는 wire zero를 `300000ms`로 해석하고 invalid raw value를 adapter 호출 전에 거부한다. Runtime event의 terminal type은 payload/usage보다 우선하며 non-terminal usage는 progress다. | | Node stall watchdog | Node가 normalized run과 raw tunnel에 하나의 activity watchdog을 적용한다. progress만 timer를 reset하며, stall은 `response_stalled` terminal 하나와 Node-owned safe metadata를 만들어 normalized `RunEvent`와 raw `ProviderTunnelFrame` wire의 optional typed `ExecutionFailure` 필드에 싣는다. stall claim 뒤에는 bounded close grace fence와 독립 exact-target health probe를 직렬 확장 없이 join한다. close grace 안에 provider return이 확인된 경우만 `Retryable` capability hint를 준다. | | Node health evidence join | stall terminal에 three-way health evidence를 싣는다: `provider_health` status와 `liveness_classification` normalization이 `available`/`request_stalled`, `unavailable`/`provider_unhealthy`, `unknown`/`health_unknown` 쌍으로 fail-closed된다. probe 성공은 progress reset·fence 변경·retry authority가 아니며 late output은 fenced 상태를 유지한다. | | health observation sequence | transport Session이 connection-scoped monotonic `health_observation_seq`를 소유한다. 새 connection은 0에서 시작해 첫 finalized observation이 1이며, 같은 connection의 normalized/tunnel observation이 source를 공유해 동시에도 유일 증가값을 받는다. internal/unbound 경로는 key를 생략한다. | | Edge terminal health handoff | Edge validates authoritative reception node/generation plus the immutable provider/adapter/target lease before applying typed stall evidence. Every validated current bound stall receives `provider_id`, validated health, and `recovery_handoff=confirmed`, while only fresh unavailable evidence lowers a separate runtime overlay; the token never grants replay eligibility. Every valid current terminal still releases its lease exactly once. | | CAPABILITIES recovery | Node runs the same bounded exact-target `ProbeHealth` and returns stable adapter/target/status plus the next Session sequence. Edge recovers exactly one matching current-generation unavailable provider only from a strictly newer `available` result; malformed, ambiguous, stale, unknown, and unavailable responses are no-ops. | | recovery candidate preference | `ProviderPoolDispatchRequest` carries `AvoidProviderID` and `AllowAvoidedProviderFallback`. Every admission (initial and queued re-resolution) prefers a runtime-eligible alternate over the avoided provider; only the explicit fallback flag (derived from exact probe-backed `available` evidence) permits re-selecting the avoided provider when no alternate exists. Zero values preserve current selection. This is selection policy only: no retry loop, slot reservation, priority change, persistence, or retry counter. | | OpenAI typed-stall consumption | Every supported Chat/Responses normalized or tunnel request has one unconditional runtime liveness owner, independent of configured semantic activation. It converts only the Edge-confirmed typed stall handoff into a raw-free StreamGate event, owns pre-commit eligibility, and closes the already fenced old transport before re-admission; Node does not grant replay authority. | | tunnel-tolerant liveness | Edge와 Node는 30초 heartbeat interval과 45초 response wait를 공통으로 사용해 긴 prompt prefill이나 streaming backpressure 중의 정상 connection을 조기에 끊지 않는다. | | reconnect/generation fencing | 현재 connection이 종료되면 해당 generation만 fence하고 Node supervisor가 reconnect한다. Heartbeat wait를 넘긴 경우의 close reason은 `heartbeat_timeout`이다. | | cancellation/command | `run_id`로 현재 run만 취소하며 command는 capabilities, transport status, Ollama API tunnel로 제한한다. | | managed credential lease | Edge가 principal·route·slot·profile·target·Node·revision·generation을 binding한 sealed lease를 발급하고 Node가 capacity admission 후 provider 실행 직전에만 연다. | ## 범위 - `session_id`는 event와 command result의 opaque correlation일 뿐이며 같은 값을 재사용해도 모든 run은 독립적이다. - provider usage, capacity, queue pressure, lifecycle, reconnect, tool calling은 Edge-Node 실행 경로에서 계속 지원한다. - single-request coordinator owns the service-level workspace admission described above as well as executor envelope privacy and the service-owned state graph. It exposes no workspace root, command executable/template/arguments, or environment values to the coordinator-facing binding. - The request-local single-request quality gate classifies provider/tool timeouts, exhausted stage/request budgets, first proven repeated action/result no-progress, malformed calls/results, context/output limits, cancellation, internal-tool failures, and workspace cleanup into the closed terminal vocabulary. It retains only fixed hashes for repetition evidence and never retries, reselects, falls back, exposes a partial success, or starts a second request after classification. - The service freezes the first public terminal candidate. Legacy successful results normalize to `end_turn`; output limits produce `length`; caller disconnect produces silent `cancelled`; validation/context become `invalid_request_error`; other errors become `api_error`. Buffered and SSE projectors share that policy, emit at most one terminal, and never expose private partial stage content for `length`. This completes deterministic S11 `error-cancel` evidence without changing the Edge-Node protobuf wire. S12 external Claude qualification on an approved IOP Node remains pending. - The request-local internal tool loop is implemented between the coordinator and the dedicated workspace wire. Strict decode and capability checks happen before wire effects; Node results are accepted only for the one pending call and return only bounded typed fields to the same optional executor continuation. Repeated or stale identities, malformed/denied calls, exhausted immutable budgets, and cancellation terminate internally without selecting another Node or involving the HTTP caller. - Request-owned plan and review artifact access is implemented between the controller and the same dedicated workspace wire. Only `SingleRequestArtifactPlan` and `SingleRequestArtifactReview` are accepted. Artifact and model-tool callers share one serialized open attempt and the same opened cleanup gate; terminal and cancellation paths wait for in-flight artifact work before issuing exactly one cleanup. Edge bounds writes before dispatch and reads before acceptance, validates the echoed kind/operation and canonical terminal, and never reselects after a generation mismatch. Node maps the closed selectors to `plan.md` and `review.md`, validates the inventoried parent/file identity with descriptor-relative no-follow reads, and never grants the public workspace tool surface access to `.iop`. - The private Plan stage is installed in the composite single-request executor at Edge input startup (`apps/edge/internal/input/manager.go`). Its provider codec accepts only frozen Plan options and selected dispatch facts, uses the admitted stage deadline and exact output limit, accepts only `RESPONSE_START`, zero or more `BODY`, then `END`, and projects all provider failures to a generic internal failure. The fixed Plan prompt and Edge-owned OpenAI `json_schema` response format request exactly `plan` and `verification`; caller/config options cannot override the format, and the strict parser retains the semantic nonempty/exact-field boundary before the closed PLAN artifact is written. - The private Work stage is installed in the composite single-request executor at Edge input startup (`apps/edge/internal/input/manager.go`). It reads only `SingleRequestArtifactPlan`, retains only request/stage/tool identifiers while waiting for the coordinator-owned continuation, and sends no `reasoning_effort` field in an initial or resumed provider request. Its provider messages contain the immutable task, PLAN, admitted tool schemas, and bounded typed tool results; Review/repair and composite installation are active, while external Claude qualification remains explicitly deferred to S12 (`claude-smoke`). - The Node-private workspace request/result wire is implemented, including catalog delivery, parser registration, optional handler behavior, stable typed failures, generation-fenced dispatch, context-cancel propagation, and request cleanup. Before ready, a non-empty catalog requires a supported `darwin|linux` host and exact entry/host matching before any root open; unsupported and cross-platform catalogs fail closed while empty catalogs remain compatible. The Node installs the workspace handler before ready and cleans active requests before closing workspace authority ahead of session/store teardown. Request authority is immutable and request-local. File operations reserve `.iop`, reject symlink/mount/replaced-parent/special-file paths before effects, process bounded list batches with deterministic truncation, and use a same-parent structured write. Command execution resolves only admitted ids to fixed templates, enters the already-opened root descriptor through `fchdir`, provides only allowlisted environment entries, shares one output cap across drained stdout/stderr, and owns the complete process group through exit, timeout, context cancel, exact request/tool cancel, or request cleanup. - managed mode는 등록과 dispatch 전에 CA로 검증된 Edge/Node workload identity를 요구한다. - revoked, disabled, expired, stale, replayed, wrong-recipient, mismatched lease는 provider나 credential fallback 없이 fail closed한다. IOP no longer provides persistent shell sessions, terminal emulation, process resume, local working-directory execution context, arbitrary host commands, or local quota/status probing. The current spec maps reviewed Node and Edge observability producers to S06 behavior and deterministic tests. Node exposes bounded stall counters/histograms and dedicated structured logs with closed label values and raw-payload exclusion. Edge service queue exposes bounded overlay evidence/transition counters and dedicated structured logs with closed label values and identity exclusion. Edge OpenAI server exposes bounded eligibility/results counters and dedicated structured logs with closed label values and identifier exclusion. All projections are local observations and do not widen the wire protocol. Single-request lifecycle observation evidence links ingress=1, request-total=1, terminal=1, stage/tool/cleanup counts, and raw-free correlation for one real POST. `iop_anthropic_single_request_ingress_total` is strictly unlabeled (no request_id, stage_id, provider identity, content, or workspace reference). Internal tool names, raw arguments, and private results are absent from public output and log projections. Stage-pure timing, cardinality-bounded labels, and privacy semantics are documented here. Actual Claude timing evidence on an approved IOP Node is explicitly deferred to `claude-smoke` (SDD S12); deterministic coordinator/tool-loop tests do not imply external qualification. ## 주요 흐름 ```mermaid sequenceDiagram participant Edge participant Node participant Provider Node->>Edge: RegisterRequest Edge-->>Node: RegisterResponse + config Node->>Edge: NodeReadyRequest Edge-->>Node: NodeReadyResponse opt admitted single-request internal workspace call Edge->>Node: WorkspaceOpenRequest once (frozen generation) Node-->>Edge: WorkspaceOpenResponse opt coordinator-owned artifact access Edge->>Node: WorkspaceArtifactRequest(PLAN or REVIEW, READ or WRITE) Node->>Node: map selector to plan.md or review.md and validate inventory Node-->>Edge: bounded typed WorkspaceArtifactResponse end loop one ordered pending call Edge->>Node: WorkspaceToolRequest(request, stage, tool) Node-->>Edge: bounded typed WorkspaceToolResponse end Edge->>Node: WorkspaceCleanupRequest once before terminal commit Node->>Node: cancel/wait request process groups and validate inventory Node-->>Edge: typed WorkspaceCleanupResponse Note over Edge: expose finalizing only after successful cleanup Note over Edge: observation: ingress=1, request-total=1, terminal=1, stage/tool/cleanup counts, raw-free correlation end Edge->>Node: ProviderTunnelRequest Node->>Provider: HTTP/SSE request Provider-->>Node: status/header/body stream Node-->>Edge: ordered ProviderTunnelFrame stream Note over Edge,Node: heartbeat 30s interval / 45s wait alt heartbeat wait exceeded Edge--xNode: current generation fenced Node->>Edge: supervised reconnect end ``` ## 계약 - Edge-Node wire: `agent-contract/inner/edge-node-runtime-wire.md` - provider execution primitives: `agent-contract/inner/execution-runtime.md` Heartbeat interval/wait는 protobuf field가 아닌 양쪽 transport 구현의 liveness profile이다. `response_stall_timeout_ms`만 provider execution request wire에 추가되며 provider response shape은 바뀌지 않는다. ## 설정/데이터/이벤트 - Edge와 Node의 현재 heartbeat interval은 30초, response wait는 45초다. - 이 값은 runtime YAML model config나 `max_tokens`/context 설정이 아니라 transport 구현 상수다. - 45초 동안 heartbeat response가 없으면 current connection을 `heartbeat_timeout`으로 닫고 provider resource를 offline 처리한 뒤 reconnect/queue 재평가를 수행한다. - response-stall timeout은 provider config가 source이며 winning candidate가 re-resolution된 뒤의 request까지 같은 effective value를 보존한다. request hard timeout, queue timeout, transport heartbeat, client response-idle timeout과 timer lifecycle은 별도 소유권이다. ## 검증 - `go test -count=1 ./packages/go/execution ./apps/node/... ./apps/edge/internal/service` - `go test -race -count=1 ./packages/go/execution ./apps/node/internal/node ./apps/edge/internal/service` - `go test -race -count=1 ./apps/edge/internal/node -run 'TestRegistryReadyOwnerSnapshot'` - `go test -race -count=1 ./apps/edge/internal/service -run 'TestSingleRequestWorkspace'` - `go test -race -count=1 ./apps/edge/internal/service -run 'Test(InternalWorkspaceTool|SingleRequestInternalToolLoop)'` - `go test -race -count=1 ./apps/edge/internal/service ./apps/node/internal/transport -run 'Test.*(WorkspaceArtifact|SingleRequestArtifact)'` - `go test -count=1 ./apps/node/internal/workspace ./apps/node/internal/node ./apps/node/internal/transport -run 'Test.*(InternalArtifact|WorkspaceArtifact)'` - `go test -race -count=1 ./apps/edge/internal/node ./apps/edge/internal/service ./apps/edge/internal/transport ./apps/node/internal/transport -run 'Test(BuildConfigPayload.*Workspace|WorkspaceWire|NodeParserMapWorkspace|SessionWorkspace|EdgeParserMapWorkspace)'` - `go test -race -count=1 ./apps/node/internal/workspace -run 'Test(CommandExecutor|WorkspaceCommandHelperProcess)'` - `go test -race -count=1 ./apps/node/internal/node -run 'TestNodeWorkspace(Command|Cancel)'` - `go test -race -count=1 ./apps/node/internal/workspace -run 'TestWorkspaceCleanup'` - `go test -race -count=1 ./apps/node/internal/node ./apps/edge/internal/service -run 'Test(NodeWorkspaceCleanup|SingleRequestCleanup)'` - `go test -count=1 ./apps/node/internal/transport ./apps/edge/internal/transport` - `go test -race -count=1 ./apps/node/internal/transport ./apps/edge/internal/transport` - 실제 provider tunnel 검증은 5초를 넘는 긴 prefill과 streaming 응답 동안 Node가 connected/healthy를 유지하고, 응답이 정상 terminal을 반환하며, `heartbeat_timeout`이 발생하지 않는지 확인한다. - `go test -count=1 ./apps/node/internal/node -run '^TestNodeLivenessObservability'` — deterministic Node stall observation with closed label values and raw-payload exclusion. - `go test -count=1 ./apps/edge/internal/service -run '^TestProviderHealthObservability'` — deterministic Edge overlay evidence/transition with closed label values and identity exclusion; `TestProviderHealthObservabilityDoesNotExposeSentinels` covers the sentinel/prohibited-value guard. - `go test -count=1 ./apps/edge/internal/openai -run '^(TestOpenAILivenessObservationSink|TestOpenAILivenessRecoveryObservability)$'` — deterministic OpenAI recovery eligibility/results with closed label values and identifier exclusion. - `go test -count=1 ./apps/edge/internal/openai -run 'TestAnthropicSingleRequestObservation'` — deterministic single-request observation evidence: ingress=1, request-total=1, terminal=1, stage/tool/cleanup counts, raw-free correlation, and unlabeled metric assertion. - `go test -count=1 ./apps/edge/internal/openai -run 'TestSingleRequest(ProviderStage|PlanStage)'` — deterministic frozen provider codec and Plan stage evidence, including high reasoning, stage-owned JSON Schema override protection, ordered tunnel frames, strict JSON, planning envelope, and `plan.md` artifact selection. - `go test -race -count=1 ./apps/edge/internal/openai -run 'TestSingleRequestWork(Stage|ToolBridge)'` — deterministic ornith-fast Work tool loop, correlation isolation, cancellation cleanup, strict completion evidence, and Work reasoning-option absence. - `go test -race ./apps/edge/internal/service ./apps/edge/internal/openai -run 'Test(SingleRequestTerminalDisposition|SingleRequestQualityGate|AnthropicSingleRequestErrorCancelMatrix|SingleRequestAnthropicStreamTerminalDisposition)' -count=1` — deterministic S11 error-cancel/length matrix, first-terminal ownership, one ingress, no second request, disconnect silence, and raw-free output evidence. ## 한계와 주의사항 - 30/45초 liveness profile은 provider 응답 token 상한이나 model context window를 늘리지 않는다. 요청 중단 원인 판정 시 model 설정과 transport disconnect를 별도로 확인한다. - 45초를 넘겨 실제 heartbeat response가 없는 connection은 기존과 같이 오프라인 처리하고 reconnect한다. - Node owns local detection, cancellation, emission fencing, confirmed/unconfirmed ownership close, exact-target probe joining, connection-scoped observation sequencing, and the bounded `iop_node_response_stalls_total` / `iop_node_response_stall_duration_seconds` / `node_response_stall_observation` projections with closed label values. - Edge owns reception-generation and immutable-lease validation, the generation-scoped runtime health overlay, `iop_edge_provider_health_evidence_total` / `iop_edge_provider_health_transitions_total` / `edge_provider_health_observation` projections with closed label values, effective admission/snapshot projection, and exact later CAPABILITIES recovery. - The always-owned supported OpenAI ingress runtime owns commit, cancellation, side-effect, snapshot, shared-budget, candidate, and replay decisions, and exposes `iop_edge_liveness_recovery_eligibility_total` / `iop_edge_liveness_recovery_results_total` / `edge_liveness_recovery_observation` projections with closed label values. - Node retry and `recovery_eligible` remain prohibited. Hard deadline and connection disconnect continue to take precedence over a simultaneous stall timer. - Operational projections never widen the wire protocol; they carry no new frame, field, ordering rule, or retry semantic. - Workspace admission and the private wire both fence the exact ready connection generation. The wire never exposes workspace fields through provider `RunRequest`, `NodeCommand`, or public API output. The executor exposes no caller access to `.iop`; only request-owned internal runtime code can derive and inventory `.iop/job/`. Structured write input is required for WRITE, while legacy content-only input remains rejected. COMMAND is non-interactive and has no shell, PTY, arbitrary argv, ambient environment, path-based cwd lookup, or persistent process ownership. Cleanup never rolls back or deletes user-requested workspace results. - The composite single-request executor is installed at Edge input startup (`apps/edge/internal/input/manager.go`), wiring the active Plan -> Work -> Review stage pipeline for single-request execution. Private stage outcomes use the implemented closed S11 terminal policy and stop without retry/fallback or a second request. Deterministic local activation and terminal evidence are proven, while actual external Claude qualification remains explicitly deferred to S12 (`claude-smoke`). - Single-request observation evidence (ingress=1, request-total=1, terminal=1, stage/tool/cleanup counts, raw-free correlation) is documented and tested. `iop_anthropic_single_request_ingress_total` is strictly unlabeled. Internal tool names, raw arguments, and private results are absent from public output and log projections. Actual Claude timing evidence on an approved IOP Node is explicitly deferred to `claude-smoke` (SDD S12); deterministic coordinator/tool-loop tests do not imply external qualification. ## 변경 기록 - 2026-08-08: Expanded workspace runtime admission to the closed `darwin|linux` implementation set with exact catalog/host matching before root open, preserved empty-catalog compatibility, and kept Windows/unknown hosts fail-closed. - 2026-08-07: Implemented the S11 `error-cancel` boundary: one frozen service terminal disposition, request-local typed stage classification, fixed-hash repetition/no-progress detection, shared buffered/SSE Anthropic mapping, silent disconnect cancellation, private-partial suppression for `max_tokens`, and deterministic one-ingress/one-terminal/no-second-request evidence. The Edge-Node protobuf wire is unchanged and S12 remains pending. - 2026-08-07: Installed the composite single-request executor at Edge input startup (`apps/edge/internal/input/manager.go`), activating the Plan -> Work -> Review stage pipeline. Production construction evidence is test-covered (`apps/edge/internal/input/manager_test.go`), while actual Claude/Mac external qualification remains explicitly deferred to S12 (`claude-smoke`). - 2026-08-02: provider tunnel의 긴 prompt prefill과 streaming backpressure를 정상 traffic으로 허용하도록 Edge/Node heartbeat profile을 30초 interval/45초 wait로 복원한 현재 구현과 회귀 검증을 반영했다 (`apps/edge/internal/transport/server.go`, `apps/node/internal/transport/client.go`). - 2026-08-04: provider response-stall timeout의 config validation, selected-candidate propagation, Node adapter-visible retention, and activity classification contract를 반영했다. - 2026-08-04: Added the shared Node run/tunnel watchdog coordinator, serialized tunnel emission fence, pre-provider admission cleanup, disconnect-bound handler lifetime, and deterministic S01/S02 manual-clock evidence. - 2026-08-04: Joined the bounded close-grace fence and the independent exact-target health probe into one stall terminal carrying three-way health evidence, and added the connection-scoped `health_observation_seq` sourced from the transport Session. - 2026-08-05: Added authoritative Edge terminal handoff, immutable lease binding, generation/sequence-fenced runtime provider health, exactly-once normalized/tunnel release, and fail-closed Session-sequenced CAPABILITIES recovery without config-health mutation or replay authorization. - 2026-08-05: Added runtime-local OpenAI consumption of confirmed typed stalls, including cancel-free old-transport close and provider-pool avoidance hints for ExactReplay. - 2026-08-05: Made supported OpenAI Chat/Responses normalized and tunnel liveness ownership unconditional and added S05 recovery/guard evidence independent of semantic policy activation. - 2026-08-06: Mapped reviewed Node, Edge overlay, and OpenAI recovery observability producers to S06 behavior with deterministic test evidence. Node exposes `iop_node_response_stalls_total`, `iop_node_response_stall_duration_seconds`, and `node_response_stall_observation` (source: `apps/node/internal/node/liveness_observability.go`; test: `TestNodeLivenessObservability`). Edge service queue exposes `iop_edge_provider_health_evidence_total`, `iop_edge_provider_health_transitions_total`, and `edge_provider_health_observation` (source: `apps/edge/internal/service/provider_health_observability.go`; test: `TestProviderHealthObservability`, `TestProviderHealthObservabilityDoesNotExposeSentinels`). Edge OpenAI server exposes `iop_edge_liveness_recovery_eligibility_total`, `iop_edge_liveness_recovery_results_total`, and `edge_liveness_recovery_observation` (source: `apps/edge/internal/openai/liveness_recovery_observability.go`; test: `TestOpenAILivenessObservationSink`, `TestOpenAILivenessRecoveryObservability`). All projections carry only closed, low-cardinality label values and exclude raw payloads, credentials, and unbounded identifiers from metric labels and general logs. The wire protocol is unchanged. - 2026-08-06: Added implemented Edge workspace admission for single-request execution: an opaque `workspace_ref` binds to one configured ready Node generation and a closed capability projection before executor startup. Admission rejects unavailable, foreign, pending, malformed, and stale candidates without fallback or reselection; Node-private wire, executor, filesystem path, and symlink enforcement remain deferred. - 2026-08-06: Added the dedicated Edge-Node workspace wire. `NodeConfigPayload` now delivers the approved catalog; `WorkspaceOpen`/`Tool`/`Cancel`/`Cleanup` messages have closed typed outcomes, immutable coordinator identities, parser registration, and an optional Node handler. Edge dispatch is generation-fenced and context cancellation sends one typed cancel. Node filesystem and process execution are intentionally deferred. - 2026-08-06: Completed the reviewed workspace file boundary repair. Edge now sends only frozen request authority, Node admits immutable catalog subsets/lower limits, and structured write reaches the file executor while legacy incomplete input remains rejected. The Go 1.24-compatible descriptor-relative no-follow write path validates before effects, bounded list processing retains fixed state, startup errors are path-free, and composition proves handler-before-ready plus workspace-before-session/store teardown. Command execution/cancellation and cleanup remain deferred. - 2026-08-07: Implemented exact-template workspace COMMAND and typed cancellation. The Node uses an inherited-root `fchdir`/`exec` shim, minimal allowlisted environment, a shared draining stdout/stderr cap, and one process-group result owner across exit, timeout, context cancel, and exact request/tool cancel. Focused race tests cover non-zero exit, output overflow, descendant termination, cross-request isolation, and configured-root rename/replacement. Artifact cleanup remains deferred. - 2026-08-07: Implemented the coordinator-owned internal workspace tool loop with closed strict schemas, one-time exact-generation open, ordered pending-call correlation, deep-copied raw-free continuation results, immutable iteration/output/deadline budgets, typed cancellation, and real one-POST multi-tool privacy evidence. - 2026-08-07: Added request-owned workspace cleanup. Node inventories its exact internal request namespace and artifacts, cancels and waits for all request command groups, refuses unowned, symlink, special-file, identity, and filesystem-boundary mismatches, and removes only validated entries with no-follow non-recursive descriptor operations. Edge gates every opened-workspace terminal path on one typed cleanup before finalizing acknowledgement; cleanup failure converts pending success while preserving existing failure or cancellation categories. - 2026-08-07: Added the closed request-owned PLAN/REVIEW artifact wire and controller lifecycle. Artifact calls share the model-tool lazy open and terminal cleanup gate, use the frozen Node generation and immutable bounds, and map only inside Node to inventoried `plan.md`/`review.md` files. Provider-specific stage drivers and actual Claude qualification remain deferred. - 2026-08-08: Synchronized single-request lifecycle observation evidence: stage-pure timing (planning/working/reviewing/repairing/finalizing/completed/failed/cancelled), tool/cleanup/total counts, cardinality-bounded labels, Node logs, and raw-free correlation. External Claude/Mac timing evidence is explicitly deferred to `claude-smoke` (SDD S12). Deterministic coordinator/tool-loop tests cover the full single-request path without implying external qualification. - 2026-08-08: Made the Gemini Plan output deterministic with an Edge-owned OpenAI `json_schema` response format for the exact `plan`/`verification` object, retained the strict nonempty parser, and added a raw-free fixed terminal-rejection event that distinguishes `malformed` from binding `validation` without widening metric labels. - 2026-08-07: Added the private Plan stage and its fail-closed provider codec. The component uses only frozen Gemini dispatch/options, ordered bounded tunnel decoding, strict small plan/verification JSON, and the closed `SingleRequestArtifactPlan` write. It is not installed; Work, Review/repair, activation, and S12 qualification remain deferred. - 2026-08-07: Added the private ornith-fast Work stage. It reads PLAN through the closed artifact controller, emits only admitted workspace schemas, bridges exact request/stage/tool results without retaining payloads, and resumes the frozen route with bounded tool evidence. Work rejects `reasoning_effort`; Review/repair, composite installation, and S12 external qualification remain deferred.