iop/agent-spec/runtime/edge-node-execution.md

spec_doc_type	spec_id	status	source_evidence
spec	runtime/edge-node-execution	구현됨	type path notes contract agent-contract/inner/execution-runtime.md Host-neutral provider execution primitives type path notes contract agent-contract/inner/edge-node-runtime-wire.md Edge-Node registration, execution, tunnel, request-owned workspace artifact, cancellation, command, and refresh wire type path notes code packages/go/execution/types.go Provider execution and event types type path notes code packages/go/execution/liveness.go Response-stall timeout default, validation, and RuntimeEvent/ProviderTunnelFrame activity classifiers type path notes code apps/node/internal/node/runtime_bridge.go Protobuf-to-execution translation with raw stall timeout validation before router/provider invocation type path notes code apps/edge/internal/transport/server.go Edge-side tunnel-tolerant heartbeat and disconnect supervision type path notes code apps/node/internal/transport/client.go Node-side tunnel-tolerant heartbeat and reconnect transport type path notes code apps/edge/internal/service/provider_tunnel.go Provider selection, credential binding validation, reception-aware terminal handoff, lease acquisition, and pre-send fencing type path notes code apps/edge/internal/service/model_queue_release.go Immutable lease validation, generation/sequence-fenced runtime health overlay, recovery handoff annotation, and exactly-once release type path notes code apps/edge/internal/service/node_command.go CAPABILITIES dispatch identity retention and exact available recovery evidence application type path notes code apps/node/internal/node/tunnel_handler.go Provider tunnel handling and recipient-sealed credential lease consumption type path notes code apps/node/internal/node/liveness_watchdog.go Shared normalized/tunnel stall coordination, close-grace ownership, serialized emission fencing, bounded probe/fence join, and connection-scoped observation sequencing type path notes code apps/node/internal/node/health_probe.go Bounded independent exact-target health probe coordinator consumed by the stall terminal join type path notes code apps/node/internal/transport/session.go Connection-scoped monotonic health-observation sequence source type path notes code packages/go/credentiallease/envelope.go Signed scope validation, recipient sealing, expiry, replay, and exact binding verification type path notes test apps/node/internal/node/command_test.go Closed provider commands plus fail-closed exact CAPABILITIES health and Session sequence regressions type path notes test apps/edge/internal/service/provider_health_overlay_test.go S04 binding, stale evidence, normalized/tunnel release races, overlay projection, and CAPABILITIES recovery evidence type path notes test apps/edge/internal/openai/stream_gate_stall_recovery_test.go S05 always-owned OpenAI recovery, new attempt/provider selection, shared budget, old-transport close, and guard terminals type path notes test apps/edge/internal/transport/heartbeat_test.go Edge heartbeat liveness profile regression type path notes test apps/node/internal/transport/heartbeat_test.go Node heartbeat liveness and idle-connection regressions type path notes test apps/node/internal/node/liveness_watchdog_test.go Manual-clock S01/S02 threshold, progress, terminal, close-grace, ownership, metadata, and late-output evidence type path notes test apps/node/internal/node/provider_tunnel_test.go Credential preflight admission release regression type path notes test apps/node/internal/transport/session_test.go Run and tunnel handler lifetime cancellation on disconnect type path notes code apps/edge/internal/service/single_request_workspace.go Exact configured workspace owner and ready-generation admission projection type path notes code apps/edge/internal/service/workspace_wire.go Exact-generation dispatch, frozen request-authority construction, closed artifact response validation, bounds, and stable failure translation type path notes code apps/edge/internal/openai/single_request_plan_stage.go Private fixed Plan stage runner, strict result decoding, and PLAN artifact write type path notes test apps/edge/internal/openai/single_request_plan_stage_test.go Deterministic Plan request/options/envelope/artifact evidence type path notes code apps/edge/internal/openai/single_request_work_stage.go Private ornith-fast Work provider/tool loop, request-safe continuation bridge, admitted tool projection, and strict completion evidence type path notes test apps/edge/internal/openai/single_request_work_stage_test.go Deterministic Work tool continuation, correlation, high-option absence, and bounded completion evidence type path notes code apps/edge/internal/service/single_request_tool_types.go Closed internal workspace schemas, strict decoding, defensive copies, and raw-free typed result projection type path notes code apps/edge/internal/service/single_request_tool_loop.go Request-local ordered tool continuation, saved-stage correlation, immutable budgets, and cancellation ownership type path notes code apps/edge/internal/service/single_request.go Validated closed terminal disposition, frozen terminal ownership, cleanup conversion, and acknowledgement stability type path notes code apps/edge/internal/openai/single_request_quality_gate.go Request-local S11 stage classification and fixed-hash repetition/no-progress detection type path notes test apps/edge/internal/openai/single_request_quality_gate_test.go S11 provider, timeout, budget, malformed, context, length, cancel, tool, and no-progress terminal evidence type path notes test apps/edge/internal/openai/single_request_handler_test.go Buffered Anthropic error/cancel/length mapping, one-ingress evidence, and private-partial exclusion type path notes test apps/edge/internal/openai/single_request_anthropic_stream_test.go Streaming terminal-disposition mapping, exactly-one terminal, disconnect silence, and private-partial exclusion type path notes code apps/edge/internal/service/single_request_artifact.go Closed PLAN/REVIEW controller API, shared lazy workspace open, bounded artifact operations, and in-flight cleanup ownership type path notes test apps/edge/internal/service/single_request_tool_loop_test.go Ordered multi-tool wire evidence plus identity, capability, stale result, budget, deadline, and cancel failures type path notes test apps/edge/internal/service/single_request_artifact_test.go Artifact-first open sharing, tool-after-artifact reuse, terminal/cancel wait, exactly-once cleanup, and pre-dispatch bounds type path notes code apps/node/internal/transport/session.go Optional workspace handler registration that preserves legacy provider Handler compatibility type path notes code apps/node/internal/workspace/runtime.go Closed Darwin/Linux host-exact immutable catalog, opened root authority, operation-aware limits, immutable request-authority copy, and lifecycle ownership type path notes test apps/node/internal/workspace/runtime_test.go Darwin/Linux positive admission, exact cross-platform mismatch, unsupported-host, empty-catalog compatibility, root identity, and redaction regressions type path notes code apps/node/internal/workspace/file_executor.go Capability-gated bounded batch listing, descriptor-relative structured write, and non-recursive delete type path notes test apps/node/internal/workspace/file_executor_test.go Reserved namespace, no-effect symlink/parent/device rejection, bounded listing, atomicity, special-file, and concurrency regressions type path notes code apps/node/internal/workspace/command_executor.go Exact command-template lookup, minimal allowlisted environment, shared output cap, active-command identity, and terminal result ownership type path notes code apps/node/internal/workspace/command_process_unix.go Darwin/Linux inherited-root fchdir/exec shim and process-group termination type path notes code apps/node/internal/workspace/cleanup.go Exactly-once request cleanup ownership, process cancellation and wait, bounded result cache, and locked internal artifact inventory read/write type path notes code apps/node/internal/workspace/cleanup_path_unix.go No-follow request namespace creation, descriptor enumeration, inventoried file reads, identity validation, and deepest-first non-recursive removal type path notes code apps/node/internal/node/workspace_handler.go Closed artifact selector mapping plus stable typed open/tool/artifact/cancel/cleanup terminals type path notes test apps/node/internal/workspace/cleanup_test.go Cleanup races, process groups, timeout, artifact read/write isolation, unsafe entry refusal, identity and device mismatch, user result preservation, and request isolation type path notes test apps/node/internal/workspace/command_executor_test.go Success, non-zero exit, timeout, context/explicit cancel, child process group, shared output, environment, request isolation, and renamed-root identity evidence type path notes test apps/node/internal/node/workspace_handler_test.go Typed command/cancel and plan/review artifact mapping, duplicate cancel, not-found, and raw-free stable error evidence type path notes test apps/edge/internal/service/single_request_workspace_test.go Workspace admission rejection, effective-limit, refresh, and generation-fence regressions type path notes test apps/edge/internal/service/workspace_wire_test.go Frozen open authority, typed workspace and artifact round trips, malformed response rejection, bounds, cancellation, and stale-generation no-reselection regressions type path notes test apps/edge/internal/service/single_request_cleanup_test.go Cleanup-before-terminal ordering, success failure conversion, cancellation category preservation, write failure, unopened workspace, and exactly-once terminal races type path notes test apps/node/internal/bootstrap/workspace_runtime_test.go Path-free startup failure, handler-before-ready composition, and registry/workspace/session/store close-order regressions type path notes code apps/node/internal/node/liveness_observability.go Node stall counter/histogram and dedicated structured log with closed label values and raw-payload exclusion type path notes test apps/node/internal/node/liveness_observability_test.go Deterministic S06 Node stall observation regression with closed label values type path notes code apps/edge/internal/service/provider_health_observability.go Edge overlay evidence/transition counters and dedicated structured log with closed label values and identity exclusion type path notes test apps/edge/internal/service/provider_health_observability_test.go Deterministic S06 Edge overlay observation regression including sentinel exclusion via TestProviderHealthObservabilityDoesNotExposeSentinels type path notes code apps/edge/internal/openai/liveness_recovery_observability.go Edge OpenAI eligibility/results counters and dedicated structured log with closed label values and identifier exclusion type path notes test apps/edge/internal/openai/liveness_recovery_observability_test.go Deterministic S06 OpenAI recovery observation regression with closed label values

Edge-Node Provider Execution

목적

Edge owns provider selection, queue admission, leases, and connection-generation fencing. Node owns local provider adapters and executes normalized runs or provider HTTP tunnels after a ready handshake.

The shared packages/go/execution package contains provider lifecycle, registry, execution events, failures, cancellation, usage, and optional tunnel/command primitives. It does not manage host programs or durable conversation state.

기능 목록

기능	설명
register/readiness	등록된 Node의 현재 connection이 readiness를 완료한 뒤에만 dispatch한다.
normalized execution	adapter + target으로 provider 실행을 선택하고 ordered RunEvent stream을 반환한다.
single-request coordinator	Immutable admission과 closed stage envelope을 service-owned state graph (accepted, planning, working, reviewing, repairing, internal_tool, finalizing, completed, failed, cancelled)로 처리한다. An internal tool result can resume only its saved stage. After a successful workspace open, every terminal path waits for one cleanup before the finalizing candidate can reach surface acknowledgement.
single-request S11 terminal policy	One validated, copy-safe terminal disposition is frozen across envelope/result/progress with kinds end_turn, length, error, and cancelled. Error classes are provider, validation, timeout, budget, repetition, malformed, context, internal_tool, and workspace_cleanup. Cleanup can replace a pending success/length before publication; no acknowledgement race can publish a second terminal.
single-request observation evidence	Stage-pure timing, tool/cleanup/total counts, cardinality-bounded labels, Node logs, and raw-free correlation are documented for the single-request path. iop_anthropic_single_request_ingress_total is strictly unlabeled. A separate edge_single_request_terminal_rejection event projects only the fixed terminal kind/error class and HTTP status, so malformed and validation remain distinguishable without raw model output. The private stage decoder accepts and discards only bounded standard Chat Completions usage bookkeeping and optional string message.reasoning_content; neither enters stage results or artifacts, while non-string reasoning and unknown envelope members fail closed. Gemini Plan/Review also admit only exact non-empty extra_content.google.thought_signature; terminal signatures are discarded and a Review tool-call signature is replayed only in the matching request-local Gemini continuation. Work, artifacts, results, and observations never retain it. The Claude qualification child disables SDK retry and session-title generation so only the actual task can consume ingress. Internal tool names, raw arguments, and private results are absent from public output and log projections. Actual Claude timing evidence on an approved IOP Node is explicitly deferred to claude-smoke (SDD S12); deterministic coordinator/tool-loop tests do not imply external qualification.
workspace admission	An opaque workspace_ref resolves only through the configured Node catalog. Edge freezes the exact configured owner, dispatch-ready connection generation, closed operation/command/environment-name capabilities, and effective limits before executor startup; unavailable, foreign, pending, malformed, and stale candidates fail closed without fallback or reselection.
workspace runtime wire	The dedicated WorkspaceOpen/Tool/Artifact/Cancel/Cleanup request-response families carry immutable coordinator identities and closed status/error codes. WorkspaceArtifact admits only enum-selected PLAN/REVIEW and READ/WRITE; it carries no relative path. Edge overwrites open capabilities with frozen request authority; Node copies only catalog-subset operations/command ids and equal-or-lower effective limits.
workspace tool executor	A validated `darwin
internal workspace tool loop	The service decodes only workspace_read, workspace_list, workspace_write, workspace_delete, and workspace_command, opens the admitted workspace once, dispatches one call at a time on the frozen generation, and delivers one deep-copied typed result to the emitting executor continuation. Unique request/stage/tool correlation, per-stage iteration/output/deadline limits, request wall-clock budget, and typed cancel fail closed without external continuation or reselection.
request-owned internal artifacts	SingleRequestController exposes closed plan/review read/write operations. Artifact calls and model workspace tools share one serialized lazy WorkspaceOpen, the exact admitted Node generation, the active stage deadline, the immutable output bound, in-flight work accounting, and one terminal cleanup. Node alone maps selectors to plan.md and review.md, and inventoried descriptor-relative reads fail closed on identity replacement.
Plan stage	The Plan runner emits the planning envelope, sends the immutable task through the frozen Gemini Chat binding with reasoning_effort=high and an Edge-owned OpenAI json_schema response format, requires one strict small plan/verification JSON result, and writes deterministic bounded Markdown through SingleRequestArtifactPlan. Stage options cannot replace the schema, and the strict parser still enforces exact nonempty canonical fields.
Work stage	The ornith-fast Work runner reads the closed PLAN artifact, projects only the admitted workspace tools, and resumes the same frozen provider route after exactly correlated Node results. It rejects any Work reasoning_effort, malformed or multiple tool calls, and empty completion or verification evidence.
request-owned cleanup	Node creates and inventories only .iop/job/<request_id> internal state, cancels and waits for all active command groups, validates the exact tree without following entries, and removes matching artifacts deepest-first with non-recursive descriptor operations. Symlinks, special files, foreign devices, identity replacements, and unowned entries fail closed. User results and sibling request state are preserved. Concurrent cleanup callers receive one bounded cached typed result.
provider raw tunnel	선택된 provider의 HTTP/SSE를 ProviderTunnelRequest/ProviderTunnelFrame으로 relay하며 순서와 단일 terminal outcome을 보장한다.
response-stall activity contract	선택된 provider의 response-stall timeout을 normalized/tunnel request에 보존한다. Node는 wire zero를 300000ms로 해석하고 invalid raw value를 adapter 호출 전에 거부한다. Runtime event의 terminal type은 payload/usage보다 우선하며 non-terminal usage는 progress다.
Node stall watchdog	Node가 normalized run과 raw tunnel에 하나의 activity watchdog을 적용한다. progress만 timer를 reset하며, stall은 response_stalled terminal 하나와 Node-owned safe metadata를 만들어 normalized RunEvent와 raw ProviderTunnelFrame wire의 optional typed ExecutionFailure 필드에 싣는다. stall claim 뒤에는 bounded close grace fence와 독립 exact-target health probe를 직렬 확장 없이 join한다. close grace 안에 provider return이 확인된 경우만 Retryable capability hint를 준다.
Node health evidence join	stall terminal에 three-way health evidence를 싣는다: provider_health status와 liveness_classification normalization이 available/request_stalled, unavailable/provider_unhealthy, unknown/health_unknown 쌍으로 fail-closed된다. probe 성공은 progress reset·fence 변경·retry authority가 아니며 late output은 fenced 상태를 유지한다.
health observation sequence	transport Session이 connection-scoped monotonic health_observation_seq를 소유한다. 새 connection은 0에서 시작해 첫 finalized observation이 1이며, 같은 connection의 normalized/tunnel observation이 source를 공유해 동시에도 유일 증가값을 받는다. internal/unbound 경로는 key를 생략한다.
Edge terminal health handoff	Edge validates authoritative reception node/generation plus the immutable provider/adapter/target lease before applying typed stall evidence. Every validated current bound stall receives provider_id, validated health, and recovery_handoff=confirmed, while only fresh unavailable evidence lowers a separate runtime overlay; the token never grants replay eligibility. Every valid current terminal still releases its lease exactly once.
CAPABILITIES recovery	Node runs the same bounded exact-target ProbeHealth and returns stable adapter/target/status plus the next Session sequence. Edge recovers exactly one matching current-generation unavailable provider only from a strictly newer available result; malformed, ambiguous, stale, unknown, and unavailable responses are no-ops.
recovery candidate preference	ProviderPoolDispatchRequest carries AvoidProviderID and AllowAvoidedProviderFallback. Every admission (initial and queued re-resolution) prefers a runtime-eligible alternate over the avoided provider; only the explicit fallback flag (derived from exact probe-backed available evidence) permits re-selecting the avoided provider when no alternate exists. Zero values preserve current selection. This is selection policy only: no retry loop, slot reservation, priority change, persistence, or retry counter.
OpenAI typed-stall consumption	Every supported Chat/Responses normalized or tunnel request has one unconditional runtime liveness owner, independent of configured semantic activation. It converts only the Edge-confirmed typed stall handoff into a raw-free StreamGate event, owns pre-commit eligibility, and closes the already fenced old transport before re-admission; Node does not grant replay authority.
tunnel-tolerant liveness	Edge와 Node는 30초 heartbeat interval과 45초 response wait를 공통으로 사용해 긴 prompt prefill이나 streaming backpressure 중의 정상 connection을 조기에 끊지 않는다.
reconnect/generation fencing	현재 connection이 종료되면 해당 generation만 fence하고 Node supervisor가 reconnect한다. Heartbeat wait를 넘긴 경우의 close reason은 heartbeat_timeout이다.
cancellation/command	run_id로 현재 run만 취소하며 command는 capabilities, transport status, Ollama API tunnel로 제한한다.
managed credential lease	Edge가 principal·route·slot·profile·target·Node·revision·generation을 binding한 sealed lease를 발급하고 Node가 capacity admission 후 provider 실행 직전에만 연다.

범위

• session_id는 event와 command result의 opaque correlation일 뿐이며 같은 값을 재사용해도 모든 run은 독립적이다.
• provider usage, capacity, queue pressure, lifecycle, reconnect, tool calling은 Edge-Node 실행 경로에서 계속 지원한다.
• single-request coordinator owns the service-level workspace admission described above as well as executor envelope privacy and the service-owned state graph. It exposes no workspace root, command executable/template/arguments, or environment values to the coordinator-facing binding.
• The request-local single-request quality gate classifies provider/tool timeouts, exhausted stage/request budgets, first proven repeated action/result no-progress, malformed calls/results, context/output limits, cancellation, internal-tool failures, and workspace cleanup into the closed terminal vocabulary. It retains only fixed hashes for repetition evidence and never retries, reselects, falls back, exposes a partial success, or starts a second request after classification.
• The service freezes the first public terminal candidate. Legacy successful results normalize to end_turn; output limits produce length; caller disconnect produces silent cancelled; validation/context become invalid_request_error; other errors become api_error. Buffered and SSE projectors share that policy, emit at most one terminal, and never expose private partial stage content for length. This completes deterministic S11 error-cancel evidence without changing the Edge-Node protobuf wire. S12 external Claude qualification on an approved IOP Node remains pending.
• The request-local internal tool loop is implemented between the coordinator and the dedicated workspace wire. Strict decode and capability checks happen before wire effects; Node results are accepted only for the one pending call and return only bounded typed fields to the same optional executor continuation. Repeated or stale identities, malformed/denied calls, exhausted immutable budgets, and cancellation terminate internally without selecting another Node or involving the HTTP caller.
• Request-owned plan and review artifact access is implemented between the controller and the same dedicated workspace wire. Only SingleRequestArtifactPlan and SingleRequestArtifactReview are accepted. Artifact and model-tool callers share one serialized open attempt and the same opened cleanup gate; terminal and cancellation paths wait for in-flight artifact work before issuing exactly one cleanup. Edge bounds writes before dispatch and reads before acceptance, validates the echoed kind/operation and canonical terminal, and never reselects after a generation mismatch. Node maps the closed selectors to plan.md and review.md, validates the inventoried parent/file identity with descriptor-relative no-follow reads, and never grants the public workspace tool surface access to .iop.
• The private Plan stage is installed in the composite single-request executor at Edge input startup (apps/edge/internal/input/manager.go). Its provider codec accepts only frozen Plan options and selected dispatch facts, uses the admitted stage deadline and exact output limit, accepts only RESPONSE_START, zero or more BODY, then END, and projects all provider failures to a generic internal failure. The fixed Plan prompt and Edge-owned OpenAI json_schema response format request exactly plan and verification; caller/config options cannot override the format, and the strict parser retains the semantic nonempty/exact-field boundary before the closed PLAN artifact is written.
• The private Work stage is installed in the composite single-request executor at Edge input startup (apps/edge/internal/input/manager.go). It reads only SingleRequestArtifactPlan, retains only request/stage/tool identifiers while waiting for the coordinator-owned continuation, and sends no reasoning_effort field in an initial or resumed provider request. Its provider messages contain the immutable task, PLAN, admitted tool schemas, and bounded typed tool results; Review/repair and composite installation are active, while external Claude qualification remains explicitly deferred to S12 (claude-smoke).
• The Node-private workspace request/result wire is implemented, including catalog delivery, parser registration, optional handler behavior, stable typed failures, generation-fenced dispatch, context-cancel propagation, and request cleanup. Before ready, a non-empty catalog requires a supported darwin|linux host and exact entry/host matching before any root open; unsupported and cross-platform catalogs fail closed while empty catalogs remain compatible. The Node installs the workspace handler before ready and cleans active requests before closing workspace authority ahead of session/store teardown. Request authority is immutable and request-local. File operations reserve .iop, reject symlink/mount/replaced-parent/special-file paths before effects, process bounded list batches with deterministic truncation, and use a same-parent structured write. Command execution resolves only admitted ids to fixed templates, enters the already-opened root descriptor through fchdir, provides only allowlisted environment entries, shares one output cap across drained stdout/stderr, and owns the complete process group through exit, timeout, context cancel, exact request/tool cancel, or request cleanup.
• managed mode는 등록과 dispatch 전에 CA로 검증된 Edge/Node workload identity를 요구한다.
• revoked, disabled, expired, stale, replayed, wrong-recipient, mismatched lease는 provider나 credential fallback 없이 fail closed한다.

IOP no longer provides persistent shell sessions, terminal emulation, process resume, local working-directory execution context, arbitrary host commands, or local quota/status probing.

The current spec maps reviewed Node and Edge observability producers to S06 behavior and deterministic tests. Node exposes bounded stall counters/histograms and dedicated structured logs with closed label values and raw-payload exclusion. Edge service queue exposes bounded overlay evidence/transition counters and dedicated structured logs with closed label values and identity exclusion. Edge OpenAI server exposes bounded eligibility/results counters and dedicated structured logs with closed label values and identifier exclusion. All projections are local observations and do not widen the wire protocol.

Single-request lifecycle observation evidence links ingress=1, request-total=1, terminal=1, stage/tool/cleanup counts, and raw-free correlation for one real POST. iop_anthropic_single_request_ingress_total is strictly unlabeled (no request_id, stage_id, provider identity, content, or workspace reference). Internal tool names, raw arguments, and private results are absent from public output and log projections. Stage-pure timing, cardinality-bounded labels, and privacy semantics are documented here. Actual Claude timing evidence on an approved IOP Node is explicitly deferred to claude-smoke (SDD S12); deterministic coordinator/tool-loop tests do not imply external qualification.

주요 흐름

sequenceDiagram
    participant Edge
    participant Node
    participant Provider

    Node->>Edge: RegisterRequest
    Edge-->>Node: RegisterResponse + config
    Node->>Edge: NodeReadyRequest
    Edge-->>Node: NodeReadyResponse
    opt admitted single-request internal workspace call
        Edge->>Node: WorkspaceOpenRequest once (frozen generation)
        Node-->>Edge: WorkspaceOpenResponse
        opt coordinator-owned artifact access
            Edge->>Node: WorkspaceArtifactRequest(PLAN or REVIEW, READ or WRITE)
            Node->>Node: map selector to plan.md or review.md and validate inventory
            Node-->>Edge: bounded typed WorkspaceArtifactResponse
        end
        loop one ordered pending call
            Edge->>Node: WorkspaceToolRequest(request, stage, tool)
            Node-->>Edge: bounded typed WorkspaceToolResponse
        end
        Edge->>Node: WorkspaceCleanupRequest once before terminal commit
        Node->>Node: cancel/wait request process groups and validate inventory
        Node-->>Edge: typed WorkspaceCleanupResponse
        Note over Edge: expose finalizing only after successful cleanup
        Note over Edge: observation: ingress=1, request-total=1, terminal=1, stage/tool/cleanup counts, raw-free correlation
    end
    Edge->>Node: ProviderTunnelRequest
    Node->>Provider: HTTP/SSE request
    Provider-->>Node: status/header/body stream
    Node-->>Edge: ordered ProviderTunnelFrame stream
    Note over Edge,Node: heartbeat 30s interval / 45s wait
    alt heartbeat wait exceeded
        Edge--xNode: current generation fenced
        Node->>Edge: supervised reconnect
    end

계약

• Edge-Node wire: agent-contract/inner/edge-node-runtime-wire.md
• provider execution primitives: agent-contract/inner/execution-runtime.md

Heartbeat interval/wait는 protobuf field가 아닌 양쪽 transport 구현의 liveness profile이다. response_stall_timeout_ms만 provider execution request wire에 추가되며 provider response shape은 바뀌지 않는다.

설정/데이터/이벤트

• Edge와 Node의 현재 heartbeat interval은 30초, response wait는 45초다.
• 이 값은 runtime YAML model config나 max_tokens/context 설정이 아니라 transport 구현 상수다.
• 45초 동안 heartbeat response가 없으면 current connection을 heartbeat_timeout으로 닫고 provider resource를 offline 처리한 뒤 reconnect/queue 재평가를 수행한다.
• response-stall timeout은 provider config가 source이며 winning candidate가 re-resolution된 뒤의 request까지 같은 effective value를 보존한다. request hard timeout, queue timeout, transport heartbeat, client response-idle timeout과 timer lifecycle은 별도 소유권이다.

검증

• go test -count=1 ./packages/go/execution ./apps/node/... ./apps/edge/internal/service
• go test -race -count=1 ./packages/go/execution ./apps/node/internal/node ./apps/edge/internal/service
• go test -race -count=1 ./apps/edge/internal/node -run 'TestRegistryReadyOwnerSnapshot'
• go test -race -count=1 ./apps/edge/internal/service -run 'TestSingleRequestWorkspace'
• go test -race -count=1 ./apps/edge/internal/service -run 'Test(InternalWorkspaceTool|SingleRequestInternalToolLoop)'
• go test -race -count=1 ./apps/edge/internal/service ./apps/node/internal/transport -run 'Test.*(WorkspaceArtifact|SingleRequestArtifact)'
• go test -count=1 ./apps/node/internal/workspace ./apps/node/internal/node ./apps/node/internal/transport -run 'Test.*(InternalArtifact|WorkspaceArtifact)'
• go test -race -count=1 ./apps/edge/internal/node ./apps/edge/internal/service ./apps/edge/internal/transport ./apps/node/internal/transport -run 'Test(BuildConfigPayload.*Workspace|WorkspaceWire|NodeParserMapWorkspace|SessionWorkspace|EdgeParserMapWorkspace)'
• go test -race -count=1 ./apps/node/internal/workspace -run 'Test(CommandExecutor|WorkspaceCommandHelperProcess)'
• go test -race -count=1 ./apps/node/internal/node -run 'TestNodeWorkspace(Command|Cancel)'
• go test -race -count=1 ./apps/node/internal/workspace -run 'TestWorkspaceCleanup'
• go test -race -count=1 ./apps/node/internal/node ./apps/edge/internal/service -run 'Test(NodeWorkspaceCleanup|SingleRequestCleanup)'
• go test -count=1 ./apps/node/internal/transport ./apps/edge/internal/transport
• go test -race -count=1 ./apps/node/internal/transport ./apps/edge/internal/transport
• 실제 provider tunnel 검증은 5초를 넘는 긴 prefill과 streaming 응답 동안 Node가 connected/healthy를 유지하고, 응답이 정상 terminal을 반환하며, heartbeat_timeout이 발생하지 않는지 확인한다.
• go test -count=1 ./apps/node/internal/node -run '^TestNodeLivenessObservability' — deterministic Node stall observation with closed label values and raw-payload exclusion.
• go test -count=1 ./apps/edge/internal/service -run '^TestProviderHealthObservability' — deterministic Edge overlay evidence/transition with closed label values and identity exclusion; TestProviderHealthObservabilityDoesNotExposeSentinels covers the sentinel/prohibited-value guard.
• go test -count=1 ./apps/edge/internal/openai -run '^(TestOpenAILivenessObservationSink|TestOpenAILivenessRecoveryObservability)$' — deterministic OpenAI recovery eligibility/results with closed label values and identifier exclusion.
• go test -count=1 ./apps/edge/internal/openai -run 'TestAnthropicSingleRequestObservation' — deterministic single-request observation evidence: ingress=1, request-total=1, terminal=1, stage/tool/cleanup counts, raw-free correlation, and unlabeled metric assertion.
• go test -count=1 ./apps/edge/internal/openai -run 'TestSingleRequest(ProviderStage|PlanStage)' — deterministic frozen provider codec and Plan stage evidence, including high reasoning, stage-owned JSON Schema override protection, ordered tunnel frames, strict JSON, planning envelope, and plan.md artifact selection.
• go test -race -count=1 ./apps/edge/internal/openai -run 'TestSingleRequestWork(Stage|ToolBridge)' — deterministic ornith-fast Work tool loop, correlation isolation, cancellation cleanup, strict completion evidence, and Work reasoning-option absence.
• go test -race ./apps/edge/internal/service ./apps/edge/internal/openai -run 'Test(SingleRequestTerminalDisposition|SingleRequestQualityGate|AnthropicSingleRequestErrorCancelMatrix|SingleRequestAnthropicStreamTerminalDisposition)' -count=1 — deterministic S11 error-cancel/length matrix, first-terminal ownership, one ingress, no second request, disconnect silence, and raw-free output evidence.

한계와 주의사항

• 30/45초 liveness profile은 provider 응답 token 상한이나 model context window를 늘리지 않는다. 요청 중단 원인 판정 시 model 설정과 transport disconnect를 별도로 확인한다.
• 45초를 넘겨 실제 heartbeat response가 없는 connection은 기존과 같이 오프라인 처리하고 reconnect한다.
• Node owns local detection, cancellation, emission fencing, confirmed/unconfirmed ownership close, exact-target probe joining, connection-scoped observation sequencing, and the bounded iop_node_response_stalls_total / iop_node_response_stall_duration_seconds / node_response_stall_observation projections with closed label values.
• Edge owns reception-generation and immutable-lease validation, the generation-scoped runtime health overlay, iop_edge_provider_health_evidence_total / iop_edge_provider_health_transitions_total / edge_provider_health_observation projections with closed label values, effective admission/snapshot projection, and exact later CAPABILITIES recovery.
• The always-owned supported OpenAI ingress runtime owns commit, cancellation, side-effect, snapshot, shared-budget, candidate, and replay decisions, and exposes iop_edge_liveness_recovery_eligibility_total / iop_edge_liveness_recovery_results_total / edge_liveness_recovery_observation projections with closed label values.
• Node retry and recovery_eligible remain prohibited. Hard deadline and connection disconnect continue to take precedence over a simultaneous stall timer.
• Operational projections never widen the wire protocol; they carry no new frame, field, ordering rule, or retry semantic.
• Workspace admission and the private wire both fence the exact ready connection generation. The wire never exposes workspace fields through provider RunRequest, NodeCommand, or public API output. The executor exposes no caller access to .iop; only request-owned internal runtime code can derive and inventory .iop/job/<request_id>. Structured write input is required for WRITE, while legacy content-only input remains rejected. COMMAND is non-interactive and has no shell, PTY, arbitrary argv, ambient environment, path-based cwd lookup, or persistent process ownership. Cleanup never rolls back or deletes user-requested workspace results.
• The composite single-request executor is installed at Edge input startup (apps/edge/internal/input/manager.go), wiring the active Plan -> Work -> Review stage pipeline for single-request execution. Private stage outcomes use the implemented closed S11 terminal policy and stop without retry/fallback or a second request. Deterministic local activation and terminal evidence are proven, while actual external Claude qualification remains explicitly deferred to S12 (claude-smoke).
• Single-request observation evidence (ingress=1, request-total=1, terminal=1, stage/tool/cleanup counts, raw-free correlation) is documented and tested. iop_anthropic_single_request_ingress_total is strictly unlabeled. Internal tool names, raw arguments, and private results are absent from public output and log projections. Actual Claude timing evidence on an approved IOP Node is explicitly deferred to claude-smoke (SDD S12); deterministic coordinator/tool-loop tests do not imply external qualification.

변경 기록

• 2026-08-08: Expanded workspace runtime admission to the closed darwin|linux implementation set with exact catalog/host matching before root open, preserved empty-catalog compatibility, and kept Windows/unknown hosts fail-closed.
• 2026-08-07: Implemented the S11 error-cancel boundary: one frozen service terminal disposition, request-local typed stage classification, fixed-hash repetition/no-progress detection, shared buffered/SSE Anthropic mapping, silent disconnect cancellation, private-partial suppression for max_tokens, and deterministic one-ingress/one-terminal/no-second-request evidence. The Edge-Node protobuf wire is unchanged and S12 remains pending.
• 2026-08-07: Installed the composite single-request executor at Edge input startup (apps/edge/internal/input/manager.go), activating the Plan -> Work -> Review stage pipeline. Production construction evidence is test-covered (apps/edge/internal/input/manager_test.go), while actual Claude/Mac external qualification remains explicitly deferred to S12 (claude-smoke).
• 2026-08-02: provider tunnel의 긴 prompt prefill과 streaming backpressure를 정상 traffic으로 허용하도록 Edge/Node heartbeat profile을 30초 interval/45초 wait로 복원한 현재 구현과 회귀 검증을 반영했다 (apps/edge/internal/transport/server.go, apps/node/internal/transport/client.go).
• 2026-08-04: provider response-stall timeout의 config validation, selected-candidate propagation, Node adapter-visible retention, and activity classification contract를 반영했다.
• 2026-08-04: Added the shared Node run/tunnel watchdog coordinator, serialized tunnel emission fence, pre-provider admission cleanup, disconnect-bound handler lifetime, and deterministic S01/S02 manual-clock evidence.
• 2026-08-04: Joined the bounded close-grace fence and the independent exact-target health probe into one stall terminal carrying three-way health evidence, and added the connection-scoped health_observation_seq sourced from the transport Session.
• 2026-08-05: Added authoritative Edge terminal handoff, immutable lease binding, generation/sequence-fenced runtime provider health, exactly-once normalized/tunnel release, and fail-closed Session-sequenced CAPABILITIES recovery without config-health mutation or replay authorization.
• 2026-08-05: Added runtime-local OpenAI consumption of confirmed typed stalls, including cancel-free old-transport close and provider-pool avoidance hints for ExactReplay.
• 2026-08-05: Made supported OpenAI Chat/Responses normalized and tunnel liveness ownership unconditional and added S05 recovery/guard evidence independent of semantic policy activation.
• 2026-08-06: Mapped reviewed Node, Edge overlay, and OpenAI recovery observability producers to S06 behavior with deterministic test evidence. Node exposes iop_node_response_stalls_total, iop_node_response_stall_duration_seconds, and node_response_stall_observation (source: apps/node/internal/node/liveness_observability.go; test: TestNodeLivenessObservability). Edge service queue exposes iop_edge_provider_health_evidence_total, iop_edge_provider_health_transitions_total, and edge_provider_health_observation (source: apps/edge/internal/service/provider_health_observability.go; test: TestProviderHealthObservability, TestProviderHealthObservabilityDoesNotExposeSentinels). Edge OpenAI server exposes iop_edge_liveness_recovery_eligibility_total, iop_edge_liveness_recovery_results_total, and edge_liveness_recovery_observation (source: apps/edge/internal/openai/liveness_recovery_observability.go; test: TestOpenAILivenessObservationSink, TestOpenAILivenessRecoveryObservability). All projections carry only closed, low-cardinality label values and exclude raw payloads, credentials, and unbounded identifiers from metric labels and general logs. The wire protocol is unchanged.
• 2026-08-06: Added implemented Edge workspace admission for single-request execution: an opaque workspace_ref binds to one configured ready Node generation and a closed capability projection before executor startup. Admission rejects unavailable, foreign, pending, malformed, and stale candidates without fallback or reselection; Node-private wire, executor, filesystem path, and symlink enforcement remain deferred.
• 2026-08-06: Added the dedicated Edge-Node workspace wire. NodeConfigPayload now delivers the approved catalog; WorkspaceOpen/Tool/Cancel/Cleanup messages have closed typed outcomes, immutable coordinator identities, parser registration, and an optional Node handler. Edge dispatch is generation-fenced and context cancellation sends one typed cancel. Node filesystem and process execution are intentionally deferred.
• 2026-08-06: Completed the reviewed workspace file boundary repair. Edge now sends only frozen request authority, Node admits immutable catalog subsets/lower limits, and structured write reaches the file executor while legacy incomplete input remains rejected. The Go 1.24-compatible descriptor-relative no-follow write path validates before effects, bounded list processing retains fixed state, startup errors are path-free, and composition proves handler-before-ready plus workspace-before-session/store teardown. Command execution/cancellation and cleanup remain deferred.
• 2026-08-07: Implemented exact-template workspace COMMAND and typed cancellation. The Node uses an inherited-root fchdir/exec shim, minimal allowlisted environment, a shared draining stdout/stderr cap, and one process-group result owner across exit, timeout, context cancel, and exact request/tool cancel. Focused race tests cover non-zero exit, output overflow, descendant termination, cross-request isolation, and configured-root rename/replacement. Artifact cleanup remains deferred.
• 2026-08-07: Implemented the coordinator-owned internal workspace tool loop with closed strict schemas, one-time exact-generation open, ordered pending-call correlation, deep-copied raw-free continuation results, immutable iteration/output/deadline budgets, typed cancellation, and real one-POST multi-tool privacy evidence.
• 2026-08-07: Added request-owned workspace cleanup. Node inventories its exact internal request namespace and artifacts, cancels and waits for all request command groups, refuses unowned, symlink, special-file, identity, and filesystem-boundary mismatches, and removes only validated entries with no-follow non-recursive descriptor operations. Edge gates every opened-workspace terminal path on one typed cleanup before finalizing acknowledgement; cleanup failure converts pending success while preserving existing failure or cancellation categories.
• 2026-08-07: Added the closed request-owned PLAN/REVIEW artifact wire and controller lifecycle. Artifact calls share the model-tool lazy open and terminal cleanup gate, use the frozen Node generation and immutable bounds, and map only inside Node to inventoried plan.md/review.md files. Provider-specific stage drivers and actual Claude qualification remain deferred.
• 2026-08-08: Synchronized single-request lifecycle observation evidence: stage-pure timing (planning/working/reviewing/repairing/finalizing/completed/failed/cancelled), tool/cleanup/total counts, cardinality-bounded labels, Node logs, and raw-free correlation. External Claude/Mac timing evidence is explicitly deferred to claude-smoke (SDD S12). Deterministic coordinator/tool-loop tests cover the full single-request path without implying external qualification.
• 2026-08-08: Made the Gemini Plan output deterministic with an Edge-owned OpenAI json_schema response format for the exact plan/verification object, retained the strict nonempty parser, and added a raw-free fixed terminal-rejection event that distinguishes malformed from binding validation without widening metric labels.
• 2026-08-07: Added the private Plan stage and its fail-closed provider codec. The component uses only frozen Gemini dispatch/options, ordered bounded tunnel decoding, strict small plan/verification JSON, and the closed SingleRequestArtifactPlan write. It is not installed; Work, Review/repair, activation, and S12 qualification remain deferred.
• 2026-08-07: Added the private ornith-fast Work stage. It reads PLAN through the closed artifact controller, emits only admitted workspace schemas, bridges exact request/stage/tool results without retaining payloads, and resumes the frozen route with bounded tool evidence. Work rejects reasoning_effort; Review/repair, composite installation, and S12 external qualification remain deferred.