iop/agent-contract/inner/edge-node-runtime-wire.md

Edge-Node Runtime Wire Contract

계약 메타

• id: iop.edge-node-runtime-wire
• boundary: inner
• status: active
• 원본 경로:
  • proto/iop/runtime.proto
  • apps/edge/internal/transport/server.go
  • apps/node/internal/transport/client.go
  • apps/node/internal/transport/session.go
  • apps/node/internal/transport/parser.go
  • apps/node/internal/bootstrap/runtime_supervisor.go
  • apps/node/internal/node/tunnel_handler.go
  • apps/node/internal/node/runtime_bridge.go
  • packages/go/credentiallease/envelope.go
  • apps/edge/internal/transport/connection_handlers.go
  • apps/edge/internal/service/model_queue_release.go
  • apps/edge/internal/service/model_queue_snapshot.go
  • apps/edge/internal/service/node_command.go
  • apps/node/internal/node/command_handler.go
  • apps/edge/internal/service/status_provider.go
  • apps/edge/internal/node/mapper.go
  • apps/node/internal/adapters/config_set.go
• human docs:
  • apps/edge/README.md
  • apps/node/README.md

읽는 조건

• Edge-Node TLS/protobuf transport, workload identity, initial/reconnect supervision, register/dispatch-ready handshake, connection generation fencing, run stream, provider raw tunnel, credential lease consumption, cancel, node command, node config refresh를 바꿀 때
• NodeReadyRequest, NodeReadyResponse, RunRequest, RunEvent, ProviderTunnelRequest, ProviderTunnelFrame, CancelRequest, NodeCommandRequest, NodeCommandResponse, NodeConfigPayload, NodeConfigRefresh* 필드를 바꿀 때
• node adapter 설정 payload나 runtime config가 Edge에서 Node로 전달되는 방식을 바꿀 때

범위

이 계약은 Edge와 Node 사이의 내부 TCP proto-socket 경계다. Edge는 Node 연결을 수락하고, Node는 연결 직후 등록 요청을 보낸다. 실행 요청과 이벤트 스트림, 취소, 조회성 명령, 설정 refresh는 같은 내부 wire 계열에서 처리한다.

주요 흐름

• register와 readiness: 수락된 하나의 TCP 연결(TcpClient)은 정확히 하나의 Node ID만 소유한다. 동일한 연결로 두 번째 Node ID 등록을 시도하면 첫 번째 binding과 generation을 바꾸지 않고 거부된다. Node가 RegisterRequest를 보내고 Edge가 RegisterResponse로 수락 여부와 NodeConfigPayload를 돌려준다. accepted registration은 Node ID의 현재 ownership을 pending으로 claim할 뿐 dispatch 가능 상태가 아니다. Node는 config 적용, adapter start, session handler 설치 뒤 NodeReadyRequest(node_id)를 보내고, Edge가 current owner를 dispatch-ready로 전환한 뒤 NodeReadyResponse로 ack한다. 이 ready ack 전에는 run, provider tunnel, command, config-refresh push와 connected availability/event가 열리지 않는다.
• connectivity supervision: Node daemon은 Fx startup 전에 원격 연결 성공을 요구하지 않고 단일 supervisor goroutine이 initial dial과 established-session reconnect를 같은 policy로 직렬 처리한다. retryable 원격 실패는 재시도하고 local config/credential fatal error, 유한 retry exhaustion, local shutdown만 process terminal로 구분한다.
• disconnect/reconnect: current dispatch-ready owner의 close/heartbeat timeout만 해당 connection generation을 fence한다. Edge는 같은 authoritative lifecycle에서 provider lease를 정확히 한 번 반환하고 resource를 offline/excluded로 만든 뒤 queue를 live candidate 기준으로 재평가한다. accepted Node의 ready transition은 새 generation resource를 활성화하고 기존 waiter를 즉시 pump한다. stale/rejected connection callback은 live state나 lifecycle event를 바꾸지 않는다.
• execution: Edge가 RunRequest를 보내고 Node가 RunEvent stream으로 실행 상태를 보낸다.
• provider raw tunnel: Edge가 기존 Edge-Node socket으로 ProviderTunnelRequest를 보내고 Node가 provider HTTP/SSE 요청을 연 뒤 ProviderTunnelFrame stream으로 provider status/header/body/end/error/usage 후보를 sequence와 함께 돌려준다. 이 경로는 OpenAI-compatible provider passthrough용이며 RunEvent 실행 stream과 분리된다.
• response_stall_timeout_ms: RunRequest.response_stall_timeout_ms와 ProviderTunnelRequest.response_stall_timeout_ms는 int64 필드로, 선택된 provider의 response-stall timeout을 밀리초 단위로 운반한다. Zero는 Node가 문서화된 기본값(300000ms)을 적용함을 의미한다. Negative 또는 overflow 값은 Node 경계에서 router/provider 호출 전에 reject된다. Edge provider-pool dispatch는 winning candidate의 effective timeout을 각 요청에 복사한다. Direct/non-pool 호출은 wire에서 zero를 사용하고 Node 기본값을 적용한다.
• response stall terminal: Node observes only the execution activity contract. On expiry it cancels and fences the local provider attempt, joins the bounded close-grace fence and an independent exact-target health probe without extending either serially, then emits exactly one normalized RunEvent{type=error} or tunnel ProviderTunnelFrame{kind=ERROR} with failure_code=response_stalled and populates the optional wire ExecutionFailure field (field 13 on RunEvent, field 15 on ProviderTunnelFrame). Terminal metadata is allowlisted (three-way health evidence as the provider_health status paired with the liveness_classification normalization — available/request_stalled, unavailable/provider_unhealthy, or unknown/health_unknown; idle duration; Node-owned run/attempt identity; fence; adapter; target; and an optional connection-scoped health_observation_seq); it contains no caller-controlled identity, raw payload, credential, or recovery_eligible. Nil and non-stalled failures leave wire ExecutionFailure absent while preserving legacy error string fields (RunEvent.Error / ProviderTunnelFrame.Error). health_observation_seq starts at one per connection and increases uniquely across the connection's normalized and tunnel observations; an unbound session omits it. Probe availability is evidence only and never resets progress, changes the fence, or authorizes retry. A confirmed fence is a capability hint only, not Node retry authorization.
• Edge terminal handoff: transport reception identity, not payload identity, supplies (node_id, connection_generation). Before a normalized or tunnel terminal can affect provider health, Edge compares that identity and the typed adapter/target evidence with the tracked immutable provider lease. A current terminal releases that lease exactly once even when optional health evidence is rejected. Edge adds provider_id, validated provider_health, and recovery_handoff=confirmed to every validated current bound stall before downstream routing, including sequence-stale request-local handoff; only a fresh unavailable observation lowers the separate runtime overlay. The handoff token is not replay approval, and Edge never adds recovery_eligible here.
• CAPABILITIES recovery probe: Node resolves the requested adapter instance, runs the bounded fail-closed exact-target ProbeHealth, and returns stable adapter_key, target, normalized provider_status, and the next Session-owned health_observation_seq. Edge retains the command's dispatch node/generation and may clear one unavailable overlay only when a higher-sequence available response identifies exactly one same-generation provider binding. Empty, malformed, ambiguous, mismatched, stale, unknown, and unavailable results do not change the overlay.
• precedence and ownership: request hard deadline, caller cancellation, and session disconnect retain their existing boundary when they win before the watchdog. A session lifetime context cancels active run and tunnel handlers on disconnect. If provider return is not confirmed during the bounded close grace, Node emits and fences the terminal but retains admission, run-manager, credential, and adapter ownership until the provider actually returns.
• managed credential delivery: after provider selection, Edge attaches an exact CredentialLeaseBinding and a short-lived signed lease sealed to the selected Node. The Node opens it only after adapter-capacity admission and immediately before provider execution, verifies signature, recipient, scope, expiry, and replay state, injects the declared auth header in memory, then zeroes plaintext material.
• provider-pool mixed dispatch: Edge service는 model group provider candidate를 선택한 뒤, 같은 selected provider/queue lease로 OpenAI-compatible provider에는 ProviderTunnelRequest, Ollama/native provider에는 normalized RunRequest를 보낸다. Edge-Node wire는 client-provided response path selector를 받지 않고, provider type만으로 후보를 제외하지 않는다.
• cancel: Edge가 provider run id를 가진 CancelRequest를 보내 현재 provider 실행을 취소한다.
• command: Edge가 NodeCommandRequest를 보내고 Node가 NodeCommandResponse로 capabilities/transport/provider lifecycle 상태를 응답한다.
• refresh: Edge가 NodeConfigRefreshRequest로 새 config payload를 보내고 Node가 NodeConfigRefreshResponse로 적용/재시작 필요/실패를 응답한다.

필드 의미

• RunRequest.adapter, RunRequest.target: 내부 실행 식별자다. 외부 OpenAI-compatible model은 Edge 입력 표면에서 이 둘로 변환되어야 한다.
• RunRequest.input: adapter가 해석할 normalized provider 실행 입력이다.
• RunRequest.metadata: caller-defined 실행 metadata다. 실행 디렉터리나 session 소유권을 전달하는 제어 표면이 아니다.
• RunEvent.type: start, delta, complete, error, cancelled 같은 실행 이벤트 종류다.
• ProviderTunnelRequest is the protobuf request for opening a provider HTTP request over the existing Edge-Node socket. It carries adapter, target, method, path, headers, final serialized body, stream, timeout_sec, metadata, session_id, and operation, separately from normalized RunRequest execution.
• ProviderTunnelRequest.operation is protobuf field 13. It identifies a named profile operation (models, chat_completions, messages, count_tokens, or responses); when it is empty, path remains the mixed-version fallback.
• ProviderTunnelRequest.credential_lease and .credential_binding are an inseparable managed pair. A configured managed Node rejects a tunnel when either is absent, invalid, expired, replayed, or scope-mismatched. Legacy requests omit both only when the credential plane is disabled.
• CredentialLeaseBinding independently names the principal, slot, route, profile, upstream target, selected Node and recipient, credential/route revisions, and projection generation. It is not caller metadata and cannot be rewritten by the Node.
• Edge and Node validate each other with CA-signed workload identities. Managed mode requires Edge server identity plus expected Node role/name, and Node client identity plus expected Edge role/name; no-cert, wrong-CA, wrong-role, and wrong-name peers fail before registration or lease delivery.
• SubmitProviderTunnelRequest.BuildBody is Edge-local only. After provider-pool selection determines the served target, Edge invokes it and serializes its returned bytes into protobuf ProviderTunnelRequest.body. It is not a protobuf field.
• The resolved ConcreteProtocolProfile travels in nested OpenAICompatAdapterConfig.protocol_profile inside the Node configuration payload. Tunnel requests carry the selected operation and bytes, not profile configuration.
• ProviderTunnelFrame is the ordered response frame. body is the passthrough source of truth and is not sent through RunEvent.delta or the Edge event bus; usage and metadata are observation candidates and are never merged into the body. RESPONSE_START occurs at most once, BODY occurs zero or more times, and exactly one terminal END or ERROR occurs. USAGE is observation-only.
• tunnel cancellation: HTTP caller disconnect, response wait timeout, 또는 Edge write failure가 발생하면 Edge는 같은 run id에 대한 CancelRequest를 보내 upstream provider request 중단을 요청한다. Node adapter는 provider request context cancellation을 관측하고 ordered error/end semantics를 유지해야 한다.
• RunEvent.metadata["openai_tool_calls"]: OpenAI-compatible provider adapter가 native tool_calls를 반환했을 때 완료 이벤트에 싣는 JSON 배열이다. Edge OpenAI-compatible 표면은 이 값을 message.tool_calls 또는 stream delta.tool_calls로 복원한다. provider assistant content 텍스트를 이 값으로 파싱/합성하지 않는다.
• RunEvent.metadata["openai_text_tool_fallback"]: OpenAI-compatible provider adapter가 backend native tool API 거부 후 tools/tool_choice를 제거하고 text tool-call instruction으로 재시도했을 때 "true"를 싣는다. 이 instruction은 backend가 system role 위치를 거부하지 않도록 leading system message에 병합한다. Edge는 이 표시가 있는 실행에서만 assistant content의 text tool-call을 OpenAI-compatible tool_calls로 복원할 수 있다.
• NodeCommandRequest.type: 실행이 아닌 조회/제어성 명령이다. adapter execution 요청과 섞지 않는다.
• NodeCommandResponse.result for CAPABILITIES uses adapter_key, target, provider_status, and health_observation_seq as the stable recovery-evidence keys. adapter and instance_key remain diagnostic capability identity; arbitrary provider metadata is not accepted as recovery evidence.
• NodeConfigPayload.adapters: Edge가 Node에 내려주는 adapter instance 설정이다.
• NodeReadyRequest.node_id: RegisterResponse가 돌려준 Node identity다. Edge registry의 internal connection generation은 이 wire/config field로 노출하지 않으며, Edge는 (node_id, current client) ownership 비교로 stale ready를 거부한다.
• NodeReadyResponse.ready: current pending owner의 첫 ready transition과 이미 ready인 같은 owner의 duplicate ready에서 true다. 첫 transition만 provider resource activation, stranded provider-pool waiter pump, node.connected event를 만든다. stale/superseded/rejected connection은 false와 reason을 받고 session을 닫아 reconnect해야 한다.
• AdapterConfig.name: node 내부 stable adapter instance identity다. 비어 있으면 legacy single-instance type 이름과 동등하다.
• NodeRuntimeConfig.concurrency: legacy compatibility runtime metadata다. 실행 admission은 이 값을 node-wide global gate로 사용하지 않고 provider/resource capacity를 기준으로 한다. Node store 위치나 실행 작업 디렉터리는 이 runtime payload에 싣지 않는다.
• reconnect.interval_sec, reconnect.max_attempts: initial connect와 established-session reconnect에 공통 적용된다. 명시적 max_attempts=0은 local shutdown까지 unlimited, 생략은 기본값 10, 양수는 정확한 유한 attempt limit, 음수는 validation error다. unlimited mode의 interval_sec는 양수여야 하며 생략은 기본값 10을 사용한다. 유한 exhaustion과 non-retryable 오류는 exit code 1, local shutdown은 정상 종료다.
• ProviderSnapshot: legacy wire name을 유지하지만 Node 아래 resource/provider 상태 snapshot으로 해석한다. category가 api, local_inference resource kind를 나타내며, provider-pool dispatch 대상은 Edge config models[].providers가 참조한 resource뿐이다. in_flight와 long_in_flight는 node_id + provider_id lease state의 현재 점유다. queued는 Edge queue에서 해당 provider를 live candidate로 포함하는 고유 pending request 수이고 long_queued는 그중 long request 수이므로 여러 provider snapshot에 같은 request가 candidate pressure로 나타날 수 있다.
• A current runtime-unavailable overlay preserves ProviderSnapshot catalog identity but projects status=unavailable, health=unavailable, and all effective capacity/load/counter fields as zero. The configured provider health is not rewritten. A newer connection generation does not inherit the old overlay.
• configured Node가 disconnected/pending이면 Node snapshot은 connected=false를 유지하고 provider catalog entry도 남는다. enabled provider의 effective snapshot은 status=unavailable, health=offline, capacity/in-flight/queued/long-context 관련 수치가 모두 0이다. reconnect ready 뒤에는 같은 resource identity의 새 generation으로 configured capacity와 admission eligibility가 복구된다.
• Node adapter instance는 normalized RunRequest와 ProviderTunnelRequest가 공유하는 local capacity gate를 사용한다. 이 gate는 Edge provider lease를 복제하는 분산 admission이 아니라 Edge queue를 우회한 실행으로부터 같은 backend를 보호하는 defense-in-depth다.

금지 사항

• Control Plane이나 Client가 Node에 직접 연결하거나 직접 스케줄링하는 계약을 만들지 않는다.
• 외부 API의 model 용어를 이 내부 경계의 대표 실행 식별자로 되살리지 않는다.
• proto/gen/iop/*.pb.go 생성물을 직접 수정하지 않는다.
• transport handler에서 console/HTTP/A2A 표면 응답을 직접 만들지 않는다. 표면별 변환은 Edge service/input 계층에 둔다.
• Node address, token, transport internals를 Control Plane status 계약으로 노출하지 않는다.
• accepted registration만으로 Node를 dispatch candidate, connected snapshot/event 또는 config refresh recipient로 취급하지 않는다.
• provider lease 반환, generation fencing, queue settlement 같은 correctness 전이를 drop 가능한 node event fanout의 성공에 의존시키지 않는다.
• OS service/Task Scheduler restart를 retryable initial connect 또는 장기 outage 복구의 correctness owner로 사용하지 않는다.
• Do not send provider plaintext, at-rest ciphertext, the recipient private key, or the issuer private key in NodeConfigPayload, logs, metrics, events, or tunnel metadata.
• Do not open a lease before adapter capacity admission, cache plaintext across requests, accept a lease for another Node/target/revision/generation, or fall back to a different same-model credential slot after a bound route fails.

운영 증거 사영 경계

Node stall, Edge provider-health overlay, and Edge OpenAI recovery operational projections are local observations derived from the established terminal, health-overlay, and recovery decisions. They introduce no new Node↔Edge frame, field, ordering rule, or retry semantic. The wire protocol remains unchanged by these projections.

• Node observes finalized stall evidence before constructing and delivering the terminal; recovered observer failure cannot suppress terminal delivery.
• Edge emits iop_edge_provider_health_evidence_total, iop_edge_provider_health_transitions_total, and edge_provider_health_observation locally after the overlay decision is finalized.
• Edge emits iop_edge_liveness_recovery_eligibility_total, iop_edge_liveness_recovery_results_total, and edge_liveness_recovery_observation locally per request lifecycle.

Operational projections exclude raw payloads, credentials, caller-controlled identities, and unbounded identifiers from metric labels and general logs. Valid typed terminal metadata (e.g. run_id, adapter, target on the allowlisted stall metadata map) remains on the wire as already required by the typed terminal contract.

변경 시 확인할 코드/테스트

• proto/iop/runtime.proto
• apps/edge/internal/transport/*_test.go
• apps/node/internal/transport/*_test.go
• apps/edge/internal/bootstrap/reconnect_readiness_integration_test.go
• apps/edge/internal/openai/provider_dispatch_test.go
• apps/edge/internal/openai/provider_selection_test.go
• apps/edge/internal/openai/provider_tunnel_test.go
• apps/edge/internal/openai/cancellation_routes_test.go
• apps/node/internal/adapters/openai_compat/*_test.go
• apps/node/internal/adapters/vllm/*_test.go
• apps/edge/internal/node/mapper_test.go
• apps/node/internal/adapters/config_set_test.go
• apps/node/internal/adapters/adapters_blackbox_test.go
• apps/node/internal/node/provider_tunnel_credential_test.go
• packages/go/credentiallease/envelope_test.go
• proto 변경 시 make proto, Client가 소비하면 make proto-dart