<!-- task=m-node-provider-execution-liveness-recovery/03+02_health_classification plan=0 tag=API milestone-task=health-classification -->

# PLAN — Node Stall Health Classification

## For the Implementing Agent

> **MANDATORY:** Do not begin until the dependency log below exists and is PASS. Implement only this checklist, preserve unrelated user changes, and keep every edit inside the `health-classification` slice. Do not update roadmap state, create follow-up plans, commit, push, or run an official code review. After implementation, fill every implementation-owned section of `CODE_REVIEW-cloud-G08.md` and leave active files in place for the review agent.

## Background

The watchdog predecessor deliberately terminates stalled attempts with `provider_health=unknown`. The approved SDD requires Node to separate request liveness from provider health by probing the exact stalled adapter/target in an independent bounded context. Available, unavailable, unsupported, error, and timeout outcomes must map to a stable three-way classification, and evidence must carry adapter/target plus a monotonic sequence scoped to the current transport connection. Edge binding validation and runtime health overlay remain the next Epic's responsibility.

This slice adds the bounded classifier, connection sequence source, and terminal enrichment for both normalized and raw tunnel paths without changing timer, fence, retry, or Edge projection ownership.

## Dependencies

- `agent-task/m-node-provider-execution-liveness-recovery/02+01_stall_watchdog/complete.log`

At plan creation the predecessor is active. Its own `+01` dependency guarantees that `01_activity_contract` has already passed before this task can start. The implementing runtime must wait for `02+01_stall_watchdog` PASS completion and extend its final APIs; it must not copy anticipated structs from this plan if predecessor review changed names while preserving the contract.

## Analysis

### Files Read

- `AGENTS.md`
- `agent-ops/rules/project/rules.md`
- `agent-ops/rules/common/rules-roadmap.md`
- `agent-ops/rules/common/rules-agent-spec.md`
- `agent-ops/rules/project/domain/node/rules.md`
- `agent-ops/rules/project/domain/platform-common/rules.md`
- `agent-ops/rules/project/domain/testing/rules.md`
- `agent-test/local/rules.md`
- `agent-test/local/node-smoke.md`
- `agent-test/local/platform-common-smoke.md`
- `agent-roadmap/phase/operational-observability-provider-management/milestones/node-provider-execution-liveness-recovery.md`
- `agent-roadmap/sdd/operational-observability-provider-management/node-provider-execution-liveness-recovery/SDD.md`
- `agent-spec/index.md`
- `agent-spec/runtime/edge-node-execution.md`
- `packages/go/execution/types.go`
- `apps/node/internal/node/command_handler.go`
- `apps/node/internal/node/node.go`
- `apps/node/internal/node/run_handler.go`
- `apps/node/internal/node/tunnel_handler.go`
- `apps/node/internal/transport/session.go`
- `apps/node/internal/transport/session_test.go`
- `apps/node/internal/adapters/mock/mock.go`
- `apps/node/internal/adapters/ollama/ollama.go`
- `apps/node/internal/adapters/vllm/provider.go`
- `apps/node/internal/adapters/openai_compat/provider.go`
- `apps/node/internal/adapters/ollama/ollama_test.go`
- `apps/node/internal/adapters/vllm/vllm_test.go`
- `apps/node/internal/adapters/openai_compat/capabilities_test.go`
- `apps/node/internal/node/run_cancel_test.go`
- `apps/node/internal/node/provider_tunnel_test.go`
- `apps/node/internal/node/node_test_support_test.go`
- `agent-contract/inner/execution-runtime.md`
- `agent-contract/inner/edge-node-runtime-wire.md`

### SDD Criteria

- SDD status: approved, D01 resolved, implementation lock released.
- Decision D01: this slice owns Node-side probe classification and evidence generation only; Edge runtime health overlay binding, unhealthy/recovery application, and stale evidence rejection belong to `failure-handoff` (`SDD.md:78-80,125`).
- Scenario: S03 / milestone task `health-classification` (`SDD.md:94`).
- Evidence row: available/unavailable/unsupported/timeout target prober fixtures, request/provider separation, adapter/target/observation sequence, and proof that probe does not reset original progress (`SDD.md:105`).
- Input/output: independent bounded `ProviderProber` context (`SDD.md:72`), three-way provider health and safe metadata (`SDD.md:75-78`).

### Verification Context

- Baseline Node, transport, execution, and race suites passed before plan creation.
- Existing adapters already optionally implement `execution.ProviderProber`; no new network client or provider-specific interface is needed.
- Tests must use injected probe functions/manual clocks and local fixtures only. They must not call real provider endpoints.
- A package-private five-second probe ceiling is an implementation bound, not a new external config surface. Tests inject a shorter/manual bound without sleeping.

### Test Coverage Gaps

- `command_handler.go:63-74` probes capabilities but maps every probe error to unavailable and uses the caller context; that behavior is not safe to reuse for liveness evidence.
- There is no independent probe coordinator or typed request-stalled/provider-unhealthy/health-unknown mapping.
- `Session` has no connection-scoped health observation counter.
- Watchdog terminals carry only the predecessor's unknown fallback and cannot distinguish unavailable from inconclusive probes.
- No test proves that a successful probe does not reset or revive the stalled request.

### Symbol References

- `packages/go/execution/types.go:67-84,142-157` — status normalization and optional target-aware `ProviderProber`.
- `apps/node/internal/node/command_handler.go:49-75` — existing capabilities probe, useful only as an adapter-interface reference.
- `apps/node/internal/node/node.go:18-65` — Node dependencies and test injection point.
- `apps/node/internal/transport/session.go:153-225` — per-connection state/lifetime boundary.
- `apps/node/internal/adapters/mock/mock.go:33-43` — deterministic available probe behavior.
- predecessor `liveness_watchdog.go` — terminal metadata/fence hook to enrich, with timer ownership left intact.

### Split Judgment

- Classification: large. It adds an external provider side effect, bounded temporal state, connection-scoped concurrency, and shared evidence across normalized/tunnel variants.
- Cohesion: probe outcome mapping and observation sequence must be attached atomically to the already-claimed stall terminal; separating them would emit incomplete or reordered evidence.
- Dependency: `02+01_stall_watchdog` completion is mandatory and encoded by `03+02`; its transitive `+01` dependency preserves the activity-contract ordering without adding an undeclared direct dependency.
- Collision check: no other active plan claimed this task id. Overlap with predecessor handler/watchdog files is intentionally serialized by completion dependencies.

### Scope Rationale

- In scope: one independent bounded target probe after stall claim, available/unavailable/unknown mapping, safe adapter/target evidence, connection-scoped monotonic sequence, and normalized/tunnel terminal enrichment.
- Out of scope: Edge registry generation binding, runtime health overlay, provider candidate exclusion/recovery, ingress retry, health recovery polling, metrics/ops evidence, provider adapter behavior changes, and config knobs for probe timeout.
- Probe results never alter the original observer's last-progress time, fence, cancellation result, or terminal count.

### Final Routing

- `evaluation_mode=first-pass`; `finalizer=finalize-task-policy.sh`, `finalizer_mode=pair`.
- Build closure: scope/context/verification/evidence/ownership/decision all closed and trusted; capability gap none.
- Build score: `scope=2`, `state=2`, `blast=1`, `evidence=1`, `verification=2` -> G08; `base_route_basis=local-fit`, `route_basis=risk-boundary`, lane `cloud`, file `PLAN-cloud-G08.md`.
- Build signals: `large_indivisible_context=false`, positive loop risks `temporal_state`, `concurrent_consistency`, `boundary_contract`, `variant_product` (`count=4`), `review_rework_count=0`, `evidence_integrity_failure=false`; risk boundary matched, recovery boundary not matched.
- Review closure: scope/context/verification/evidence/ownership/decision all closed and trusted; capability gap none.
- Review score: `scope=2`, `state=2`, `blast=1`, `evidence=1`, `verification=2` -> G08; `route_basis=official-review`, lane `cloud`, adapter/model `codex/gpt-5.6-sol`, reasoning `xhigh`, file `CODE_REVIEW-cloud-G08.md`.

## Implementation Checklist

- [ ] [API-1] Add an independent bounded target-aware probe classifier with fail-closed unknown semantics.
- [ ] [API-2] Generate connection-scoped monotonic observation sequence and enrich both stall terminal variants atomically.
- [ ] [TEST-1] Verify all probe outcomes, sequence scope, identity evidence, terminal invariants, and no progress reset deterministically.
- [ ] [DOC-1] Update the matching execution spec and execution/Edge-Node wire contracts for Node-produced health evidence and the explicit Edge ownership exclusion.
- [ ] Run every command in Final Verification and record exact results in `CODE_REVIEW-cloud-G08.md`.
- [ ] Fill implementation-owned sections in CODE_REVIEW-*-G??.md with actual implementation notes and verification output.

### [API-1] Classify a bounded probe independently of the stalled request

**Problem**

The existing capabilities command calls `ProviderProber` in its request context and treats errors as unavailable (`apps/node/internal/node/command_handler.go:63-74`). A stalled-request classifier must not inherit canceled request context or promote unsupported/timeouts/errors to provider-wide unhealthy.

**Solution**

Add `apps/node/internal/node/health_probe.go` with a package-private classifier:

- capture the resolved adapter and exact target from the stalled attempt;
- if the adapter does not implement `ProviderProber`, return `provider_health=unknown` and `liveness_classification=health_unknown` without a network call;
- otherwise run `ProbeProvider` in a new context rooted independently from the canceled request, bounded by a package-private `5 * time.Second` ceiling. Inject the context/timeout hook for deterministic tests; do not add a config or protobuf field;
- after the call, check the probe context deadline/cancellation before interpreting the adapter result. Timeout/cancel, returned error, unknown/unrecognized status, or identity-inconclusive result maps to unknown;
- a valid available result maps to `provider_health=available`, `liveness_classification=request_stalled`; a valid unavailable result—including an exact target reported absent/unserved—maps to `provider_health=unavailable`, `liveness_classification=provider_unhealthy`;
- validate/canonicalize returned adapter instance and target against the requested adapter/target. Do not let returned metadata override safe Node-owned identity or copy arbitrary provider detail into terminal metadata;
- the probe result is evidence only. It cannot call the predecessor observer's activity/reset method, change attempt fence, or trigger retry.

Run fence close-wait and health probe concurrently after the stall claim so their independent bounds do not add serial latency; assemble terminal evidence only when both bounded results are known.

**Modified files**

- [ ] `apps/node/internal/node/node.go`
- [ ] `apps/node/internal/node/health_probe.go`
- [ ] `apps/node/internal/node/liveness_watchdog.go`

**Test decision**

Required. Use function adapters for success, absent target, unavailable, unknown, error, unsupported, and timeout. No external endpoints.

**Verification**

- `go test -count=1 ./apps/node/internal/node`
- `go test -race -count=1 ./apps/node/internal/node`

### [API-2] Sequence and attach safe evidence on the current connection

**Problem**

Node evidence must be ordered within a connection, but neither `Node` nor `Session` owns a connection-scoped monotonic counter. A process-global counter cannot give Edge the reset boundary required by the next Epic.

**Solution**

- Add an atomic `uint64` health-observation counter to `transport.Session` and an increment method used only when a stall health observation is finalized. A newly created/reconnected Session starts at zero; the first emitted observation is one. Session close does not reuse the object or counter.
- The watchdog terminal builder requests exactly one sequence per finalized stall, after classification and before the exactly-once terminal send. Normalized and tunnel attempts on the same Session share the counter and therefore cannot duplicate or decrease values under concurrency.
- Attach `health_observation_seq` as base-10 text plus Node-owned `adapter` and `target` to the same metadata map used by the predecessor's normalized Failure/tunnel ERROR. Preserve `run_id`, `attempt_id`, `idle_duration_ms`, `attempt_fence`, stable failure code, and retryable semantics.
- For internal/nil-session calls where no connection boundary exists, omit the sequence instead of inventing a process generation; production listener paths must always supply the current Session.
- Do not add connection generation, provider id binding, stale-sequence rejection, or runtime health mutation. The next Epic binds this evidence to its reception connection and immutable dispatch.

**Modified files**

- [ ] `apps/node/internal/transport/session.go`
- [ ] `apps/node/internal/node/run_handler.go`
- [ ] `apps/node/internal/node/tunnel_handler.go`
- [ ] `apps/node/internal/node/liveness_watchdog.go`

**Test decision**

Required. Cover sequential and concurrent run/tunnel observations on one session, reset on a new session, nil-session omission, and exactly-one increment per terminal.

**Verification**

- `go test -count=1 ./apps/node/internal/transport ./apps/node/internal/node`
- `go test -race -count=3 ./apps/node/internal/transport ./apps/node/internal/node`

### [TEST-1] Prove request/provider separation and evidence invariants

**Problem**

A happy-path available probe alone cannot prove fail-closed classification, target-awareness, sequence ownership, or that probe activity is isolated from the dead attempt.

**Solution**

Add focused tests with the predecessor's fake clock and controlled providers:

- available -> request-stalled; unavailable/network target result and exact target absent -> provider-unhealthy; unsupported, deadline, canceled probe, returned error, unknown status, and identity mismatch -> health-unknown;
- probe receives the stalled adapter/target and a live independent context even though execution context is canceled;
- advancing/completing the probe never resets the original idle timer, suppresses the stall terminal, changes confirmed/unconfirmed fence, or emits a provider progress event;
- normalized and tunnel metadata contain only stable safe keys, the expected identity/classification, and increasing sequence; raw body, reasoning, prompt, credential, provider detail, and `recovery_eligible` are absent;
- concurrent observations on one session are unique/monotonic as a set, and a fresh session begins at one;
- each surface still emits exactly one terminal and late provider emissions remain fenced.

Reuse existing Node transport fixtures; add no adapter implementation changes.

**Modified files**

- [ ] `apps/node/internal/node/health_probe_test.go`
- [ ] `apps/node/internal/node/liveness_watchdog_test.go`
- [ ] `apps/node/internal/transport/session_test.go`

**Test decision**

Required; this is the S03 evidence set.

**Verification**

- `go test -count=10 ./apps/node/internal/node ./apps/node/internal/transport`
- `go test -race -count=3 ./apps/node/internal/node ./apps/node/internal/transport`

### [DOC-1] Document Node evidence and preserve the Edge boundary

**Problem**

After implementation the contracts must distinguish request stall from provider health and state exactly what the connection-scoped sequence does—and does not—authorize.

**Solution**

Update the matching execution spec and both contracts with:

- independent bounded probe input and three-way classification;
- fail-closed unknown cases and exact target/adapter identity rules;
- one connection-scoped monotonically increasing sequence, reset on new Session;
- identical normalized/tunnel metadata and raw/secret exclusions;
- an explicit statement that probe success is not original-request progress and does not imply retry;
- an explicit ownership boundary: Edge reception-generation binding, immutable dispatch validation, stale evidence rejection, runtime health overlay, recovery, and candidate selection remain `failure-handoff`/later work.

Do not document Edge projection as implemented and do not modify roadmap/spec status.

**Modified files**

- [ ] `agent-contract/inner/execution-runtime.md`
- [ ] `agent-contract/inner/edge-node-runtime-wire.md`
- [ ] `agent-spec/runtime/edge-node-execution.md`

**Test decision**

No separate doc test; official review maps statements to the S03 fixtures.

**Verification**

- `git diff --check`

## Modified Files Summary

| Path | Action | Checklist |
|------|--------|-----------|
| `apps/node/internal/node/node.go` | modify | API-1 |
| `apps/node/internal/node/health_probe.go` | add | API-1 |
| `apps/node/internal/node/liveness_watchdog.go` | modify | API-1, API-2 |
| `apps/node/internal/transport/session.go` | modify | API-2 |
| `apps/node/internal/node/run_handler.go` | modify | API-2 |
| `apps/node/internal/node/tunnel_handler.go` | modify | API-2 |
| `apps/node/internal/node/health_probe_test.go` | add | TEST-1 |
| `apps/node/internal/node/liveness_watchdog_test.go` | modify | TEST-1 |
| `apps/node/internal/transport/session_test.go` | modify | TEST-1 |
| `agent-contract/inner/execution-runtime.md` | modify | DOC-1 |
| `agent-contract/inner/edge-node-runtime-wire.md` | modify | DOC-1 |
| `agent-spec/runtime/edge-node-execution.md` | modify | DOC-1 |
| `agent-task/m-node-provider-execution-liveness-recovery/03+02_health_classification/CODE_REVIEW-cloud-G08.md` | update evidence | all |

## Final Verification

1. `go test -count=1 ./packages/go/execution ./apps/node/internal/node ./apps/node/internal/transport`
2. `go test -count=10 ./apps/node/internal/node ./apps/node/internal/transport`
3. `go test -race -count=3 ./packages/go/execution ./apps/node/internal/node ./apps/node/internal/transport`
4. `go test -count=1 ./...`
5. `./scripts/e2e-smoke.sh`
6. `make readability-audit`
7. `git diff --check`

Record exact results in the review stub. External provider smoke is intentionally excluded because deterministic probe fixtures cover the classification contract without requiring credentials or mutating provider state.

**After completing all code changes, fill implementation-owned sections in `CODE_REVIEW-*-G??.md`.**
