<!-- task=m-node-provider-execution-liveness-recovery/01_activity_contract plan=6 tag=REVIEW_REVIEW_REVIEW_REVIEW_API milestone-task=activity-contract -->

# PLAN — Make Queued Timeout Evidence Causal

## For the Implementing Agent

> **MANDATORY:** Implement only this review-derived test and evidence checklist. Preserve unrelated user changes and keep edits inside the `activity-contract` slice. Do not modify production behavior, roadmap state, contracts, specs, schemas, or generated bindings. Run the listed verification, paste literal stdout/stderr into `CODE_REVIEW-cloud-G06.md`, and leave both active files in place for official review. If blocked, record only the exact blocker, attempted commands/output, and resume condition in implementation-owned evidence fields. Do not ask the user, call user-input tools, create control-plane stop files, classify the next state, archive files, or write `complete.log`.

## Background

The provider timeout matrix now covers normalized and tunnel surfaces in immediate and queued modes, but its queued setup releases provider 2 before applying the runtime refresh, allowing synchronous queue pumping against the old store. The tunnel branch also omits the planned tunnel-correlation assertion, and the verification record reconstructs verbose output for non-verbose commands. This follow-up makes the test ordering causal and the evidence literal without changing production behavior.

## Archive Evidence Snapshot

- Current plan after archive: `agent-task/m-node-provider-execution-liveness-recovery/01_activity_contract/plan_cloud_G06_5.log`.
- Current review after archive: `agent-task/m-node-provider-execution-liveness-recovery/01_activity_contract/code_review_cloud_G06_5.log`.
- Verdict: FAIL with 3 Required findings, 0 Suggested findings, and 0 Nit findings.
- Passing reviewer checks: generators, Flutter tests, focused and broad Go tests, race tests, vet, full Go suite, auxiliary E2E, fresh reconnect diagnostic, formatting, and whitespace validation. The current test file has no readability violation.
- Failing reviewer evidence: provider 2 is released before runtime refresh, tunnel wire identity omits `tunnel_id`, and focused/readability outputs are reconstructed rather than literal output from the listed commands.
- Mandatory carryover: refresh while both leases are held, prove the waiter remains pending, release provider 2 only afterward, assert tunnel correlation in both tunnel variants, and record literal terminal output. Preserve `milestone-task=activity-contract`; do not update the roadmap.

## Analysis

### Files Read

- `AGENTS.md`
- `agent-ops/rules/project/rules.md`
- `agent-ops/rules/common/rules-roadmap.md`
- `agent-ops/rules/common/rules-agent-spec.md`
- `agent-ops/rules/project/domain/edge/rules.md`
- `agent-ops/rules/project/domain/testing/rules.md`
- `agent-test/local/rules.md`
- `agent-test/local/edge-smoke.md`
- `agent-roadmap/current.md`
- `agent-roadmap/phase/operational-observability-provider-management/milestones/node-provider-execution-liveness-recovery.md`
- `agent-roadmap/sdd/operational-observability-provider-management/node-provider-execution-liveness-recovery/SDD.md`
- `agent-spec/index.md`
- `agent-spec/runtime/edge-node-execution.md`
- `agent-spec/runtime/provider-pool-config-refresh.md`
- `agent-contract/index.md`
- `agent-contract/inner/execution-runtime.md`
- `agent-contract/inner/edge-node-runtime-wire.md`
- `agent-contract/inner/edge-config-runtime-refresh.md`
- `apps/edge/internal/service/provider_stall_timeout_test.go`
- `apps/edge/internal/service/model_queue_admission.go`
- `apps/edge/internal/service/model_queue_release.go`
- `apps/edge/internal/service/model_queue_types.go`
- `apps/edge/internal/service/queue_reservation.go`
- `apps/edge/internal/service/service.go`
- `apps/edge/internal/service/provider_resolution.go`
- `apps/edge/internal/service/run_submit.go`
- `apps/edge/internal/service/provider_tunnel.go`
- `apps/edge/internal/service/run_types.go`
- `apps/edge/internal/service/provider_pool_admission_test.go`
- `apps/edge/internal/service/run_dispatch_internal_test.go`
- `agent-task/m-node-provider-execution-liveness-recovery/01_activity_contract/PLAN-cloud-G06.md`
- `agent-task/m-node-provider-execution-liveness-recovery/01_activity_contract/CODE_REVIEW-cloud-G06.md`
- `agent-task/m-node-provider-execution-liveness-recovery/01_activity_contract/plan_cloud_G08_4.log`
- `agent-task/m-node-provider-execution-liveness-recovery/01_activity_contract/code_review_cloud_G08_4.log`

### SDD Criteria

- SDD: `agent-roadmap/sdd/operational-observability-provider-management/node-provider-execution-liveness-recovery/SDD.md`; status `[승인됨]`, SDD lock released, no user review.
- First-line scope: `milestone-task=activity-contract`; targeted scenario S01.
- S01 Evidence Map requires config validation and normalized/tunnel activity/deadline/transport evidence. The current slice additionally preserves the selected provider's immutable timeout across queue selection, as required by the Edge-Node wire/config contracts and living specs.
- The checklist therefore keeps both wire surfaces, the final selected-provider identity, queue settlement, tunnel correlation, and literal verification output together. Watchdog, cancellation, retry, health overlay, and roadmap state remain excluded.

### Verification Context

- No separate verification-context handoff was supplied. Repository-native evidence came from the local rules, Edge smoke profile, current plan/review, queue implementation, related tests, and fresh reviewer commands.
- Environment: local checkout `/config/workspace/iop-s1`; Go module `/config/workspace/iop-s1/go.mod`; Go `1.26.2 linux/arm64`; no external provider credentials or remote runner required.
- Fresh reviewer results: focused matrix and ten repetitions passed; broad targeted/race/vet/full suites passed; auxiliary E2E passed; the reconnect diagnostic passed on rerun; `gofmt -l` and `git diff --check` were clean.
- `make readability-audit` exits 2 on retained Edge read-set and central Agent-Ops baseline failures. `build/readability-audit.json` reports `violations: null` for `apps/edge/internal/service/provider_stall_timeout_test.go`; the follow-up must not introduce a current-file violation.
- Exact-output constraint: successful non-verbose `go test` emits package `ok` lines, not `=== RUN`; named subtest evidence must use an explicitly verbose command. Fresh execution is required (`-count=1` or the specified `-count=10`); cached output is not accepted.
- Confidence: high. The queue release path synchronously calls `pumpAllLocked`, so source ordering is sufficient to prove the current test can dispatch before refresh.

### Test Coverage Gaps

- Immediate normalized/tunnel selected-provider identity: covered and passing.
- Queued normalized/tunnel final provider: covered, but the current setup does not causally depend on runtime refresh because provider 2 is released first.
- Tunnel run identity: covered. Tunnel correlation identity: not covered.
- Queue/lease settlement: covered by `assertQueueSettled`.
- Literal command evidence: incomplete because the active review reconstructs output for non-verbose focused commands and abbreviates readability output.

### Symbol References

- None. No production symbol is renamed, removed, or added.

### Split Judgment

- Keep one plan. Refresh-before-release ordering, selected-provider dispatch, tunnel correlation, and the exact verification record form one compact test-evidence invariant in a single test file and its review artifact.

### Scope Rationale

- In scope: reorder the queued test transition, add a pending-state assertion after refresh, assert `ProviderTunnelRequest.tunnel_id`, and replace reconstructed verification evidence with literal command output.
- Out of scope: production queue/service changes, timeout schema/config changes, watchdog/timer/cancellation/retry/health behavior, contracts/specs/roadmap, generated bindings, commit, and push.

### Final Routing

- `status=routed`; `evaluation_mode=isolated-reassessment`; `finalizer=finalize-task-policy.sh`; `finalizer_mode=pair`.
- Build closures: `scope_closed=true`, `context_closed=true`, `verification_closed=true`, `evidence_trusted=true`, `ownership_closed=true`, `decision_closed=true`; no capability gap.
- Build score: `scope=1`, `state=2`, `blast=0`, `evidence=1`, `verification=2` -> G06.
- Build signals: `large_indivisible_context=false`; matched loop risks `temporal_state`, `concurrent_consistency`, `boundary_contract`, `variant_product` (`count=4`); `review_rework_count=4`; `evidence_integrity_failure=true`; risk and recovery boundaries matched.
- Build route: `base_route_basis=local-fit`; `route_basis=recovery-boundary`; lane `cloud`; file `PLAN-cloud-G06.md`.
- Review closures and score match the build packet (`1+2+0+1+2=G06`); route `official-review`, lane `cloud`, adapter/model `codex/gpt-5.6-sol`, reasoning `xhigh`, file `CODE_REVIEW-cloud-G06.md`.

## Implementation Checklist

- [ ] [REVIEW_REVIEW_REVIEW_REVIEW_TEST-1] Make queued refresh ordering causal and assert tunnel correlation identity on both tunnel variants.
- [ ] [REVIEW_REVIEW_REVIEW_REVIEW_VERIFY-1] Run the final verification commands and record literal, non-reconstructed stdout/stderr and exit status.
- [ ] Fill implementation-owned sections in CODE_REVIEW-*-G??.md with actual implementation notes and verification output.

### [REVIEW_REVIEW_REVIEW_REVIEW_TEST-1] Make refresh and tunnel identity independently observable

**Problem**

At `apps/edge/internal/service/provider_stall_timeout_test.go:288-292`, the current queued path is:

```go
requireProviderPoolPending(t, svc.queue, 1)
r2.release("make-prov2-available")

store2 := buildTimeoutMatrixStore(provType, "disabled")
svc.SetRuntimeConfig(store2, catalog, policy)
```

`r2.release` synchronously pumps the queue, so the request can dispatch against the old store before the refresh. At `apps/edge/internal/service/provider_stall_timeout_test.go:335-349`, the tunnel branch asserts `run_id` but not the generated `tunnel_id`.

**Solution**

- Build and apply the store with provider 1 disabled while both synthetic reservations remain held.
- Reassert the provider-pool pending count after refresh to prove refresh alone did not dispatch the waiter.
- Release provider 2 only after that barrier; then require the result to identify provider 2, target 2, timeout 60000, and the expected execution path/queue reason on `RunDispatch` and the real protobuf.
- In the tunnel wire branch, assert `TunnelId == runID + "-tunnel"` before the existing adapter/target/timeout and no-extra-wire assertions.
- Preserve channel/deadline bounds, idempotent close/lease settlement, and the four existing matrix names. Do not add correctness sleeps or production hooks.

Expected ordering:

```go
requireProviderPoolPending(t, svc.queue, 1)
store2 := buildTimeoutMatrixStore(provType, "disabled")
svc.SetRuntimeConfig(store2, catalog, policy)
requireProviderPoolPending(t, svc.queue, 1)
r2.release("make-prov2-available")
```

**Modified Files and Checklist**

- [ ] `apps/edge/internal/service/provider_stall_timeout_test.go` — reorder refresh/release, add the post-refresh pending barrier, and assert tunnel correlation.

**Test Strategy**

- Update `TestProviderPoolResponseStallTimeoutIdentityMatrix`; retain `normalized_immediate`, `normalized_queued`, `tunnel_immediate`, and `tunnel_queued`.
- One verbose fresh run must show every variant. Ten fresh repetitions must pass without a timeout, duplicate wire, or unsettled lease/counter.

**Verification**

- `go test -v -count=1 ./apps/edge/internal/service -run 'ResponseStallTimeout|ProviderStallTimeout'`
- `go test -count=10 ./apps/edge/internal/service -run 'ResponseStallTimeout|ProviderStallTimeout'`

### [REVIEW_REVIEW_REVIEW_REVIEW_VERIFY-1] Preserve literal terminal evidence

**Problem**

`agent-task/m-node-provider-execution-liveness-recovery/01_activity_contract/CODE_REVIEW-cloud-G06.md:176-215` records verbose output under non-verbose commands, and its readability section abbreviates actual output with ellipsis. That evidence cannot be trusted as literal stdout/stderr.

**Solution**

- Run every command in Final Verification after the test change.
- For each command, record the exact command, exit status, and literal stdout/stderr. For an empty successful output, keep an empty fenced block and state only the exit status outside it.
- Use the explicit `-v` focused command for named matrix/subtest inventory. Do not reconstruct timings, compress repetitions into prose inside an output block, or add ellipses.
- Record the reconnect diagnostic's terminal PASS. Record the full readability failure exactly, then use the deterministic `jq` check to prove the changed test file has no violation.

**Modified Files and Checklist**

- [ ] `agent-task/m-node-provider-execution-liveness-recovery/01_activity_contract/CODE_REVIEW-cloud-G06.md` — fill implementation notes and literal verification output only.

**Test Strategy**

- Required as evidence-integrity recovery. The official reviewer must be able to match every claim to the command output and test source without inferring omitted text.

**Verification**

- All commands in Final Verification.

## Modified Files Summary

| Path | Action | Checklist |
|---|---|---|
| `apps/edge/internal/service/provider_stall_timeout_test.go` | modify | REVIEW_REVIEW_REVIEW_REVIEW_TEST-1 |
| `agent-task/m-node-provider-execution-liveness-recovery/01_activity_contract/CODE_REVIEW-cloud-G06.md` | update evidence | REVIEW_REVIEW_REVIEW_REVIEW_VERIFY-1 |

## Final Verification

1. `go version && go env GOMOD`
2. `go test -v -count=1 ./apps/edge/internal/service -run 'ResponseStallTimeout|ProviderStallTimeout'`
3. `go test -count=10 ./apps/edge/internal/service -run 'ResponseStallTimeout|ProviderStallTimeout'`
4. `go test -count=1 ./packages/go/streamgate ./apps/edge/internal/openai ./apps/edge/internal/service ./apps/edge/internal/controlplane`
5. `go test -count=1 ./...`
6. `./scripts/e2e-smoke.sh`
7. `IOP_DEV_RECONNECT_BIND_TIMEOUT=45 ./scripts/dev/edge-node-reconnect-diagnostic.sh`
8. `make readability-audit`
9. `jq -e '.files[] | select(.path == "apps/edge/internal/service/provider_stall_timeout_test.go") | .violations == null' build/readability-audit.json`
10. `gofmt -l packages/go/execution packages/go/config apps/edge/internal/service apps/node/internal/node`
11. `git diff --check`
12. `git diff --stat`

Expected results: commands 1-7 and 9-12 exit 0; command 8 may retain the exact pre-existing ratchet failure only, while command 9 must return `true`. The focused verbose run must name all four matrix variants, the ten-run command must pass, the reconnect diagnostic must reach its terminal PASS, formatting output must be empty, and no current test-file readability violation or whitespace error may appear.

**After completing all code changes, fill implementation-owned sections in `CODE_REVIEW-*-G??.md`.**
