<!-- task=m-node-provider-execution-liveness-recovery/10+09_stall_recovery plan=10 tag=REVIEW_API milestone-task=bounded-retry -->

# Complete Responses Cross-Path Recovery Request Assertions

## For the Implementing Agent

Implement only the test assertions below. Run every verification command exactly as written, fill the implementation-owned sections of `CODE_REVIEW-cloud-G03.md` with actual notes and complete raw output, keep the active pair in place, and report ready for review. Finalization belongs to the code-review skill. If blocked, record the exact blocker, attempted commands/output, and resume condition only in implementation-owned evidence fields. Do not ask the user, call user-input tools, create control-plane stop files, classify the next state, archive logs, or write `complete.log`.

## Background

The recovery overlay now preserves the normalized Responses execution fields and all declared commands pass. The path-switch matrix still proves the selected response transport with scripted frames rather than proving the replacement request sent through that transport. The follow-up closes only that deterministic evidence gap; production code is already correct and remains outside the write boundary.

## Archive Evidence Snapshot

- The reviewed plan=9 pair is archived in this task directory as `plan_cloud_G06_9.log` and `code_review_cloud_G06_9.log` with verdict `FAIL`.
- Required R1: the `normalized_to_provider_tunnel` rows do not inspect the recorded tunnel request, while normalized replacements assert only `TimeoutSec`; scripted success frames are independent of request body and metadata.
- Fresh reviewer reruns passed all twelve declared commands, and source review confirmed that recovery `PrepareRun` overlays `TimeoutSec`, `MaxQueue`, and `QueueTimeoutMS` correctly.
- Routing signals are `review_rework_count=7` and `evidence_integrity_failure=false`.

## Finding Resolution Map

| Finding | Mode | Exact Fix / Evidence | Changed or Satisfied Precondition |
|---------|------|----------------------|-----------------------------------|
| Required R1 | `direct-fix` | In `apps/edge/internal/openai/stream_gate_stall_recovery_test.go`, inspect the recorded attempt-B request for both Responses path-switch directions: normalized prompt/input/metadata/execution values and tunnel timeout/stream/metadata/target-rewritten body. | The cross-path rows become sensitive to request-context loss instead of succeeding from pre-scripted response frames alone. |

## Analysis

### Files Read

- `apps/edge/internal/openai/stream_gate_stall_recovery_test.go`
- `apps/edge/internal/openai/provider_test_support_test.go`
- `apps/edge/internal/openai/responses_stream_gate.go`
- `apps/edge/internal/openai/responses_handler.go`
- `apps/edge/internal/openai/dispatch_context.go`
- `apps/edge/internal/openai/route_resolution.go`
- `apps/edge/internal/service/provider_pool.go`
- `packages/go/config/edge_types.go`
- `agent-contract/inner/execution-runtime.md`
- `agent-contract/inner/edge-config-runtime-refresh.md`
- `agent-contract/inner/edge-node-runtime-wire.md`
- `agent-contract/outer/openai-compatible-api.md`
- `agent-spec/runtime/stream-evidence-gate.md`
- `agent-spec/input/openai-compatible-surface.md`
- `agent-spec/runtime/edge-node-execution.md`
- `agent-spec/runtime/provider-pool-config-refresh.md`
- `agent-roadmap/phase/operational-observability-provider-management/PHASE.md`
- `agent-roadmap/phase/operational-observability-provider-management/milestones/node-provider-execution-liveness-recovery.md`
- `agent-roadmap/sdd/operational-observability-provider-management/node-provider-execution-liveness-recovery/SDD.md`
- `agent-test/local/rules.md`
- `agent-test/local/edge-smoke.md`
- `agent-test/local/platform-common-smoke.md`
- `agent-task/archive/2026/08/m-node-provider-execution-liveness-recovery/09+08_retry_candidate_policy/complete.log`
- `agent-task/m-node-provider-execution-liveness-recovery/10+09_stall_recovery/plan_cloud_G06_9.log`
- `agent-task/m-node-provider-execution-liveness-recovery/10+09_stall_recovery/code_review_cloud_G06_9.log`

### SDD Criteria

- SDD: `agent-roadmap/sdd/operational-observability-provider-management/node-provider-execution-liveness-recovery/SDD.md`; approved, lock released, and no user-review gate is active.
- Contribution: `milestone-task=bounded-retry`; targeted Acceptance Scenario S05.
- S05 requires confirmed, transport-uncommitted, uncancelled, side-effect-safe recovery within the shared fault budget, with a new run identity and bounded dispatch. Its Evidence Map requires commit-boundary/shared-budget, provider-pool failover, no-owner terminal, new run identity, and bounded-dispatch evidence.
- The implementation checklist preserves the existing S05 lifecycle checks and adds the missing selected-request assertions at the production handler/admission seam.

### Verification Context

- No separate verification handoff was supplied. Repository-native evidence comes from the current checkout, the archived plan=9 review, `agent-test/local/rules.md`, `agent-test/local/edge-smoke.md`, and `agent-test/local/platform-common-smoke.md`.
- Verification runs in `/config/workspace/iop-s1`. The focused and full matrix commands use fresh iterations; package, race, vet, deterministic Edge smoke, fake-vLLM full-cycle smoke, provider-capacity smoke, and whitespace checks remain the final regression set.
- The fake-vLLM and capacity profiles need no user-controlled credential, device, remote runner, or live provider. No external verification preflight is required.
- Current precondition: the production overlay at `responses_stream_gate.go:1079-1086` is correct and all twelve reviewer commands pass.
- Constraint: use the existing `scriptedPoolRunService.snapshot` request records; do not change production behavior or make scripted response frames depend on a new fake transport.
- Gap: the current matrix discards `tunnelRequests`, and its normalized request assertion covers only timeout.
- Confidence: high. The missing evidence is local to one table-driven production-handler test and the test double already records both request types.

### Test Coverage Gaps

- Tunnel-to-normalized: path selection, provider avoidance, dispatch count, closes, output sanitation, terminal count, and timeout are covered; normalized prompt/input/metadata and the remaining execution fields are not asserted together.
- Normalized-to-tunnel: path selection and lifecycle are covered; the replacement tunnel request's timeout, stream flag, metadata, and target-rewritten body are not inspected.
- Same-path, safety-guard, shared-budget, compatibility, race, vet, and smoke coverage already passes and requires no new fixture.

### Symbol References

- None. No symbol is renamed or removed.

### Split Judgment

- Keep one plan. One compact table-driven assertion block must compare the request selected for attempt B with the direction encoded by each row; splitting it would duplicate the same fixture and verification.
- The dependent subtask `10+09_stall_recovery` requires predecessor index 09. It is satisfied by `agent-task/archive/2026/08/m-node-provider-execution-liveness-recovery/09+08_retry_candidate_policy/complete.log`.

### Scope Rationale

- Exclude `apps/edge/internal/openai/responses_stream_gate.go`: fresh source review and all reviewer commands confirm the three-field overlay is correct.
- Exclude `apps/edge/internal/openai/provider_test_support_test.go`: the existing snapshot already returns recorded run and tunnel requests.
- Exclude service, contract, spec, config, proto, and smoke-script edits because no runtime meaning changes.
- Exclude new standalone tests; the existing matrix is the required production-handler regression surface.

### Final Routing

- `evaluation_mode=isolated-reassessment`; `finalizer=finalize-task-policy.sh`; `finalizer_mode=pair`; no capability gap.
- Build closures are all closed. Grade scores are scope=0, state=1, blast=0, evidence=1, verification=1, producing G03. Base route basis is `local-fit`; final route basis is `recovery-boundary`; lane `cloud`; filename `PLAN-cloud-G03.md`.
- Build signals: `large_indivisible_context=false`; matched loop risks are `temporal_state`, `boundary_contract`, and `variant_product`; count=3; `review_rework_count=7`; `evidence_integrity_failure=false`; risk boundary not matched; recovery boundary matched.
- Review closures are all closed. Grade scores are scope=0, state=1, blast=0, evidence=1, verification=1; route basis `official-review`; lane `cloud`; grade `G03`; filename `CODE_REVIEW-cloud-G03.md`; adapter `codex`; model `gpt-5.6-sol`; reasoning effort `xhigh`.

## Implementation Checklist

- [ ] REVIEW_API-1 makes both semantic-false and semantic-true Responses cross-path rows inspect the actual attempt-B request, proving normalized prompt/input/metadata/execution values and tunnel timeout/stream/metadata/target-rewritten body while retaining provider avoidance, distinct identities, bounded dispatch, exactly-once closes, sanitized output, and one public terminal.
- [ ] Run and record every exact final verification command separately after REVIEW_API-1 is complete.
- [ ] Fill implementation-owned sections in CODE_REVIEW-*-G??.md with actual implementation notes and verification output.

### [REVIEW_API-1] Prove both selected replacement request contexts

**Problem:** `apps/edge/internal/openai/stream_gate_stall_recovery_test.go:308` discards the recorded tunnel requests. Lines 312-320 inspect only `TimeoutSec` for normalized replacements, so the new reverse path-switch rows at lines 270-271 can pass from scripted frames even when attempt B receives a wrong tunnel body or metadata.

**Solution:** Capture both request slices from `scriptedPoolRunService.snapshot`. Branch on `tc.replacementPath`: for normalized attempt B, assert the last run request's prompt, input prompt, model/stream metadata, timeout, queue fields, estimate, and context class; for tunnel attempt B, assert timeout, stream flag, model/stream metadata, estimate/context class, and the body returned by `BuildBody("served-b")` contains the rewritten target plus the original Responses input/stream values. Keep all current lifecycle and public-output assertions.

Before:

```go
// stream_gate_stall_recovery_test.go:308
pools, cancels, _, _, runRequests, _ := service.snapshot()
if pools != 2 || len(cancels) != 0 {
    t.Fatalf("dispatch/cancel lifecycle=(%d,%v), want (2,none)", pools, cancels)
}
if tc.replacementPath == normPath {
    if len(runRequests) == 0 {
        t.Fatalf("expected at least one normalized run request, got 0")
    }
    replacementRun := runRequests[len(runRequests)-1]
    if replacementRun.TimeoutSec != 5 {
        t.Fatalf("normalized replacement TimeoutSec = %d, want ingress timeout 5", replacementRun.TimeoutSec)
    }
}
```

After:

```go
pools, cancels, _, _, runRequests, tunnelRequests := service.snapshot()
// Keep the existing lifecycle assertions, then inspect the actual request for
// attempt B according to tc.replacementPath. Rebuild the tunnel body with
// "served-b" and decode/assert its model, input, and stream values.
```

**Modified Files and Checklist:**

- [ ] `apps/edge/internal/openai/stream_gate_stall_recovery_test.go`: add direction-specific attempt-B request assertions to the existing matrix.
- [ ] `agent-task/m-node-provider-execution-liveness-recovery/10+09_stall_recovery/CODE_REVIEW-cloud-G03.md`: record actual decisions and complete raw verification output.

**Test Strategy:** Extend `TestOpenAIStallRecoveryMatrix`; do not create another test or change the scripted service. The focused command runs both Responses path-switch directions for semantic false and true ten times, and the full matrix protects every existing lifecycle product.

**Verification:** `go test -count=10 ./apps/edge/internal/openai -run 'TestOpenAIStallRecoveryMatrix/recover/responses/(provider_tunnel_to_normalized|normalized_to_provider_tunnel)'` must pass all four cross-path rows with the new request assertions.

## Dependencies and Execution Order

1. Predecessor `09+08_retry_candidate_policy` is satisfied by `agent-task/archive/2026/08/m-node-provider-execution-liveness-recovery/09+08_retry_candidate_policy/complete.log`.
2. Complete REVIEW_API-1, then run every final verification command and fill the active review evidence.

## Modified Files Summary

| File | Item |
|------|------|
| `apps/edge/internal/openai/stream_gate_stall_recovery_test.go` | REVIEW_API-1 |
| `agent-task/m-node-provider-execution-liveness-recovery/10+09_stall_recovery/CODE_REVIEW-cloud-G03.md` | REVIEW_API-1 |

## Final Verification

1. `go test -count=10 ./apps/edge/internal/openai -run 'TestOpenAIStallRecoveryMatrix/recover/responses/(provider_tunnel_to_normalized|normalized_to_provider_tunnel)'` — PASS both directions in both semantic modes and prove the selected attempt-B request context.
2. `go test -count=10 ./apps/edge/internal/openai -run '^TestOpenAIStallRecoveryMatrix$'` — PASS every same-path, cross-path, safety-guard, shared-budget, close, and terminal row.
3. `go test -count=10 ./apps/edge/internal/openai -run '^TestOpenAISemanticGateDisabledCompatibility$'` — PASS endpoint-native compatibility rows.
4. `go test -count=20 ./apps/edge/internal/openai -run '^(TestOpenAIStallRecoveryFilter|TestOpenAIAttemptControllerConfirmedStall|TestOpenAIAttemptDispatcherStalledProvider)$'` — PASS every iteration.
5. `go test -count=1 ./packages/go/streamgate ./apps/edge/internal/openai ./apps/edge/internal/service ./apps/edge/internal/controlplane` — PASS.
6. `go test -race -count=3 ./apps/edge/internal/service` — PASS with no race report.
7. `go test -race -count=3 ./apps/edge/internal/openai` — PASS with no race report.
8. `go vet ./packages/go/streamgate ./apps/edge/internal/service ./apps/edge/internal/openai ./apps/edge/internal/controlplane` — no diagnostics.
9. `./scripts/e2e-smoke.sh` — PASS.
10. `IOP_VLLM_MODE=fake ./scripts/e2e-openai-vllm.sh` — PASS.
11. `./scripts/e2e-provider-capacity-smoke.sh` — PASS.
12. `git diff --check` — no whitespace errors.

After completing all code changes, fill implementation-owned sections in `CODE_REVIEW-*-G??.md`.
