<!-- task=m-node-provider-execution-liveness-recovery/10+09_stall_recovery plan=4 tag=REVIEW_API milestone-task=bounded-retry -->

# Always-On OpenAI Stall Recovery Ownership

## For the Implementing Agent

Implement only the two direct fixes below, run every verification command exactly as written, and fill all implementation-owned sections of `CODE_REVIEW-cloud-G08.md` with actual notes and complete raw output. Preserve the already implemented raw-free typed mapper, confirmed-transport close, provider avoidance, and shared StreamGate recovery budget. Keep active files in place and report ready for review; finalization belongs to the code-review skill.

## Background

The first implementation added typed `response_stalled` mapping and a private request-local recovery filter, but the filter exists only inside the StreamGate runtime and every supported Chat/Responses entry point still selects that runtime through `streamGateEnabled()`, which returns the semantic `openai.stream_evidence_gate.enabled` flag. Because that flag defaults to false, the default supported OpenAI paths have no liveness recovery owner. The implementation also supplied only mapper/filter unit tests instead of the S05 endpoint/path/config product matrix and recorded incomplete race output.

## Archive Evidence Snapshot

- The reviewed plan=3 pair is archived in this task directory as `plan_cloud_G08_3.log` and `code_review_cloud_G08_3.log` with verdict `FAIL`.
- Required R1: supported OpenAI Chat/Responses normalized and tunnel requests bypass the private liveness owner when `stream_evidence_gate.enabled=false`; semantic filter enablement and liveness runtime ownership must be separated without changing normal disabled-semantic wire behavior.
- Required R2: `stream_gate_stall_recovery_test.go` contains only mapper/filter units, not the S05 lifecycle matrix, and the implementation artifact's combined race output recorded only the service package line.
- Fresh reviewer evidence passed the focused stall tests, relevant non-race packages, vet, `git diff --check`, and an independently rerun `go test -race -count=3 ./apps/edge/internal/openai`; those passes validate the implemented subset but do not close R1 or R2.

## Analysis

### Files Read

- `apps/edge/internal/openai/stream_gate_runtime.go`, `stream_gate_policy.go`, `stream_gate_filters.go`, `stream_gate_dispatcher.go`, `stream_gate_release_sink.go`
- `apps/edge/internal/openai/chat_handler.go`, `chat_completion.go`, `buffered_sse.go`, `normalized_sse.go`, `provider_tunnel.go`, `responses_handler.go`, `responses_stream_gate.go`, `run_result.go`
- `apps/edge/internal/openai/stream_gate_stall_recovery_test.go`, `stream_gate_pipeline_test.go`, `stream_gate_vertical_slice_test.go`
- `packages/go/streamgate/runtime.go`, `recovery_coordinator.go`, `commit_boundary.go`, `filter_registry.go`
- `agent-contract/inner/execution-runtime.md`, `agent-contract/inner/edge-config-runtime-refresh.md`, `agent-contract/outer/openai-compatible-api.md`
- `agent-spec/runtime/stream-evidence-gate.md`, `agent-spec/input/openai-compatible-surface.md`, `agent-spec/runtime/edge-node-execution.md`
- `agent-roadmap/phase/operational-observability-provider-management/PHASE.md`, active milestone, and approved milestone SDD
- `agent-test/local/rules.md`, `edge-smoke.md`, `platform-common-smoke.md`

### SDD and Contract Criteria

- The active approved SDD's S05 and Evidence Map assign bounded retry ownership to the supported OpenAI-compatible host. A no-owner typed terminal is evidence only for unsupported or non-OpenAI surfaces.
- Every supported Chat/Responses normalized or tunnel request must have exactly one internal liveness owner regardless of semantic gate configuration. The existing config flag and `filters[]` continue to control semantic filtering, evidence holding, and provider capability admission only.
- Exact replay remains eligible only for an Edge-confirmed typed handoff while transport is uncommitted, the caller is not cancelled, no tool/side-effect boundary exists, the request snapshot is available, and the shared request/strategy budget remains.
- Recovery uses a new attempt/run identity, avoids the failed provider once, and permits same-provider fallback only for exact `available` evidence. Generic, unconfirmed, post-commit, cancelled, unsafe, exhausted, unsupported, and no-owner cases remain one sanitized terminal.
- Normal responses with semantic filtering disabled must retain the legacy public status, headers, JSON/SSE bytes, ordering, cancellation behavior, strict/tool validation, reasoning fallback, finish reason, passthrough behavior, usage finalization, and single terminal.

### Root Cause

- `stream_gate_runtime.go:797-804` conflates two decisions: whether a supported OpenAI response has a request-local runtime owner and whether configured semantic evidence filtering is enabled.
- `chat_handler.go:259,336`, `responses_handler.go:151,376,508,532`, `chat_completion.go:42`, `buffered_sse.go:18`, `normalized_sse.go:41`, and `provider_tunnel.go:33,579` consequently preserve legacy ownerless branches under the default false flag.
- The private stall registration is correctly separate from configured filters, but it is constructed only after entering the runtime. Candidate capability admission is also guarded by the same predicate, so changing the predicate to always true without a separate semantic switch would incorrectly enable configured semantic policy.
- The current three `TestOpenAIStall*` tests stop at mapper/filter state. They do not drive the handler/runtime/admission/renderer lifecycle or prove the required variant product and exactly-once outcomes.

### Finding Resolution Map

| Finding | Resolution | Direct Fix Boundary |
|---------|------------|---------------------|
| Required R1 | `direct-fix` | Split semantic enablement from supported-path runtime ownership, route every supported OpenAI path through exactly one request runtime, preserve disabled-semantic compatibility, and synchronize active contracts/specs. |
| Required R2 | `direct-fix` | Add deterministic S05 full-lifecycle matrix and disabled-semantic compatibility tests, then record fresh complete output for every exact verification command. |

### Split Judgment

Keep one plan. The same request-local authority owns typed mapping, caller commit, cancellation, side-effect state, shared budget, old-attempt close, failed-provider avoidance, re-admission, and final rendering across all endpoint/path/config variants. Splitting ownership from matrix verification would allow a partial change to pass unit tests while retaining duplicate dispatch or an ownerless branch.

### Scope Rationale

Do not add another retry loop, liveness counter, Core/Node recovery owner, config field, metric, wire field, or non-OpenAI owner. Do not expose raw provider messages or arbitrary metadata. Preserve the existing typed mapper and service candidate policy unless a direct call-site adjustment is required by the ownership split.

### Final Routing

- `evaluation_mode=isolated-reassessment`; finalizer=`finalize-task-policy.sh pair` executed once after plan semantics were frozen.
- All build/review closures are true. Build and review scores are `(2,2,1,1,2)` => G08.
- Build base is `local-fit`; `large_indivisible_context=false`; positive loop risks are `temporal_state`, `concurrent_consistency`, `boundary_contract`, and `variant_product` (4).
- `review_rework_count=1`, `evidence_integrity_failure=true`; recovery boundary selects `PLAN-cloud-G08.md`.
- Official review selects `CODE_REVIEW-cloud-G08.md` (`codex`, `gpt-5.6-sol`, `xhigh`).

## Implementation Checklist

- [ ] REVIEW_API-1 gives every supported OpenAI Chat/Responses normalized and tunnel request exactly one private typed-stall recovery owner independent of semantic gate enablement, while the flag and configured filters alone control semantic filter registration, evidence policy, and capability admission and disabled-semantic non-stall behavior remains wire-compatible.
- [ ] REVIEW_API-2 adds deterministic full-lifecycle tests for the S05 endpoint/path/config matrix, alternate and same-provider selection, every unsafe/no-owner terminal row, shared-budget/new-identity/exactly-once invariants, and disabled-semantic compatibility; all exact verification output is recorded completely.
- [ ] Synchronize the active execution/config/OpenAI contracts and matching specs so they state always-on supported-path liveness ownership and semantic-only flag behavior without claiming unsupported surfaces recover.
- [ ] Fill implementation-owned sections in `CODE_REVIEW-cloud-G08.md` with actual implementation notes, deviations, design decisions, and complete raw command output.

### [REVIEW_API-1] Separate semantic activation from liveness runtime ownership

**Problem:** `streamGateEnabled()` returns the semantic config flag and guards both runtime entry and semantic capability admission. The default false setting therefore bypasses the only private stall registration on every supported path, violating API-2 and S05. Simply returning true would also apply configured semantic filters/capability admission when operators disabled them and previously caused compatibility regressions.

**Solution:** Introduce explicit, separately named decisions for (a) supported OpenAI response-runtime ownership and (b) semantic gate activation. Route all supported Chat/Responses normalized and tunnel response lifecycles through the existing request-local StreamGate host exactly once. When semantic activation is false, construct only baseline/no-op mechanics plus the private stall registration, do not apply configured semantic filter registrations or their provider candidate predicate, and release ordinary events at the legacy-compatible boundary. When true, preserve current semantic filter registry, selector, hold, and capability behavior. Keep the existing confirmed-stall mapper/filter, shared recovery coordinator, confirmed transport close, provider avoidance, and terminal renderer as the sole liveness flow.

The disabled-semantic path must preserve cancellation, strict/tool validation and retries, reasoning-only fallback, finish reasons, SSE role/delta/`[DONE]` order, non-stream JSON, tunnel status/header/body order, usage finalization, and exactly one terminal. Fix compatibility in the shared runtime/release adapter rather than retaining an ownerless handler branch or adding a second retry loop.

**Modified Files and Checklist:**

- [ ] `apps/edge/internal/openai/stream_gate_runtime.go`: split the predicates, make supported-path runtime ownership unconditional, and keep one private stall registration per request.
- [ ] `apps/edge/internal/openai/stream_gate_policy.go`: suppress configured semantic registrations and capability requirements when semantic activation is false while preserving current enabled behavior.
- [ ] `apps/edge/internal/openai/chat_handler.go`, `responses_handler.go`: use semantic activation only around provider candidate capability admission and use response-runtime ownership for result handling.
- [ ] `apps/edge/internal/openai/chat_completion.go`, `buffered_sse.go`, `normalized_sse.go`, `provider_tunnel.go`: remove ownerless supported response branches and route through the single runtime owner.
- [ ] `apps/edge/internal/openai/stream_gate_release_sink.go`, `responses_stream_gate.go`: preserve endpoint-native disabled-semantic JSON/SSE/tunnel ordering, terminal, cancellation, strict/tool, reasoning, and usage semantics where the always-on runtime exposes a mismatch.
- [ ] `agent-contract/inner/execution-runtime.md`, `agent-contract/inner/edge-config-runtime-refresh.md`, `agent-contract/outer/openai-compatible-api.md`: replace runtime-enabled ownership claims with always-on supported-path liveness ownership and semantic-only flag semantics.
- [ ] `agent-spec/runtime/stream-evidence-gate.md`, `agent-spec/input/openai-compatible-surface.md`, `agent-spec/runtime/edge-node-execution.md`: synchronize current implementation and verification pointers.

**Reviewer Checkpoints:**

- Every supported Chat/Responses normalized/tunnel entry point reaches exactly one request runtime when semantic activation is both false and true; unsupported/non-OpenAI paths do not gain an owner.
- Candidate capability admission and configured semantic filters are inactive when the flag is false and unchanged when true.
- No new retry/counter/owner exists in StreamGate Core, Edge service, or Node; confirmed old transports still close without duplicate `CancelRun`.
- Disabled-semantic successful and terminal responses preserve endpoint-native public behavior and exactly-once usage/terminal ownership.

### [REVIEW_API-2] Prove the S05 lifecycle matrix and restore evidence trust

**Problem:** The existing stall tests validate only raw-free mapping and filter intent. They do not prove handler/runtime ownership, recovery admission, provider choice, identity, budget, public rendering, or the endpoint/path/config product. The prior artifact also omitted part of a combined race result.

**Solution:** Extend `stream_gate_stall_recovery_test.go` with deterministic handler/runtime integration fixtures named `TestOpenAIStallRecoveryMatrix` and `TestOpenAISemanticGateDisabledCompatibility`. Drive Chat and Responses across normalized and tunnel paths with semantic activation enabled and disabled. For each supported combination prove alternate recovery for `available|unavailable|unknown`, same-provider fallback only for `available`, terminal for unavailable/unknown same-only, and exactly one replacement dispatch with a new run/attempt identity and one public terminal. Cover unsupported/no-owner, generic/unconfirmed, post-commit, caller cancel, tool/side-effect, missing snapshot, and exhausted shared budget as terminal without re-admission. Prove normal disabled-semantic JSON/SSE/tunnel behavior, strict/tool validation, reasoning/finish rendering, cancellation, usage, and ordering against existing compatibility expectations.

Run race packages separately and capture the entire output of every exact command in the review artifact. Do not summarize a missing package result as success.

**Modified Files and Checklist:**

- [ ] `apps/edge/internal/openai/stream_gate_stall_recovery_test.go`: add the complete S05 lifecycle and semantic-disabled compatibility matrix with deterministic dispatch, identity, provider, terminal, cancel, and budget assertions.
- [ ] `agent-task/m-node-provider-execution-liveness-recovery/10+09_stall_recovery/CODE_REVIEW-cloud-G08.md`: record actual changes and complete unabridged verification output.

**Reviewer Checkpoints:**

- The matrix contains both endpoints, both normalized/tunnel paths, and both semantic flag states; assertions prove runtime ownership rather than calling the filter directly.
- Alternate and same-provider rows assert provider selection, one recovery dispatch, a new identity, shared budget consumption, old transport close behavior, and one public terminal.
- Every unsafe/no-owner row asserts zero recovery dispatch and sanitized terminal behavior.
- Verification output includes separate complete service and OpenAI race results and all repository-native smoke results.

## Dependencies and Execution Order

1. Preserve the reviewed typed-stall implementation and completed predecessor contracts.
2. Implement REVIEW_API-1 before relying on new matrix expectations.
3. Implement REVIEW_API-2, synchronize docs/specs, then run every final verification command from a clean command invocation.

## Modified Files Summary

| File | Item |
|------|------|
| `apps/edge/internal/openai/stream_gate_runtime.go` | REVIEW_API-1 |
| `apps/edge/internal/openai/stream_gate_policy.go` | REVIEW_API-1 |
| `apps/edge/internal/openai/chat_handler.go` | REVIEW_API-1 |
| `apps/edge/internal/openai/responses_handler.go` | REVIEW_API-1 |
| `apps/edge/internal/openai/chat_completion.go` | REVIEW_API-1 |
| `apps/edge/internal/openai/buffered_sse.go` | REVIEW_API-1 |
| `apps/edge/internal/openai/normalized_sse.go` | REVIEW_API-1 |
| `apps/edge/internal/openai/provider_tunnel.go` | REVIEW_API-1 |
| `apps/edge/internal/openai/stream_gate_release_sink.go` | REVIEW_API-1 |
| `apps/edge/internal/openai/responses_stream_gate.go` | REVIEW_API-1 |
| `apps/edge/internal/openai/stream_gate_stall_recovery_test.go` | REVIEW_API-2 |
| `agent-contract/inner/execution-runtime.md` | REVIEW_API-1 |
| `agent-contract/inner/edge-config-runtime-refresh.md` | REVIEW_API-1 |
| `agent-contract/outer/openai-compatible-api.md` | REVIEW_API-1 |
| `agent-spec/runtime/stream-evidence-gate.md` | REVIEW_API-1 |
| `agent-spec/input/openai-compatible-surface.md` | REVIEW_API-1 |
| `agent-spec/runtime/edge-node-execution.md` | REVIEW_API-1 |
| `agent-task/m-node-provider-execution-liveness-recovery/10+09_stall_recovery/CODE_REVIEW-cloud-G08.md` | REVIEW_API-1, REVIEW_API-2 |

## Final Verification

Fresh output is required. Record each command and its complete output separately.

1. `go test -count=20 ./apps/edge/internal/openai -run '^(TestOpenAIStallRecoveryFilter|TestOpenAIAttemptControllerConfirmedStall|TestOpenAIAttemptDispatcherStalledProvider)$'` — PASS every iteration.
2. `go test -count=10 ./apps/edge/internal/openai -run '^TestOpenAIStallRecoveryMatrix$'` — PASS every iteration and execute every endpoint/path/config subtest.
3. `go test -count=10 ./apps/edge/internal/openai -run '^TestOpenAISemanticGateDisabledCompatibility$'` — PASS every iteration.
4. `go test -count=1 ./packages/go/streamgate ./apps/edge/internal/openai ./apps/edge/internal/service ./apps/edge/internal/controlplane` — PASS.
5. `go test -race -count=3 ./apps/edge/internal/service` — PASS with no race report.
6. `go test -race -count=3 ./apps/edge/internal/openai` — PASS with no race report.
7. `go vet ./packages/go/streamgate ./apps/edge/internal/service ./apps/edge/internal/openai ./apps/edge/internal/controlplane` — no diagnostics.
8. `./scripts/e2e-smoke.sh` — PASS.
9. `IOP_VLLM_MODE=fake ./scripts/e2e-openai-vllm.sh` — PASS.
10. `./scripts/e2e-provider-capacity-smoke.sh` — PASS.
11. `git diff --check` — no whitespace errors.

After completing all code changes, fill every implementation-owned section in `CODE_REVIEW-cloud-G08.md` and stop with the active pair in place.
