<!-- task=m-iop-hot-path-one-shot-execution/20+17,19_smoke_harness plan=4 tag=REVIEW_REVIEW_TEST milestone-task=hot-smoke -->

# Make the Hot Path smoke manifest fail closed on actual evidence

## For the Implementing Agent

Implement every checklist item, run the exact verification commands, and fill the implementation-owned sections of `CODE_REVIEW-cloud-G09.md` with actual notes and stdout/stderr. Keep the active PLAN/CODE_REVIEW files in place and report ready for review. If blocked, record only the exact blocker, attempted commands/output, and resume condition in implementation-owned evidence fields. Do not ask the user, call user-input tools, create control-plane stop files, classify the next state, archive logs, or write `complete.log`; finalization is owned by the code-review skill.

## Background

The previous harness implementation produced a structurally valid manifest even when both agent executables exited immediately without output. This follow-up makes execution, observation, workspace, schema, and redaction evidence fail closed so the downstream credentialed S16 run cannot report scenario expectations as observed results.

## Archive Evidence Snapshot

- `agent-task/m-iop-hot-path-one-shot-execution/20+17,19_smoke_harness/plan_local_G08_3.log` and `agent-task/m-iop-hot-path-one-shot-execution/20+17,19_smoke_harness/code_review_cloud_G08_3.log` close plan 3 with `FAIL`: four Required findings, zero Suggested findings, and zero Nits.
- Fresh reviewer reproduction used `/bin/false` for both agents and an initially empty observation directory. `--run` exited 0, direct cases were recorded as `completed/success`, timeout cases as `cancelled`, visible events were `terminal_error/no_events`, cancellation was `triggered=false,target=none`, and 24 observation rows were synthesized.
- Required rework: derive case results from actual exit/protocol/cancellation evidence, consume rather than synthesize production observation evidence, make workspace evidence content-sensitive, enforce the supplied fixed-matrix schema, and keep every persisted artifact free of raw prompt/output/credential material.
- Split prerequisites remain satisfied by `agent-task/archive/2026/08/m-iop-hot-path-one-shot-execution/17+14,15,16_endpoint_error_matrix/complete.log` and `agent-task/archive/2026/08/m-iop-hot-path-one-shot-execution/19+17,18_observation_lifecycle/complete.log`.

## Analysis

### Files Read

- `agent-task/m-iop-hot-path-one-shot-execution/20+17,19_smoke_harness/PLAN-local-G08.md`
- `agent-task/m-iop-hot-path-one-shot-execution/20+17,19_smoke_harness/CODE_REVIEW-cloud-G08.md`
- `agent-task/m-iop-hot-path-one-shot-execution/20+17,19_smoke_harness/code_review_cloud_G08_2.log`
- `agent-roadmap/current.md`
- `agent-roadmap/phase/knowledge-tool-optimization-extension/PHASE.md`
- `agent-roadmap/phase/knowledge-tool-optimization-extension/milestones/iop-hot-path-one-shot-execution.md`
- `agent-roadmap/sdd/knowledge-tool-optimization-extension/iop-hot-path-one-shot-execution/SDD.md`
- `agent-spec/runtime/stream-evidence-gate.md`
- `agent-contract/outer/openai-compatible-api.md`
- `agent-contract/outer/anthropic-compatible-api.md`
- `agent-ops/rules/project/domain/testing/rules.md`
- `agent-test/local/rules.md`
- `agent-test/local/testing-smoke.md`
- `scripts/e2e-hot-path-agents.sh`
- `scripts/fixtures/hot-path-agent-smoke-manifest.schema.json`
- `agent-task/archive/2026/08/m-iop-hot-path-one-shot-execution/17+14,15,16_endpoint_error_matrix/complete.log`
- `agent-task/archive/2026/08/m-iop-hot-path-one-shot-execution/19+17,18_observation_lifecycle/complete.log`

### SDD Criteria

- SDD: `agent-roadmap/sdd/knowledge-tool-optimization-extension/iop-hot-path-one-shot-execution/SDD.md`; status `[승인됨]`; SDD lock released; no `USER_REVIEW.md`.
- First-line contribution remains `milestone-task=hot-smoke`; targeted Acceptance Scenario is S16.
- S16 and its Evidence Map require actual Claude/Pi streaming logs plus workspace before/after evidence. They require visible stage output, artifact lifecycle, and endpoint-standard terminal evidence rather than requested-scenario labels.
- The checklist therefore repairs evidence derivation and schema/redaction trust only. Actual credentialed Claude/Pi execution remains downstream evidence and is not claimed by this child.

### Verification Context

- No verification handoff was supplied. Repository-native fallback comes from the testing domain rule, `agent-test/local/rules.md`, `agent-test/local/testing-smoke.md`, the approved SDD, the two outer protocol contracts, and fresh reviewer probes.
- Workdir is `/config/workspace/iop-s0`; current checkout uses the available Go toolchain, Bash, jq, and GNU coreutils. No credential or network access is required or permitted for this child.
- Fresh baseline: `bash -n`, the current schema shape command, `--self-test`, `go test -race -count=1 ./packages/go/streamgate ./packages/go/config ./apps/edge/internal/openai ./apps/edge/internal/service`, and `git diff --check` exited 0. The focused `/bin/false` reproduction also exited 0 and contradicted the claimed evidence.
- The final self-test must run the production `do_run`/manifest path with deterministic fakes, include immediate-failure and sensitive-output negative controls, and reject contradictions. Go cache output is not accepted (`-count=1`).
- External Verification Preflight: not applicable. Actual credentials, provider calls, Make integration, deployment, and field/full-cycle execution remain excluded.
- Confidence is high because the failing case was reproduced on the exact active source with deterministic local binaries and no external dependency.

### Test Coverage Gaps

- The self-test verifies expected fake output but has no early-exit/no-output negative control, so hard-coded scenario results pass.
- The self-test creates the same observation rows later accepted as production evidence; it never proves consumption of independently produced observations or rejection of missing/mismatched correlation.
- Schema rejection covers length, one forbidden key, and one enum only; it does not test duplicate ids with distinct rows, id/agent/scenario mismatch, terminal/event contradiction, cancellation mismatch, or use of the supplied fixture.
- Redaction checks only the final manifest and does not seed or inspect persisted argv/stdout evidence files.
- Workspace hashing covers sorted paths but not contents, so repair/content changes are not observable.

### Symbol References

- No symbol is renamed or removed. Changes stay inside the new standalone harness and its schema.

### Split Judgment

- Keep one plan. Execution capture, observation/workspace correlation, schema enforcement, and persisted-artifact redaction form one evidence-integrity invariant; any subset could still emit a misleading manifest.
- Predecessor 17 is satisfied by `agent-task/archive/2026/08/m-iop-hot-path-one-shot-execution/17+14,15,16_endpoint_error_matrix/complete.log`.
- Predecessor 19 is satisfied by `agent-task/archive/2026/08/m-iop-hot-path-one-shot-execution/19+17,18_observation_lifecycle/complete.log`.

### Scope Rationale

- Modify only the harness, its schema, and active review evidence. Do not change production Edge/Node code, Makefile targets, deployment/config, credential handling, or tracked smoke output.
- Do not run installed Claude/Pi binaries or providers. The child closes the deterministic evidence collector; the downstream smoke child owns actual S16 execution and Make integration.
- Do not add a package dependency unless an already available repository-native schema validator is found; the current manifests contain no JSON Schema validator dependency.

### Final Routing

- `evaluation_mode=isolated-reassessment`; finalizer=`finalize-task-policy.sh pair`.
- Build closures are all true. Scores are 2/2/1/2/2, grade G09, base/final route `grade-boundary`, lane `cloud`, filename `PLAN-cloud-G09.md`.
- Review closures are all true. Scores are 2/2/1/2/2, grade G09, route `official-review`, lane `cloud`, filename `CODE_REVIEW-cloud-G09.md`.
- `large_indivisible_context=false`; matched loop risks are `temporal_state`, `boundary_contract`, `structured_interpretation`, and `variant_product` (4). `review_rework_count=2`; `evidence_integrity_failure=true`; risk and recovery boundaries both match but do not replace the grade-boundary basis.
- No capability gap exists; all fixes and deterministic verification are repository-local.

## Implementation Checklist

- [ ] [REVIEW_REVIEW_TEST-1] Make case execution, terminal/cancellation, observation, and workspace evidence derive from actual correlated facts and fail closed on absence or contradiction.
- [ ] [REVIEW_REVIEW_TEST-2] Make the supplied schema the fixed-matrix validation source and ensure every persisted harness artifact is allowlisted/redacted, with non-vacuous negative self-tests.
- [ ] [REVIEW_REVIEW_TEST-3] Run every final syntax, schema, behavioral, common-regression, and diff verification command with fresh evidence.
- [ ] Fill implementation-owned sections in CODE_REVIEW-*-G??.md with actual implementation notes and verification output.

### [REVIEW_REVIEW_TEST-1] Derive evidence from actual execution

**Problem:** `scripts/e2e-hot-path-agents.sh:486` discards child status, `scripts/e2e-hot-path-agents.sh:512` substitutes scenario expectations, and `scripts/e2e-hot-path-agents.sh:508` creates its own production observation rows. `tree_sha256` at line 94 hashes only file names. A no-output `/bin/false` run therefore produces a successful manifest with synthetic observations.

**Solution:** Capture child wait status and the parsed native terminal, then derive one case result only when exit, terminal, cancellation, observation, and workspace facts form the expected scenario-specific combination. Generate observation fixtures only in self-test setup; production `--run` must consume independently present per-case evidence and reject missing, duplicate, mismatched request ids/stages, or impossible ordering. Hash workspace relative paths and file bytes, and assert before/after lifecycle invariants.

Before (`scripts/e2e-hot-path-agents.sh:486-517`):

```bash
wait "$child_pid" 2>/dev/null || true
visible_events=$(parse_visible_events "$agent" "$out_file" "$cancelled")
obs_file=$(write_observation_log "$OBSERVATION_DIR" "$case_id" "$request_id" "$scenario")
expectation=$(scenario_expectation "$scenario")
outcome="${expectation%%:*}"
```

After:

```bash
child_status=0
wait "$child_pid" 2>/dev/null || child_status=$?
visible_events=$(parse_visible_events "$agent" "$out_file" "$cancelled")
observation=$(load_observation_evidence "$case_id" "$request_id")
case_result=$(derive_and_validate_case_result "$scenario" "$child_status" "$cancelled" "$visible_events" "$observation" "$snapshot_before" "$snapshot_after")
```

**Modified Files and Checklist:**

- [ ] Update `scripts/e2e-hot-path-agents.sh` to capture actual process/terminal/cancel state and reject missing or contradictory evidence before manifest write.
- [ ] Separate self-test observation fixture creation from production observation consumption and enforce exact request/stage correlation.
- [ ] Make workspace snapshots content-sensitive and assert direct/no-artifact, pass/repair/removed, write-failure, and cancel/orphan facts.
- [ ] Add a deterministic immediate-exit/no-output negative control that must fail without writing a manifest.

**Test Strategy:** Extend the embedded `--self-test`; no separate test file is needed because it already owns isolated fake binaries, observations, and workspaces. Add assertion labels for early exit, missing/mismatched observation, terminal/event contradiction, cancellation-not-triggered, and content-only workspace changes.

**Verification:** `./scripts/e2e-hot-path-agents.sh --self-test` exits 0 only after proving every malformed case is rejected and the valid 2x5 fake matrix still passes.

### [REVIEW_REVIEW_TEST-2] Enforce schema and persisted-artifact safety

**Problem:** `scripts/fixtures/hot-path-agent-smoke-manifest.schema.json:33` fixes only cardinality, while the runtime validator at `scripts/e2e-hot-path-agents.sh:686` is a separate partial jq contract and never applies `--fixture`. Raw prompt-bearing argv and unredacted stdout remain under the caller observation directory at lines 421 and 435, but only the final manifest is scanned.

**Solution:** Encode the ten exact `id`/`agent`/`scenario` rows and scenario-specific result/cancel/cleanup relations in the supplied schema, and make runtime validation consume that file as its source. Persist only allowlisted visible-event/observation summaries and digests; keep any raw capture under an owned disposable directory and remove it after normalization. Seed sensitive fake stdout and assert all surviving files are clean.

Before (`scripts/e2e-hot-path-agents.sh:686-698`):

```bash
validate_manifest() {
    local doc="$1"
    jq -e '.schema_version == "1" and (.cases | length) == 10' >/dev/null <<<"$doc"
}
```

After:

```bash
validate_manifest() {
    local schema="$1" doc="$2"
    validate_against_supplied_schema "$schema" "$doc"
    validate_runtime_correlations "$doc"
    validate_persisted_artifact_allowlist
}
```

**Modified Files and Checklist:**

- [ ] Update `scripts/fixtures/hot-path-agent-smoke-manifest.schema.json` with ten exact row identities and cross-field case contracts; reject extra/duplicate/mismatched rows.
- [ ] Update `scripts/e2e-hot-path-agents.sh` so `--fixture` controls validation rather than serving only as a hash input.
- [ ] Keep raw capture disposable and persist only schema-allowlisted, redacted evidence.
- [ ] Add negative tests for duplicate/missing/distinct duplicate ids, id/agent/scenario mismatch, terminal/event mismatch, cancellation mismatch, alternate malformed fixture, and sensitive stdout/prompt leakage across all surviving artifacts.

**Test Strategy:** Extend embedded self-test mutations and inspect the complete surviving artifact set. Do not download or create a repository-local validator tool. If a generic Draft 2020-12 validator is unavailable, implement the exact closed schema subset used here and prove that changing the supplied fixture changes acceptance.

**Verification:** The schema jq assertion and `--self-test` both exit 0; the self-test must demonstrate non-vacuous rejection for every listed invariant and zero sensitive matches outside disposable raw capture.

### [REVIEW_REVIEW_TEST-3] Run fresh final verification

**Problem:** The previous commands passed even though the behavioral oracle accepted a completely failed run. Fresh regression evidence is required after replacing the oracle.

**Solution:** Run the exact commands below after the two evidence-contract fixes. Preserve complete stdout/stderr in the review file and explain any command deviation.

Before (`CODE_REVIEW-cloud-G08.md:131-156`):

```text
self-test: PASS, but no early-exit/no-output negative control
common regression: PASS
```

After:

```text
syntax/schema/self-test/common regression/diff: PASS
focused malformed executions: rejected before manifest output
```

**Modified Files and Checklist:**

- [ ] Fill `agent-task/m-iop-hot-path-one-shot-execution/20+17,19_smoke_harness/CODE_REVIEW-cloud-G09.md` with actual design decisions, deviations, and full fresh outputs.

**Test Strategy:** No additional production test package is needed. The embedded behavioral oracle and existing race-enabled common regression cover this test-only script/schema change.

**Verification:** Run every command in Final Verification; every command exits 0 and no malformed run writes a success manifest.

## Dependencies and Execution Order

1. Predecessor 17 is complete at `agent-task/archive/2026/08/m-iop-hot-path-one-shot-execution/17+14,15,16_endpoint_error_matrix/complete.log`.
2. Predecessor 19 is complete at `agent-task/archive/2026/08/m-iop-hot-path-one-shot-execution/19+17,18_observation_lifecycle/complete.log`.
3. Implement REVIEW_REVIEW_TEST-1, then REVIEW_REVIEW_TEST-2, then REVIEW_REVIEW_TEST-3.

## Modified Files Summary

| File | Item |
|---|---|
| `scripts/e2e-hot-path-agents.sh` | REVIEW_REVIEW_TEST-1, REVIEW_REVIEW_TEST-2 |
| `scripts/fixtures/hot-path-agent-smoke-manifest.schema.json` | REVIEW_REVIEW_TEST-2 |
| `agent-task/m-iop-hot-path-one-shot-execution/20+17,19_smoke_harness/CODE_REVIEW-cloud-G09.md` | REVIEW_REVIEW_TEST-3 |

## Final Verification

```bash
bash -n scripts/e2e-hot-path-agents.sh
jq -e '(.properties.cases.prefixItems | length) == 10 and .properties.cases.items == false and ([.properties.cases.prefixItems[].properties.id.const] | length == 10) and ([.properties.cases.prefixItems[].properties.id.const] | unique | length == 10)' scripts/fixtures/hot-path-agent-smoke-manifest.schema.json
./scripts/e2e-hot-path-agents.sh --self-test
go test -race -count=1 ./packages/go/streamgate ./packages/go/config ./apps/edge/internal/openai ./apps/edge/internal/service
git diff --check
```

Expected: every command exits 0 with fresh evidence. The self-test proves the valid 2x5 matrix and rejects failed execution, missing/mismatched observation, terminal/cancel/workspace contradiction, malformed schema relations, and sensitive persisted artifacts without credentials or network calls.

After completing all code changes, fill implementation-owned sections in `CODE_REVIEW-*-G??.md`.
