<!-- task=m-principal-provider-credential-slot-routing/08+03,04,07_secure_delivery plan=5 tag=REVIEW_REVIEW_REVIEW_REVIEW_API milestone-task=secure-transport,credential-lease,adapter-injection,revocation -->

# Code Review Reference - REVIEW_REVIEW_REVIEW_REVIEW_API

> **[IMPLEMENTING AGENT — READ FIRST] Filling in this file is the mandatory final step of implementation.**
> The task is NOT complete until every implementation-owned section below is filled in.
> Complete the `Implementation Checklist`; the final checklist item is mandatory before saving.
> Fill implementation-owned sections, then stop with active files in place and report ready for review.
> If implementation is blocked, record the exact blocker, attempted commands/output, and resume condition only in implementation-owned evidence fields.
> Do not ask the user directly, present choices, call user-input tools, create control-plane stop files, or classify the next state.
> Finalization (`Code Review Result`, log rename, `complete.log`, archive moves, `Review-Only Checklist`) is review-agent-only, even after compaction/resume.
> Follow the ownership table at the bottom of this file for which sections you own.

## Overview

date=2026-08-02
task=m-principal-provider-credential-slot-routing/08+03,04,07_secure_delivery, plan=5, tag=REVIEW_REVIEW_REVIEW_REVIEW_API

## Archive Evidence Snapshot

- `plan_cloud_G10_4.log` and `code_review_cloud_G10_4.log`: current-pair predicted archives; verdict `FAIL` with 1 Required, 0 Suggested, and 0 Nit findings. Fresh reviewer verification passed, but dependency inspection proved that projection fan-out and shutdown are not bounded across a stalled socket write.
- `plan_cloud_G10_3.log` and `code_review_cloud_G10_3.log`: prior loop with two Required identity/freshness findings. The current source and fresh reviewer tests prove that enrollment-name binding and immutable same-generation renewal are now implemented.
- `agent-task/archive/2026/08/m-principal-provider-credential-slot-routing/03+01,02_projection_auth/complete.log` and `agent-task/archive/2026/08/m-principal-provider-credential-slot-routing/07+01,02,05_secret_material/complete.log`: secret-free projection/cache and secret-at-rest prerequisites remain satisfied.

## For the Review Agent

> **[REVIEW AGENT ONLY]** The finalization steps below are review-agent only. Implementing agents must not execute this section.

Compare implementation of each item against source files and verify that output in `Verification Results` matches code.
Review completion means the following steps are finished:

1. Append verdict and `review_rework_count` / `evidence_integrity_failure` routing signals.
2. Archive `CODE_REVIEW-cloud-G07.md` → `code_review_cloud_G07_5.log` and `PLAN-cloud-G07.md` → `plan_cloud_G07_5.log`.
3. If PASS, write `complete.log` and move active task directory to `agent-task/archive/YYYY/MM/m-principal-provider-credential-slot-routing/08+03,04,07_secure_delivery/`. If WARN/FAIL, fully write the next filesystem state required by the code-review skill.
4. If PASS and task group is `m-principal-provider-credential-slot-routing`, preserve the first-line `milestone-task` metadata in `complete.log` and report it for the runtime aggregation event. Roadmap state evaluation belongs to `sync-milestone-workstate`.
5. Check applicable `Review-Only Checklist` items at the final `.log` location before reporting.

---

## Implementation Item Completion

| Item | Status |
|------|---------|
| REVIEW_REVIEW_REVIEW_REVIEW_API-1 Bound and cancel projection fan-out | [x] |

## Implementation Checklist

- [x] Bound and parallelize projection pushes so context cancellation or a per-client deadline closes and unblocks a stalled Edge, every snapshotted Edge is attempted independently, errors are aggregated, and server stop completes promptly; pass deterministic stalled-client and shutdown regressions.
- [x] Run the focused, race, full secure-delivery, vet, diff, and secret-structure verification commands with fresh output.
- [x] Fill implementation-owned sections in CODE_REVIEW-*-G??.md with actual implementation notes and verification output.

## Review-Only Checklist

> **[REVIEW AGENT ONLY]** This checklist is used only by the review agent.
> Implementing agents must not modify or check this section.

- [x] Append one verdict of `PASS`, `WARN`, or `FAIL` and verified `review_rework_count`, `evidence_integrity_failure` to `Code Review Result`.
- [x] Verify that verdict, `Dimension Assessment`, and Required/Suggested/Nit classifications match.
- [x] Archive active `CODE_REVIEW-*-G??.md` to `code_review_cloud_G07_5.log`.
- [x] Archive active `PLAN-*-G??.md` to `plan_cloud_G07_5.log`.
- [x] Verify that the Agent-Ops managed block in `.gitignore` unignores `agent-task/**/*.md` and `agent-task/**/*.log` and ignores `agent-roadmap/current.md`.
- [ ] If PASS, write `complete.log` based on `agent-ops/skills/common/code-review/templates/complete-log-template.md` and leave no active `.md` files.
- [ ] If PASS, move active task directory `agent-task/m-principal-provider-credential-slot-routing/08+03,04,07_secure_delivery/` to `agent-task/archive/YYYY/MM/m-principal-provider-credential-slot-routing/08+03,04,07_secure_delivery/` and update this checklist at the final archive path.
- [ ] If PASS and task group is `m-principal-provider-credential-slot-routing`, preserve and report `milestone-task` metadata for runtime aggregation, without modifying roadmap or directly calling `update-roadmap`.
- [ ] If PASS for split work, remove empty active parent `agent-task/m-principal-provider-credential-slot-routing/` or verify it was kept due to remaining siblings/files.
- [x] If WARN/FAIL, write the next filesystem state matching code-review verdict and do not write `complete.log`.

## Deviations from Plan

- The plan's `After` sketch inlined the fan-out loop directly in `BroadcastProjection` with the `projectionPushTimeout` constant. To keep the public `BroadcastProjection(ctx)` signature unchanged (it is called from `apps/control-plane/cmd/control-plane/server.go:114` and from the refresh loop) while letting the focused stalled-edge regression drive a short, deterministic per-client deadline, the fan-out body was factored into a private `broadcastProjection(ctx, pushTimeout)` seam; the public method delegates with the constant. This stays inside the plan's allowance to "add private projection-send helpers used only by `BroadcastProjection`" and does not change any exported symbol or command in the Verification block.
- The focused test file calls the internal `broadcastProjection(ctx, 300ms)` seam (same package, white-box) to assert the deadline path without a 10s wait. The named tests in the Verification commands are unchanged; only the deadline value differs from the production constant, so no `Verification Results` command was altered.

## Key Design Decisions

- **Concurrent, independently bounded pushes.** `broadcastProjection` snapshots the active clients under `activeMu`, sorts that snapshot by `edgeID`, then starts one goroutine per target that calls `pushProjection`. Results are written into a pre-sized `failures[index]` slice in that stable order and joined with `errors.Join`, so aggregation stays deterministic and no client waits behind another. `broadcastMu` still serializes whole batches, preserving one-batch-at-a-time semantics.
- **The deadline is enforced by the caller, not by proto-socket.** proto-socket's `SendRequest` starts its request timer only after `QueuePacket` returns, and `TcpClient.WritePacket` writes with no deadline, so a non-reading Edge can stall the socket write indefinitely (the internal timer never arms). `pushProjection` therefore runs the typed request on its own goroutine and `select`s on the result, `ctx.Done()`, and an independent `time.NewTimer(pushTimeout)`. The outer timer is created before the request goroutine arms its internal timer, so for a stalled write/response the outer deadline is authoritative.
- **Unblock by closing, then drain.** On cancellation or deadline, `pushProjection` calls `target.client.Close()`, which closes the underlying `net.Conn` and unblocks the stalled `writeFull`/`QueuePacket`, then blocks on `<-result` (a buffered channel) to guarantee the request goroutine has exited before returning. No send goroutine or pending request outlives the batch. The closed client is removed by the existing token-fenced disconnect listener via `clearActiveClient`, so a stale disconnect cannot evict a newer reconnection.
- **Prompt stop.** Because the refresh loop calls `BroadcastProjection(ctx)` with the refresh context, `stopProjectionRefresh`'s `cancel()` propagates into every in-flight `pushProjection`, which closes its stalled client and returns; `sends.Wait()` unblocks, the refresh goroutine observes `ctx.Done()` and exits, and `Stop` proceeds to `server.Stop()` without waiting on the non-reading Edge.
- **`projectionClient` lifted to package scope** so both the snapshot loop and the `pushProjection` helper share one target type; no exported symbol was renamed or removed.

## Reviewer Checkpoints

- Confirm `BroadcastProjection` starts every snapshotted Edge push independently while preserving serialization between projection batches.
- Confirm caller cancellation and the per-client deadline close a stalled client, unblock its socket write, and wait for the request goroutine to exit without leaks.
- Confirm `EdgeServer.Stop` promptly cancels an in-flight periodic refresh before waiting and that the existing disconnect listener safely removes the closed client.
- Confirm result collection remains deterministic, aggregates all Edge-qualified errors, and preserves responsive acceptance, higher-generation, and rejection behavior.
- Confirm the focused stalled-client tests, wire race test, full secure-delivery package/process suite, vet, diff, and secret-structure guard all pass with fresh output.

## Verification Results

### Focused Fan-out and Cancellation

```bash
env GOCACHE=/config/tmp/gocache-secure-delivery GOTMPDIR=/config/workspace/.gotmp-iop-secure-delivery TMPDIR=/config/workspace/.gotmp-iop-secure-delivery go test -count=1 ./apps/control-plane/internal/wire -run 'TestEdgeServer(BroadcastProjectionDoesNotLetStalledEdgeBlockHealthyEdge|StopCancelsStalledProjectionRefresh|PeriodicallyRefreshesEveryActiveEdge|BroadcastProjectionAttemptsAllActiveEdges)'
```

```
ok  	iop/apps/control-plane/internal/wire	0.482s
exit=0
```

Verbose confirmation of the four named cases (single run):

```
--- PASS: TestEdgeServerPeriodicallyRefreshesEveryActiveEdge (0.11s)
--- PASS: TestEdgeServerBroadcastProjectionAttemptsAllActiveEdges (0.00s)
--- PASS: TestEdgeServerBroadcastProjectionDoesNotLetStalledEdgeBlockHealthyEdge (0.30s)
--- PASS: TestEdgeServerStopCancelsStalledProjectionRefresh (0.02s)
PASS
```

The stalled-edge case returns within the ~300ms per-client deadline (bounded), and stop cancels the in-flight refresh in 0.02s.

### Wire Race Verification

```bash
env GOCACHE=/config/tmp/gocache-secure-delivery GOTMPDIR=/config/workspace/.gotmp-iop-secure-delivery TMPDIR=/config/workspace/.gotmp-iop-secure-delivery go test -count=1 -race ./apps/control-plane/internal/wire
```

```
ok  	iop/apps/control-plane/internal/wire	3.011s
exit=0
```

### Toolchain and Prerequisite Preflight

```bash
command -v go && go version && go env GOROOT
test -f agent-task/archive/2026/08/m-principal-provider-credential-slot-routing/07+01,02,05_secret_material/complete.log
```

```
/config/.local/bin/go
go version go1.26.2 linux/arm64
/config/opt/go
exit=0
# test -f ... 07+01,02,05_secret_material/complete.log
exit=0
```

### Shared Package Tests

```bash
env GOCACHE=/config/tmp/gocache-secure-delivery GOTMPDIR=/config/workspace/.gotmp-iop-secure-delivery TMPDIR=/config/workspace/.gotmp-iop-secure-delivery go test -count=1 ./packages/go/auth ./packages/go/config ./packages/go/credentiallease
```

```
ok  \tiop/packages/go/auth\t10.087s
ok  \tiop/packages/go/config\t0.852s
ok  \tiop/packages/go/credentiallease\t0.030s
exit=0
```

### Control Plane, Edge, and Node Tests

```bash
env GOCACHE=/config/tmp/gocache-secure-delivery GOTMPDIR=/config/workspace/.gotmp-iop-secure-delivery TMPDIR=/config/workspace/.gotmp-iop-secure-delivery go test -count=1 ./apps/control-plane/... ./apps/edge/... ./apps/node/...
```

```
ok  \tiop/apps/control-plane/cmd/control-plane\t6.167s
ok  \tiop/apps/control-plane/internal/credentiallease\t0.484s
ok  \tiop/apps/control-plane/internal/credentialops\t3.258s
ok  \tiop/apps/control-plane/internal/credentialseal\t0.362s
ok  \tiop/apps/control-plane/internal/credentialstore\t5.752s
ok  \tiop/apps/control-plane/internal/wire\t2.224s
ok  \tiop/apps/edge/cmd/edge\t0.968s
ok  \tiop/apps/edge/internal/authprojection\t0.090s
ok  \tiop/apps/edge/internal/bootstrap\t5.677s
ok  \tiop/apps/edge/internal/configrefresh\t0.717s
ok  \tiop/apps/edge/internal/controlplane\t6.663s
ok  \tiop/apps/edge/internal/edgecmd\t0.395s
ok  \tiop/apps/edge/internal/edgevalidate\t0.106s
ok  \tiop/apps/edge/internal/events\t0.065s
ok  \tiop/apps/edge/internal/input\t0.151s
ok  \tiop/apps/edge/internal/input/a2a\t0.138s
ok  \tiop/apps/edge/internal/node\t0.086s
ok  \tiop/apps/edge/internal/openai\t7.494s
ok  \tiop/apps/edge/internal/opsconsole\t0.100s
ok  \tiop/apps/edge/internal/service\t5.960s
ok  \tiop/apps/edge/internal/transport\t4.871s
ok  \tiop/apps/node/cmd/node\t0.101s
ok  \tiop/apps/node/internal/adapters\t0.057s
?   \tiop/apps/node/internal/adapters/mock\t[no test files]
ok  \tiop/apps/node/internal/adapters/ollama\t0.027s
ok  \tiop/apps/node/internal/adapters/openai_compat\t0.161s
ok  \tiop/apps/node/internal/adapters/vllm\t0.144s
ok  \tiop/apps/node/internal/bootstrap\t1.552s
ok  \tiop/apps/node/internal/node\t0.884s
ok  \tiop/apps/node/internal/router\t0.512s
ok  \tiop/apps/node/internal/store\t0.125s
ok  \tiop/apps/node/internal/transport\t5.613s
exit=0
```

### Affected Race Tests

```bash
env GOCACHE=/config/tmp/gocache-secure-delivery GOTMPDIR=/config/workspace/.gotmp-iop-secure-delivery TMPDIR=/config/workspace/.gotmp-iop-secure-delivery go test -count=1 -race ./packages/go/auth ./packages/go/credentiallease ./apps/control-plane/internal/credentiallease ./apps/control-plane/internal/wire ./apps/edge/internal/authprojection ./apps/edge/internal/controlplane ./apps/edge/internal/service ./apps/edge/internal/transport ./apps/node/internal/node
```

```
ok  \tiop/packages/go/auth\t11.237s
ok  \tiop/packages/go/credentiallease\t1.348s
ok  \tiop/apps/control-plane/internal/credentiallease\t1.371s
ok  \tiop/apps/control-plane/internal/wire\t3.132s
ok  \tiop/apps/edge/internal/authprojection\t1.109s
ok  \tiop/apps/edge/internal/controlplane\t7.666s
ok  \tiop/apps/edge/internal/service\t7.067s
ok  \tiop/apps/edge/internal/transport\t5.960s
ok  \tiop/apps/node/internal/node\t2.527s
exit=0
```

### Three-process Secure-delivery Verification

```bash
env GOCACHE=/config/tmp/gocache-secure-delivery GOTMPDIR=/config/workspace/.gotmp-iop-secure-delivery TMPDIR=/config/workspace/.gotmp-iop-secure-delivery make test-control-plane-edge-wire
```

```
[cp-edge-wire] Control Plane edge wire port ready
[cp-edge-wire] CP: hello accepted
[cp-edge-wire] Edge: connected to control plane
[cp-edge-wire] CP: edge disconnected
[cp-edge-wire] Control Plane-Edge wire smoke PASSED.
IOP_SECURE_DELIVERY_E2E=1 go test -count=1 -run '^TestSecureDeliveryThreeProcess$' ./apps/control-plane/cmd/control-plane
exit=0
```

### Vet, Diff, and Secret-structure Guard

```bash
env GOCACHE=/config/tmp/gocache-secure-delivery GOTMPDIR=/config/workspace/.gotmp-iop-secure-delivery TMPDIR=/config/workspace/.gotmp-iop-secure-delivery go vet ./apps/control-plane/... ./apps/edge/... ./apps/node/... ./packages/go/...
git diff --check
if rg --sort path -n 'bytes[[:space:]]+(secret|provider_secret)|map<string, string>[[:space:]]+(credential|provider_auth)' proto/iop/runtime.proto; then exit 1; fi
```

```
go vet ./apps/control-plane/... ./apps/edge/... ./apps/node/... ./packages/go/...
exit=0

git diff --check
exit=0

secret-structure guard (proto/iop/runtime.proto)
exit=0
```

---

> **[IMPLEMENTING AGENT — BEFORE SAVING] Have you filled in every implementation-owned section?**
> If anything is blank, go back and fill it in before saving this file.
> Leave review-agent-only sections unchanged.

## Section Ownership

| Section | Owner | Note |
|---------|-------|------|
| Header comment, Overview, Review Agent Instructions | Fixed at stub creation | Implementing agent must not modify or execute these (archive, complete.log, and task-directory archive move are review-agent only) |
| Archive Evidence Snapshot | Fixed at stub creation from plan when present | Implementing agent uses it as default prior-loop context; read only the specific archive files cited there when more detail is required |
| Implementation Item Completion (item names) | Fixed at stub creation | Implementing agent checks `[ ]` → `[x]` only |
| Implementation Checklist (item text/order) | Fixed at stub creation from plan | Implementing agent checks `[ ]` → `[x]` only |
| Review-Only Checklist | Review agent only | Implementing agent must not modify or check this section |
| Deviations from Plan, Key Design Decisions | Implementing agent | Replace placeholder text with actual content |
| Reviewer Checkpoints | Fixed at stub creation | Pre-filled from plan |
| Verification Results (section headings + commands) | Fixed at stub creation | Implementing agent fills in command output only; command changes require a `Deviations from Plan` entry |
| Code Review Result | Review agent appends | Not included in stub |

## Code Review Result

- Overall Verdict: FAIL
- Dimension Assessment:
  - Correctness: Fail — a projection call waiting for `broadcastMu` cannot observe caller cancellation, so a periodic refresh queued behind another batch can still delay `EdgeServer.Stop` until that batch releases the lock.
  - Completeness: Fail — per-client sends are bounded, but the serialized batch-admission path remains uncancellable and therefore does not provide the planned end-to-end cancellation bound.
  - Test coverage: Fail — the new tests cover a stalled client inside the active refresh batch, but not cancellation while a refresh is waiting behind another broadcast batch.
  - API contract: Pass — public method signatures and protobuf shapes are unchanged, and the fan-out keeps deterministic Edge-qualified aggregation.
  - Code quality: Pass — the concurrent fan-out, indexed result collection, client close/drain path, and token-fenced disconnect cleanup are otherwise clear and race-safe under the executed suite.
  - Implementation deviation: Fail — the plan requires context cancellation and prompt server stop while preserving serialized batches, but `sync.Mutex.Lock` at batch admission is not context-aware.
  - Verification trust: Fail — all planned focused and wire race tests pass, but a fresh reviewer reproducer canceled a call blocked behind `broadcastMu` and it remained blocked beyond 200ms, contradicting the claimed production cancellation path.
  - Spec conformance: Fail — SDD S02 and S10-S13 require bounded refresh and fail-closed secure delivery; an uncancellable queued refresh can delay shutdown and projection freshness.
- Findings:
  - Required — `apps/control-plane/internal/wire/edge_server.go:331` and `apps/control-plane/internal/wire/edge_server.go:284`: `broadcastProjection` acquires `broadcastMu` with an unconditional `Lock`. If a mutation-triggered broadcast holds the batch lock, the periodic refresh can enter `BroadcastProjection(ctx)` and block before any per-client `select`; `stopProjectionRefresh` cancels that context and waits at `apps/control-plane/internal/wire/edge_server.go:303-304`, but cancellation cannot wake the lock waiter. A focused reviewer reproducer held `broadcastMu`, started `broadcastProjection`, canceled its context, and observed that the call was still blocked after 200ms. Replace batch admission with a context-aware serialization primitive (or an equivalent lifecycle design), check cancellation before snapshot creation, and add a deterministic regression that proves a queued broadcast and `Stop` return promptly after cancellation while whole batches remain serialized.
- Routing Signals: `review_rework_count=5`, `evidence_integrity_failure=true`
- Next Step: Create and execute a freshly routed follow-up PLAN/CODE_REVIEW pair for context-aware projection batch admission and shutdown cancellation, preserving the S10-S13 secure-delivery scope.
