99 lines
5.1 KiB
Text
99 lines
5.1 KiB
Text
<!-- task=m-thin-agent-model-comparison-benchmark plan=1 tag=TEST milestone-task=single-attempt-matrix,minimal-result-table,single-pass-scorecard,bounded-conclusion -->
|
|
|
|
# Code Review Reference - TEST
|
|
|
|
> **[IMPLEMENTING AGENT — READ FIRST]** Fill every implementation-owned section, run the plan verification, paste actual output, and leave this active pair in place. Do not archive files, write `complete.log`, or classify the next state.
|
|
|
|
## Overview
|
|
|
|
date=2026-08-14
|
|
task=m-thin-agent-model-comparison-benchmark, plan=1, tag=TEST
|
|
|
|
## Archive Evidence Snapshot
|
|
|
|
- Replaced unstarted pair: `plan_local_G08_0.log`, `code_review_cloud_G08_0.log`; no prior verdict.
|
|
- Replan closes the missing authenticated catalog call, exact caller/render procedure, and evidence write boundary while preserving the benchmark scope.
|
|
|
|
## For the Review Agent
|
|
|
|
Rerun applicable deterministic checks and inspect immutable external evidence. Append the official verdict only after implementation is submitted. On PASS, archive this pair with suffix `1`, write `complete.log` preserving the first-line metadata, and move the task directory under the dated archive. Roadmap aggregation is a later `sync-milestone-workstate` action.
|
|
|
|
## Implementation Item Completion
|
|
|
|
| Item | Status |
|
|
|---|---|
|
|
| TEST-1 Consume the Immutable Nine-Row Matrix | [ ] |
|
|
| TEST-2 Render, Score Once, and Conclude | [ ] |
|
|
|
|
## Implementation Checklist
|
|
|
|
- [ ] Pass the authenticated catalog and runtime identity gate before creating any producer workspace.
|
|
- [ ] Create nine empty workspaces and execute each fixed caller/model row exactly once, preserving one immutable record per row with no retry/resume/recovery.
|
|
- [ ] Fill the nine-row result table from immutable evidence; record caller-provided usage or `미제공`, never an estimate or substituted zero.
|
|
- [ ] Assign a shuffled opaque ID after all attempts, copy each exact scorable source, and render it exactly once at desktop and mobile viewport.
|
|
- [ ] Score each scorable opaque artifact once with locked anchors and direct source/render evidence, then verify arithmetic.
|
|
- [ ] Write a bounded conclusion comparing only successful scorable results and separating success/time/usage from quality.
|
|
- [ ] Run final attempt-count, render-count, placeholder, retry, secret, arithmetic, and scope checks.
|
|
- [ ] Fill implementation-owned sections in CODE_REVIEW-*-G??.md with actual implementation notes and verification output.
|
|
|
|
## Review-Only Checklist
|
|
|
|
> **[REVIEW AGENT ONLY]** Implementing agents must not modify this checklist.
|
|
|
|
- [ ] Append one verdict with verified routing signals.
|
|
- [ ] Verify verdict, dimensions, and finding severities agree.
|
|
- [ ] Rerun required deterministic verification and inspect the nine immutable attempt ledgers/streams.
|
|
- [ ] Record evidence, root cause, selected fix, files/tests, and acceptance commands for each Required/Suggested finding.
|
|
- [ ] Archive this file to `code_review_cloud_G08_1.log` and the plan to `plan_local_G08_1.log`.
|
|
- [ ] Verify the managed `.gitignore` block and artifact visibility.
|
|
- [ ] On PASS, write `complete.log`, preserve milestone metadata, move the task directory to the dated archive, and update this checklist there.
|
|
- [ ] On WARN/FAIL, create only the next state required by the code-review skill and do not write `complete.log`.
|
|
|
|
## Deviations from Plan
|
|
|
|
_Replace with actual deviations or `None`._
|
|
|
|
## Key Design Decisions
|
|
|
|
_Replace with actual implementation decisions._
|
|
|
|
## Reviewer Checkpoints
|
|
|
|
- Confirm the authenticated catalog body check passed before any producer workspace existed.
|
|
- Confirm the exact expanded command for each of nine rows, one ledger/stream per row, and no hidden caller retry or session continuation.
|
|
- Confirm no product/config/script/manifest/state-store change entered the worktree.
|
|
- Confirm route facts were absent from every opaque scoring directory until scores were frozen.
|
|
- Confirm usage was caller-provided or `미제공`; failures/unscorable artifacts were not converted to zero.
|
|
- Confirm each scorable source has two one-shot renders, direct anchor evidence, and correct arithmetic.
|
|
|
|
## Verification Results
|
|
|
|
### External gate and producer attempts
|
|
|
|
Paste the redacted authenticated gate output, exact expanded commands, sole exit status, and each `attempt.txt`. Do not paste credentials or raw sensitive provider payloads.
|
|
|
|
_Replace with actual output._
|
|
|
|
### Local deterministic checks
|
|
|
|
Run the exact final checks from `PLAN-local-G08.md` and paste stdout/stderr plus exit statuses.
|
|
|
|
_Replace with actual output._
|
|
|
|
### Manual scorecard review
|
|
|
|
Record reviewer arithmetic, anchor/evidence, opaque-blinding, render-count, usage, and bounded-conclusion findings.
|
|
|
|
_Replace with actual findings._
|
|
|
|
---
|
|
|
|
## Section Ownership
|
|
|
|
| Section | Owner | Note |
|
|
|---|---|---|
|
|
| Header, overview, archive snapshot, reviewer instructions | Fixed | Implementer must not modify |
|
|
| Implementation item/checklist status | Implementer | Check only after actual completion |
|
|
| Review-Only Checklist | Review agent | Implementer must not modify |
|
|
| Deviations, decisions, verification results | Implementer, then reviewer | Replace placeholders with actual evidence |
|
|
| Code Review Result | Review agent | Appended only during official review |
|