From 1f8040c74cba8b6d57a37965be4d931e9fb6b12a Mon Sep 17 00:00:00 2001 From: toki Date: Thu, 30 Jul 2026 05:41:05 +0900 Subject: [PATCH] =?UTF-8?q?fix(agent-ops):=20=EC=99=B8=EB=B6=80=20?= =?UTF-8?q?=EC=8B=A4=ED=96=89=20=EC=B0=A8=EB=8B=A8=EC=9D=84=20=EC=82=AC?= =?UTF-8?q?=EC=9A=A9=EC=9E=90=20=EB=A6=AC=EB=B7=B0=EB=A1=9C=20=EC=A0=84?= =?UTF-8?q?=ED=99=98=ED=95=9C=EB=8B=A4?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 자동 실행할 수 없는 외부 검증에서 후속 PLAN을 반복하지 않고 사용자 조치와 재개 조건을 파일 기반 gate로 남기기 위해 정책과 런타임 검증을 맞춘다. --- agent-ops/rules/common/rules-roadmap.md | 2 +- agent-ops/skills/common/code-review/SKILL.md | 45 ++++++---- .../common/code-review/agents/openai.yaml | 2 +- .../templates/complete-log-template.md | 2 +- .../templates/user-review-template.md | 18 ++-- agent-ops/skills/common/plan/SKILL.md | 12 +-- .../orchestrate-agent-task-loop/SKILL.md | 2 +- .../scripts/dispatch.py | 27 ++++-- .../tests/test_dispatch.py | 87 +++++++++++++++++++ 9 files changed, 151 insertions(+), 46 deletions(-) diff --git a/agent-ops/rules/common/rules-roadmap.md b/agent-ops/rules/common/rules-roadmap.md index f2c5dc8..54704a7 100644 --- a/agent-ops/rules/common/rules-roadmap.md +++ b/agent-ops/rules/common/rules-roadmap.md @@ -171,7 +171,7 @@ - SDD 대상 Milestone은 런타임 완료 이벤트의 `complete.log`에 있는 `Roadmap Completion`과 최종 검증 evidence가 SDD `Evidence Map`을 충족해야 roadmap Task 체크 후보가 된다. 단, 사용자가 명시적으로 evidence를 전달한 수동 `update-roadmap` 갱신에서는 Evidence Map 충족 근거를 보조 근거로 사용할 수 있다. - 런타임 완료 이벤트가 최종 archive 경로만 갖고 있으면 `agent-task/archive/YYYY/MM/m-/...`를 `agent-task/m-/...` 형태의 `origin-task`로 정규화해 전달한다. - 런타임 호출에서 매칭되는 활성 Milestone이 없거나 둘 이상이면 추정하지 말고 수동 target 선택이 필요하다고 보고한다. -- `WARN` 또는 `FAIL`은 Milestone 완료 업데이트를 하지 않는다. 일반적으로 같은 `m-` task group에서 후속 계획/리뷰를 이어가지만, code-review의 user-review gate가 트리거되면 연결된 Milestone 잠금 결정을 `USER_REVIEW.md`에 남긴다. +- `WARN` 또는 `FAIL`은 Milestone 완료 업데이트를 하지 않는다. 일반적으로 같은 `m-` task group에서 후속 계획/리뷰를 이어간다. code-review의 `milestone-lock` user-review gate가 트리거되면 연결된 Milestone 잠금 결정을 `USER_REVIEW.md`에 남기고, `external-execution` gate가 트리거되면 Milestone 문서를 바꾸지 않은 채 task-local `USER_REVIEW.md`에 필요한 외부 실행 조치와 재개 조건을 남긴다. - `[스케치]` Milestone은 Milestone 기반 `agent-task` 생성 대상이 아니다. 런타임이나 plan 스킬은 이를 구현 작업으로 라우팅하지 않고 `[계획]` 승격 필요를 보고한다. ## 완료 리뷰 diff --git a/agent-ops/skills/common/code-review/SKILL.md b/agent-ops/skills/common/code-review/SKILL.md index 058ecbb..1e5c71b 100644 --- a/agent-ops/skills/common/code-review/SKILL.md +++ b/agent-ops/skills/common/code-review/SKILL.md @@ -15,38 +15,45 @@ plan skill -> finalize-task-routing -> implementation -> code-review skill +----- WARN/FAIL: invoke plan skill with raw findings -+ ``` -Implementation agents never decide or request user review. They record implementation, verification, deviation, and blocker evidence in implementation-owned review fields. The official code-review agent alone evaluates the selected Milestone state and, when the review-agent-owned gate is justified, writes `USER_REVIEW.md` from `agent-ops/skills/common/code-review/templates/user-review-template.md`. +Implementation agents never decide or request user review. They record implementation, verification, deviation, and blocker evidence in implementation-owned review fields. The official code-review agent alone evaluates the review-agent-owned gate and, when justified, writes `USER_REVIEW.md` from `agent-ops/skills/common/code-review/templates/user-review-template.md`. ## Core Loop Rules - Trigger: Korean or English active-task review requests, including `리뷰 진행해` and `리뷰해줘`, must use this skill when an active `CODE_REVIEW-*-G??.md` or `USER_REVIEW.md` exists under `agent-task/*/` or `agent-task/*/*/`, excluding `agent-task/archive/**`. - Finalize every selected active state: for `CODE_REVIEW-*-G??.md`, append one verdict, prepare the required next state, archive the active review and plan files, then materialize exactly one next state; for `USER_REVIEW.md` completion, update the stop state, write `complete.log`, and archive the task. - Next state: `PASS` writes `complete.log` and moves the task under `agent-task/archive/YYYY/MM/`; if the task group is `m-`, report completion metadata for the runtime event. `WARN` or `FAIL` normally invokes `agent-ops/skills/common/plan/SKILL.md`, which must run `finalize-task-routing` before writing the next active pair; if the user-review gate triggers, write `USER_REVIEW.md` instead. A completed `USER_REVIEW.md` uses the same terminal `complete.log` and archive path as `PASS`. -- The user-review gate is review-agent-owned and triggers only when current repository evidence proves that a concrete selected Milestone `구현 잠금 > 결정 필요` item blocks the next safe implementation step. Generic status fields or blocker text written by implementation are never a user-review request. +- The user-review gate is review-agent-owned and triggers only when current evidence proves either that a concrete selected Milestone `구현 잠금 > 결정 필요` item blocks the next safe implementation step or that required external verification cannot proceed without a user-controlled capability or authorization. Generic status fields or blocker text written by implementation are never a user-review request. - Do not replace `USER_REVIEW.md` with an inline user question. When the user-review gate triggers, write the file-based stop state and report its path. - Do not ask for confirmation before WARN/FAIL follow-up files. If the user-review gate triggers, write `USER_REVIEW.md`; otherwise invoke the plan skill with the current raw findings and let it write the smallest concrete follow-up after fresh routing. - Recovery: if a prior turn appended a verdict without archive or next-state files, do not append another verdict; resume Step 5 preparation/archive from that verdict. If exactly one member of the pair was archived after both archive destinations had been preflighted, verify the archived member and remaining source/destination, finish that archive, then use the post-archive recovery below. If both logs exist with a verdict but the required next state is absent, reconstruct it from those exact logs: PASS resumes `complete.log`; WARN/FAIL reruns the plan skill in `write` mode with raw archived findings and `isolated-reassessment`; a valid user-review gate rerenders `USER_REVIEW.md`. If a prior turn resolved `USER_REVIEW.md` without `complete.log`, resume at the matching finalization step. ## User Review Gate -`USER_REVIEW.md` is a loop stop state only for selected Milestone lock decisions. Default to a normal WARN/FAIL follow-up; the gate requires positive evidence that the blocker is already represented, or must be represented, as a Milestone `구현 잠금 > 결정 필요` item. +`USER_REVIEW.md` is a loop stop state with exactly one of these types: + +- `milestone-lock`: a concrete selected Milestone `구현 잠금 > 결정 필요` item requires a user decision. +- `external-execution`: required verification needs an exact user-controlled runner, device, credential, interactive session, evidence handoff, or explicit authorization that no currently authorized executor can use. + +Apply these rules: - Compute `review-number` as `count(agent-task/{task_name}/code_review_*.log) + 1` before archiving the active review. -- Repeated `WARN`/`FAIL`, loop exhaustion, missing verification evidence, and test environment blockers do not trigger `USER_REVIEW.md` by themselves. Invoke the plan skill for a narrower follow-up or report the non-roadmap blocker as verification evidence that remains unresolved. -- Resolve the selected Milestone from `Roadmap Targets` or another exact active Milestone path already fixed by the task. Read its current `구현 잠금 > 결정 필요` items and independently determine whether one exact unresolved decision blocks the next safe implementation step. -- External environment/secret/service setup, unsupported device, generic scope conflict, agent execution limits, missing handoff evidence, incomplete verification records, arbitrary `상태` text, or anything not tied to that exact Milestone lock never triggers the gate. Archive the review and invoke the plan skill for a normal WARN/FAIL follow-up when implementation can continue, or report the non-user-review blocker. -- `USER_REVIEW.md` is created from `agent-ops/skills/common/code-review/templates/user-review-template.md`, filled with the archived loop history, current archived plan/review paths, verdict, loop count, blocking evidence, connection target, and the exact Milestone decision item. +- Repeated `WARN`/`FAIL`, loop exhaustion, missing verification evidence, and a transient test failure do not trigger `USER_REVIEW.md` by themselves. +- For `milestone-lock`, resolve the selected Milestone from `Roadmap Targets` or another exact active Milestone path already fixed by the task. Read its current `구현 잠금 > 결정 필요` items and require one exact unresolved decision that blocks the next safe implementation step. +- For `external-execution`, first resolve the repository-declared runner, transport, workdir, credentials source, and safe read-only preflight. Use an already authorized configured executor, including SSH or another declared remote runner, when it can perform the step. A current-host OS mismatch, missing local command, closed current-host localhost port, agent execution limit, or incomplete evidence is not enough while such an executor remains usable. +- Trigger `external-execution` only when the required target and attempted routing/preflight are concrete, the next verification step is required for the verdict, no authorized automatic route can perform it, and progress requires a user to grant access or authorization, prepare or operate a user-controlled environment, or supply the required evidence. Do not create another follow-up PLAN that repeats the same inaccessible preflight. +- Generic scope conflict, missing optional handoff evidence, arbitrary `상태` text, and repository-fixable setup remain normal WARN/FAIL follow-up inputs. +- Create `USER_REVIEW.md` from `agent-ops/skills/common/code-review/templates/user-review-template.md`. Fill the archived loop history, current archived plan/review paths, verdict, loop count, blocking evidence, exact target, one gate type, required user action or decision, and resume condition. ## User Review Resolution -When an active `USER_REVIEW.md` exists and the linked Milestone decision closes the task as complete/PASS, finalization is still owned by this skill. +When an active `USER_REVIEW.md` exists and its recorded user action or decision closes the task as complete/PASS, finalization is still owned by this skill. - Read `USER_REVIEW.md`, archived `plan_*.log`, and archived `code_review_*.log` in that task directory. -- Verify the linked decision and any follow-up evidence are sufficient to close the task. If a new implementation plan is needed instead, do not close; route back to the plan skill, which archives `USER_REVIEW.md` to `user_review_N.log` before writing a new plan. -- Update `USER_REVIEW.md` in place to show a resolved state, final verdict, loop history, fulfilled decision items, and the evidence that closed the stop state. +- Verify the recorded action or decision and any follow-up evidence are sufficient to close the task. For `external-execution`, access or authorization that merely enables verification normally resumes through a new plan; user-supplied final evidence may close the task only when it satisfies the archived acceptance criteria. If a new implementation or verification plan is needed, do not close; route back to the plan skill, which archives `USER_REVIEW.md` to `user_review_N.log` before writing a new plan. +- Update `USER_REVIEW.md` in place to show a resolved state, final verdict, loop history, fulfilled user actions or decisions, and the evidence that closed the stop state. - Write `complete.log` from `agent-ops/skills/common/code-review/templates/complete-log-template.md` before moving or archiving the task artifacts. Include both the original archived review verdict and the user-review resolution line in `루프 이력`. - Then apply the same task-directory archive move and `m-` PASS completion metadata rules as a normal `PASS`. -- Do not leave an active task directory that contains `USER_REVIEW.md` and `*.log` files but no `complete.log` after the linked Milestone decision resolves the task as complete/PASS. +- Do not leave an active task directory that contains `USER_REVIEW.md` and `*.log` files but no `complete.log` after the recorded action or decision resolves the task as complete/PASS. ## Workflow Contract @@ -105,7 +112,7 @@ Directory states: | `PLAN-*-G??.md` + `CODE_REVIEW-*-G??.md` with appended verdict | Review finalization pending; do not append another verdict, resume Step 5 preparation/archive | | Exactly one active pair member + its newly archived counterpart | Partial archive after a preflighted finalization; verify both identities, finish the remaining archive, then resume post-archive recovery | | `complete.log` + `*.log` files | Task complete (PASS or user-review-resolved PASS), before final task-directory archive move | -| `USER_REVIEW.md` + `*.log` files | Automatic loop stopped; linked Milestone lock decision must be resolved before creating another plan | +| `USER_REVIEW.md` + `*.log` files | Automatic loop stopped; its recorded Milestone decision or external-execution user action must be resolved before creating another plan | | `agent-task/archive/YYYY/MM/{task_name}/complete.log` + `*.log` files | Archived completed task path (PASS or user-review-resolved PASS); not active | | Only `*.log` files (no `complete.log`) | If the newest review log has a verdict and its required next state is absent, post-archive finalization is pending; otherwise the task is terminated mid-loop or abandoned | @@ -128,8 +135,8 @@ Classify the combined set of active `CODE_REVIEW-*-G??.md` and `USER_REVIEW.md` | Result | Action | |--------|--------| | Exactly one active path and it is `CODE_REVIEW-*-G??.md` | Review that task; exactly one `PLAN-*-G??.md` is normally expected beside it. If the review already has a verdict and its exact plan counterpart was just archived, use partial-archive recovery instead of reporting a missing plan. | -| Exactly one active path and it is `USER_REVIEW.md`, with a linked Milestone completion/resolution decision | Perform User Review Resolution for that task. | -| One or more active paths and every active path is `USER_REVIEW.md`, with no completion/resolution decision | Report that the linked Milestone decision is required and list the paths. | +| Exactly one active path and it is `USER_REVIEW.md`, with a recorded user action/decision resolution | Perform User Review Resolution for that task. | +| One or more active paths and every active path is `USER_REVIEW.md`, with no action/decision resolution | Report the required user action or decision and list the paths. | | No active paths | Apply the finalization-recovery scan below; stop only when it finds no recoverable task. | | Multiple active paths | If the user/runtime named a task group, task path, or subtask directory that identifies exactly one active path, use that directory. Otherwise list paths and stop with an ambiguity report; do not choose by agent judgment and do not create a user-review request for routing ambiguity. | @@ -169,7 +176,7 @@ Before writing the verdict: - If a checklist item contains integrated verification for a feature, treat that feature item as incomplete until both implementation evidence and the matching verification output are present. Do not accept a separate unchecked completion-criteria item as a substitute. - Confirm the implementation marked the matching checklist items in the active review file, including the mandatory `CODE_REVIEW-*-G??.md` evidence item; repair clear artifact drift when evidence supports completion. - Treat review artifact gaps as failures only when they prevent judging implementation correctness, tests, contracts, or verification trust. -- Treat every generic `상태` field and implementation blocker record as ordinary evidence, never as a request to stop for the user. Evaluate the user-review gate independently from the current selected Milestone only after a WARN/FAIL finding requires a next state. +- Treat every generic `상태` field and implementation blocker record as ordinary evidence, never as a request to stop for the user. Evaluate both user-review gate types independently only after a WARN/FAIL finding requires a next state. - Grep renamed/removed symbols for stale references. - Confirm every required test exists, name matches, and assertions are meaningful. - Cross-check claimed verification output in the active review file against actual code and project commands. @@ -257,7 +264,7 @@ Complete log template: For `WARN` or `FAIL`, materialize the next state prepared in Step 5 immediately after archive: -- If the user-review gate triggered, write the prepared body to `agent-task/{task_name}/USER_REVIEW.md`. It must use only `milestone-lock`, contain every archived loop entry plus the exact linked decision, and contain no placeholder. Do not write active PLAN/CODE_REVIEW files or `complete.log`. +- If the user-review gate triggered, write the prepared body to `agent-task/{task_name}/USER_REVIEW.md`. It must use exactly one supported type, `milestone-lock` or `external-execution`, contain every archived loop entry plus the exact required user action or decision, and contain no placeholder. Do not write active PLAN/CODE_REVIEW files or `complete.log`. - Otherwise write `prepared_plan` and `prepared_review` byte-for-byte to their routed basenames. Do not rerun, adjust, compare, or upgrade their lane/G after archive. - Verify the written follow-up pair contains the predicted archived plan/review paths in identical `Archive Evidence Snapshot` sections and contains no unresolved token from the review-stub template inventory. Unrelated braces in commands or code are allowed. - Re-run `dispatch.py --workspace --validate-plan ` and require exit code `0` to confirm that the byte-for-byte materialized PLAN retained the validated write claim. @@ -275,8 +282,8 @@ After Step 6: - If verdict is `PASS` and `{task_group}` matches `m-`, do not resolve the roadmap target and do not call `update-roadmap`. Report completion event metadata after the task archive move: `origin-task=agent-task/{task_name}` from the original active task path, `task-group={task_group}`, `milestone-slug=`, final archive path, `complete.log` path, archived plan/review log paths, and `roadmap-completion=`. - The runtime consumes that completion event, checks current state, and calls `update-roadmap` if needed. `update-roadmap` only checks Milestone Task ids when `complete.log` contains `Roadmap Completion`; if the section is absent, roadmap Task completion is a no-op even for `m-*` task groups. - `WARN` and `FAIL` do not update the roadmap Milestone; the follow-up plan remains under the same `m-` task group when the original task was Milestone-linked. -- `USER_REVIEW` does not update the roadmap Milestone and does not produce PASS completion metadata. Keep the active task directory in place with `USER_REVIEW.md` and archived plan/review logs until the linked Milestone lock decision is resolved. -- If `USER_REVIEW.md` is later resolved as complete/PASS by linked Milestone decision and evidence, write `complete.log`, move the task artifacts to archive, and report `m-*` PASS completion metadata just like a normal `PASS`. +- `USER_REVIEW` does not update the roadmap Milestone and does not produce PASS completion metadata. Keep the active task directory in place with `USER_REVIEW.md` and archived plan/review logs until its recorded user action or decision is resolved. +- If `USER_REVIEW.md` is later resolved as complete/PASS by the recorded action or decision and evidence, write `complete.log`, move the task artifacts to archive, and report `m-*` PASS completion metadata just like a normal `PASS`. - For `PASS`, open the moved `agent-task/archive/YYYY/MM/{final_task_name}/{current_review_archive_name}`, where `{final_task_name}` is the archived task path, including `{task_group}/` for split work. - For user-review-resolved PASS, confirm the moved archive contains the resolved `USER_REVIEW.md`, `complete.log`, and the existing archived `plan_*.log` / `code_review_*.log`; do not recreate an active review file only to add a new checklist item. - For `WARN` or `FAIL`, open `agent-task/{task_name}/{current_review_archive_name}`. @@ -324,6 +331,6 @@ Report Required/Suggested counts, archive names, the final task archive path for - WARN/FAIL follow-up: the plan input omitted prior route fields, revalidated outcome/acceptance/exclusions from current evidence, used the completed in-memory PLAN as the packet, and copied identical `Archive Evidence Snapshot` sections into the new plan/review pair. - Follow-up plans and review stubs keep implementation agents limited to implementation/test/evidence and contain no implementation-owned user-review request section. - USER_REVIEW: `USER_REVIEW.md` exists from template, no active `PLAN-*.md` or `CODE_REVIEW-*.md` remains, and no `complete.log` was written. -- Review-agent-owned USER_REVIEW: the generated `USER_REVIEW.md` records the exact active Milestone decision and repository evidence that made automatic continuation unsafe. +- Review-agent-owned USER_REVIEW: the generated `USER_REVIEW.md` records one supported gate type, the exact Milestone decision or external-execution user action, and evidence that made automatic continuation unsafe. - USER_REVIEW resolved as PASS: archived task contains both resolved `USER_REVIEW.md` and `complete.log`. - The applicable review-agent-only finalization checklist was completed before reporting. diff --git a/agent-ops/skills/common/code-review/agents/openai.yaml b/agent-ops/skills/common/code-review/agents/openai.yaml index 1ebc3a3..f053bc0 100644 --- a/agent-ops/skills/common/code-review/agents/openai.yaml +++ b/agent-ops/skills/common/code-review/agents/openai.yaml @@ -1,4 +1,4 @@ interface: display_name: "Code Review" short_description: "Review and route task loops" - default_prompt: "Use $code-review to review or resolve the active task, archive it, and on WARN/FAIL invoke $plan so $finalize-task-routing creates the next routed pair." + default_prompt: "Use $code-review to review or resolve the active task, archive it, and on WARN/FAIL either create a justified milestone-lock/external-execution USER_REVIEW stop or invoke $plan for the next routed pair." diff --git a/agent-ops/skills/common/code-review/templates/complete-log-template.md b/agent-ops/skills/common/code-review/templates/complete-log-template.md index 0875439..ea9b9da 100644 --- a/agent-ops/skills/common/code-review/templates/complete-log-template.md +++ b/agent-ops/skills/common/code-review/templates/complete-log-template.md @@ -13,7 +13,7 @@ | Plan | Review | Verdict | 메모 | |------|--------|---------|------| | `plan_{build_lane}_GNN_N.log` | `code_review_{review_lane}_GNN_N.log` | PASS/WARN/FAIL | {main outcome or follow-up reason} | -| `USER_REVIEW.md` | linked Milestone decision | PASS/RESOLVED | {only when a user-review stop was resolved as the terminal completion path; remove this row otherwise} | +| `USER_REVIEW.md` | recorded user action or decision | PASS/RESOLVED | {only when a user-review stop was resolved as the terminal completion path; remove this row otherwise} | ## 구현/정리 내용 diff --git a/agent-ops/skills/common/code-review/templates/user-review-template.md b/agent-ops/skills/common/code-review/templates/user-review-template.md index e7a968a..cdf9d56 100644 --- a/agent-ops/skills/common/code-review/templates/user-review-template.md +++ b/agent-ops/skills/common/code-review/templates/user-review-template.md @@ -10,11 +10,11 @@ USER_REVIEW ## 사유 -- 유형: milestone-lock -- 연결 대상: {agent-roadmap/phase//milestones/.md} +- 유형: {milestone-lock | external-execution} +- 연결 대상: {agent-roadmap/phase//milestones/.md | exact runner/device/service/access target} - 현재 리뷰 회차: {review-number} - 최종 판정: {WARN or FAIL} -- 요약: {Milestone 구현 잠금 항목이 실구현을 차단한 이유} +- 요약: {Milestone 결정 또는 user-controlled external execution이 다음 안전한 단계를 차단한 이유} ## 루프 이력 @@ -30,21 +30,21 @@ USER_REVIEW - 현재 archive review: `{current-archived-review-log}` - 검증 명령: `{command or 없음}` - 실제 출력: {stdout/stderr excerpt or saved output path} -- 차단 판단 근거: {연결 대상의 Milestone 결정 필요 항목과 일치하는 근거} +- 차단 판단 근거: {Milestone 결정과 일치하는 근거 | declared runner/transport를 확인하고도 자동 실행할 수 없으며 사용자 조치가 필요한 근거} -## 연결 결정 필요 +## 사용자 조치 또는 결정 -- [ ] {Milestone `구현 잠금 > 결정 필요`에 기록된 결정 항목} +- [ ] {Milestone `구현 잠금 > 결정 필요` 항목 | exact access/authorization/environment/evidence action} ## 재개 조건 -- {위 결정이 Milestone 구현 잠금에 반영되고, 필요한 경우 해당 USER_REVIEW.md가 user_review_N.log로 해결되어야 한다} +- {위 사용자 조치 또는 결정이 충족되었음을 확인하는 구체적인 evidence와 후속 review/plan 진입 조건} ## 다음 실행 힌트 -- {resolve-review 또는 update-roadmap으로 반영할 대상 경로와 후속 plan/review 재개 조건} +- {resolve-review, update-roadmap, external verification replan 중 맞는 진입점과 대상 경로} ## 종료 규칙 -- 연결 결정과 evidence가 이 stop state를 완료/PASS로 해소하면 `USER_REVIEW.md`를 해소 상태로 갱신하고, `agent-ops/skills/common/code-review/templates/complete-log-template.md` 기준 `complete.log`를 작성한 뒤 task directory를 archive로 이동한다. +- 기록된 사용자 조치 또는 결정과 evidence가 이 stop state를 완료/PASS로 해소하면 `USER_REVIEW.md`를 해소 상태로 갱신하고, `agent-ops/skills/common/code-review/templates/complete-log-template.md` 기준 `complete.log`를 작성한 뒤 task directory를 archive로 이동한다. - 새 구현이 필요하면 `plan` 스킬이 `USER_REVIEW.md`를 `user_review_N.log`로 아카이브한 뒤 새 `PLAN-*-G??.md` / `CODE_REVIEW-*-G??.md`를 작성한다. diff --git a/agent-ops/skills/common/plan/SKILL.md b/agent-ops/skills/common/plan/SKILL.md index 06355e7..69276bf 100644 --- a/agent-ops/skills/common/plan/SKILL.md +++ b/agent-ops/skills/common/plan/SKILL.md @@ -16,7 +16,7 @@ code-review skill -> verdict + archive, complete.log and task-directory archive runtime -> for m-prefixed PASS completion events, state check and optional update-roadmap call ``` -`code-review` may stop the automatic loop with `USER_REVIEW.md` only when a selected Milestone `구현 잠금 > 결정 필요` item blocks implementation. Repeated non-PASS reviews, test environment blockers, external environment/secret/service setup, generic scope changes, and verification evidence gaps are not user-review reasons by themselves; they should become normal follow-up plans or unresolved verification evidence. Plan creation after `USER_REVIEW.md` requires the linked Milestone decision to be resolved. If the decision closes the task as complete/PASS, code-review resolves `USER_REVIEW.md`, writes `complete.log`, and archives the task instead of creating a new plan. +`code-review` may stop the automatic loop with `USER_REVIEW.md` for either a selected Milestone `구현 잠금 > 결정 필요` item (`milestone-lock`) or required external verification that cannot proceed without a user-controlled runner, device, credential, interactive session, evidence handoff, or explicit authorization (`external-execution`). A current-host mismatch or missing command is not enough when a repository-declared runner or authorized executor can perform the step automatically. Repeated non-PASS reviews and missing evidence are not user-review reasons by themselves. Plan creation after `USER_REVIEW.md` requires its recorded user action or decision to be resolved. If that resolution closes the task as complete/PASS, code-review writes `complete.log` and archives the task instead of creating a new plan. The plan file and review stub must be self-sufficient for implementation agents that do not read this skill. Implementing agents fill the active review's implementation evidence. They never decide whether user review is needed, write a user-review request, ask the user for a decision, create `USER_REVIEW.md`, or modify runtime-owned artifacts; those control-plane responsibilities belong to the official code-review skill and runtime. @@ -57,7 +57,7 @@ Role boundary rules: - Implementing agents fill implementation-owned `CODE_REVIEW-*-G??.md` sections, keep active files in place, and report ready for review. - If implementation cannot continue, implementing agents record the exact blocker, attempted commands/output, and resume condition only in `Verification Results` or `Deviations from Plan` (legacy: `검증 결과` or `계획 대비 변경 사항`), then leave the active files in place for official review. - During implementation, do not ask the user directly, present choices, call user-input tools, or create control-plane stop files. The official reviewer owns all next-state classification. -- Required UI evidence capture that needs a user-owned device, emulator, permission, secret, or interactive access unavailable to the agent is a verification blocker, not a user-review reason by itself. Record attempted commands and blocker evidence in `Verification Results` or `Deviations from Plan` (legacy: `검증 결과` or `계획 대비 변경 사항`) so code-review can write a normal follow-up or unresolved verification report. +- Required evidence capture that needs a user-owned runner, device, emulator, permission, secret, interactive access, evidence handoff, or explicit external authorization unavailable to the agent is a verification blocker for the implementing agent. Record the declared execution target, attempted routing/preflight commands, actual output, authorization state, and resume condition in `Verification Results` or `Deviations from Plan` (legacy: `검증 결과` or `계획 대비 변경 사항`) so the official reviewer can evaluate the `external-execution` gate. - Finalization (`Code Review Result` [legacy: `코드리뷰 결과`], plan/review log rename, `complete.log`, task artifact archive moves, review-only checklist) is code-review-skill only. Split decision policy: @@ -122,7 +122,7 @@ Directory states: | `PLAN-*-G??.md` + filled `CODE_REVIEW-*-G??.md` without verdict | Ready for code-review skill | | `PLAN-*-G??.md` + `CODE_REVIEW-*-G??.md` with appended verdict | Review finalization is pending. Continue only when code-review invokes `prepare-follow-up`; `write` mode must not overwrite this state. | | `complete.log` + `*.log` files | Task complete (PASS or user-review-resolved PASS), before final task-directory archive move | -| `USER_REVIEW.md` + `*.log` files | Automatic loop stopped; linked Milestone lock decision must be resolved before creating another plan | +| `USER_REVIEW.md` + `*.log` files | Automatic loop stopped; its recorded Milestone decision or external-execution user action must be resolved before creating another plan | | `agent-task/archive/YYYY/MM/{task_name}/complete.log` + `*.log` files | Archived completed task path (PASS or user-review-resolved PASS); not active | | Only `*.log` files (no `complete.log`) | Inspect the newest review log. A verdict with no required next state is post-archive finalization pending; otherwise the task is terminated mid-loop or abandoned. | @@ -148,7 +148,7 @@ Also note active user-review stops, excluding `agent-task/archive/**`: The routed plan file is the loop entry point. A missing active plan normally means only that no plan has been started for a new task; do not create task files for casual analysis, status, or review requests unless the user explicitly asks for a plan. -If no active plan exists but one or more `USER_REVIEW.md` files exist, report that the linked Milestone decision is required and list the paths unless one path is explicitly selected for resolution or replanning. If a selected active task directory contains `USER_REVIEW.md`, read it before planning. Do not write a new follow-up plan unless the linked Milestone decision has been resolved or the new plan explicitly replans around that recorded decision. When planning resumes from `USER_REVIEW.md`, archive it to `user_review_N.log` in the same task directory before writing the new active plan/review pair, and record the resolved decision in the new plan `Background` or `Analysis` (legacy: `배경` or `분석 결과`). +If no active plan exists but one or more `USER_REVIEW.md` files exist, report the recorded user action or decision and list the paths unless one path is explicitly selected for resolution or replanning. If a selected active task directory contains `USER_REVIEW.md`, read it before planning. Do not write a new follow-up plan unless the recorded action or decision has been resolved or the new plan explicitly replans around that resolution. When planning resumes from `USER_REVIEW.md`, archive it to `user_review_N.log` in the same task directory before writing the new active plan/review pair, and record the resolved action or decision in the new plan `Background` or `Analysis` (legacy: `배경` or `분석 결과`). If a selected task directory contains both `USER_REVIEW.md` and active `PLAN-*-G??.md` or `CODE_REVIEW-*-G??.md`, report an inconsistent loop state and do not overwrite either state until a later explicit command selects either user-review resolution or the active plan/review path. @@ -221,7 +221,7 @@ In `write` mode, complete this step before writing new active files. In `prepare - In `write` mode, ensure `.gitignore` has the Agent-Ops managed gitignore block before renaming any active file to `*.log` or creating local `agent-roadmap/current.md`. Prefer `source agent-ops/bin/ai-ignore.sh && agent_ops_ensure_gitignore_task_artifact_block .gitignore`; if the helper is unavailable, add or update a block containing `!agent-task/`, `!agent-task/**/`, `!agent-task/**/*.md`, `!agent-task/**/*.log`, and `agent-roadmap/current.md`. In `prepare-follow-up` mode, only inspect the block and return `gitignore_repair_needed: true|false`; code-review performs any repair after preparation. - Count existing `plan_*.log` as `current_plan_archive_number`. If an active plan exists, parse `current_build_lane` and `current_build_grade` from that active filename and set `current_plan_archive_name=plan_{current_build_lane}_{current_build_grade}_{current_plan_archive_number}.log`. Require that destination not to exist. In `write` mode rename that exact active file to the calculated name; in `prepare-follow-up` only return the name and number. Never use the newly routed build lane/grade to archive the old active plan. - Count existing `code_review_*.log` as `current_review_archive_number`. If an active review exists, parse `current_review_lane` and `current_review_grade` from that active filename and set `current_review_archive_name=code_review_{current_review_lane}_{current_review_grade}_{current_review_archive_number}.log`. Require that destination not to exist. In `write` mode rename that exact active file to the calculated name; in `prepare-follow-up` only return the name and number. Never use the newly routed review lane/grade to archive the old active review. -- Count existing `user_review_*.log` as `current_user_review_archive_number`. If `USER_REVIEW.md` exists and the linked Milestone decision is resolved for replanning, set `current_user_review_archive_name=user_review_{current_user_review_archive_number}.log`; require that destination not to exist and rename it only in `write` mode. +- Count existing `user_review_*.log` as `current_user_review_archive_number`. If `USER_REVIEW.md` exists and its recorded user action or decision is resolved for replanning, set `current_user_review_archive_name=user_review_{current_user_review_archive_number}.log`; require that destination not to exist and rename it only in `write` mode. - Compute `post_archive_plan_log_count`, `post_archive_review_log_count`, and `post_archive_user_review_log_count` from the filesystem after actual archive in `write` mode, or from the predicted addition of each current active file in `prepare-follow-up` mode. Set `plan_number=post_archive_plan_log_count`. The new pair's future archive suffixes are `plan_log_number=post_archive_plan_log_count` and `review_log_number=post_archive_review_log_count`. These are intentionally different concepts from `current_plan_archive_number` and `current_review_archive_number`. @@ -351,7 +351,7 @@ Do not write or return a prepared pair when either routing target is not `routed - Both first lines match ``. - The review stub was rendered from `agent-ops/skills/common/plan/templates/review-stub-template.md` after routing and has no unresolved known template token. - In `write` mode, previous active files, if any, were archived with lane/grade parsed from their own basenames and the correct current archive suffixes. In `prepare-follow-up` mode, those archive names were only predicted. -- In `write` mode when resuming from `USER_REVIEW.md`, it was archived to the calculated `current_user_review_archive_name` and the resolved linked Milestone decision was recorded in the new plan. +- In `write` mode when resuming from `USER_REVIEW.md`, it was archived to the calculated `current_user_review_archive_name` and the resolved user action or decision was recorded in the new plan. - `Roadmap Targets` exists only when PASS should check explicit Milestone Task ids, the task group is `m-` for the listed Milestone path, and every listed Task id exists in the selected active Milestone. - If `Roadmap Targets` exists in the plan, the review stub contains the identical section. If it does not exist in the plan, the review stub omits it too. - If the selected Milestone has `SDD: 필요`, the plan's `Analysis > SDD Criteria` (legacy: `분석 결과 > SDD 기준`) proves that the implementation checklist and final verification were derived from the approved SDD Acceptance Scenarios and Evidence Map. Missing SDD mapping blocks plan creation. diff --git a/agent-ops/skills/project/orchestrate-agent-task-loop/SKILL.md b/agent-ops/skills/project/orchestrate-agent-task-loop/SKILL.md index 638aa93..d9d1d67 100644 --- a/agent-ops/skills/project/orchestrate-agent-task-loop/SKILL.md +++ b/agent-ops/skills/project/orchestrate-agent-task-loop/SKILL.md @@ -151,7 +151,7 @@ When recovering a KST-night `local-G07`–`local-G08` Laguna locator or a termin - While one task recovers or becomes blocked, continue every ready/running task that neither requires it as a predecessor nor collides with its retained workspace claim. Internal recovery or blocking must not trigger an arbitrary complete-candidate rescan. - If review shared-state preflight fails, block only ready review tasks and still start every worker/self-check with a disjoint claim in the same pass. The complete scan after `complete.log` must preserve the existing snapshot rather than reread already running task directories, avoiding races with parallel archive moves that could stop another process. - For KST-night `local-G07`–`local-G08` Laguna locator `context-limit`/`session-stall`, prefer the Prompt Contract's same-session resume and display `Pi세션연속재시작`. Use a fresh session and `세션응답복구재시도` only for other legacy Pi `session-stall` recovery. -- Do not stop for user review based on filename alone. Recognize a `user-review` terminal blocker only when the active task's `USER_REVIEW.md` contains `상태: USER_REVIEW`, `유형: milestone-lock`, a real `agent-roadmap/**/milestones/*.md` target, non-`없음`/`미정` blocker rationale, unresolved decisions, and resume conditions that prevent the next safe implementation step. If the form is incomplete or conflicts with active PLAN/CODE_REVIEW, block it as a task-state contract error instead. +- Do not stop for user review based on filename alone. Recognize a `user-review` terminal blocker only when the active task's `USER_REVIEW.md` contains `상태: USER_REVIEW`, exactly one supported type, a concrete target, non-`없음`/`미정` blocker rationale, unresolved user actions or decisions, and resume conditions that prevent the next safe implementation step. For `milestone-lock`, require a real `agent-roadmap/**/milestones/*.md` target. For `external-execution`, require an exact runner/device/service/access target and evidence that no authorized automatic executor can perform the required verification. If the form is incomplete or conflicts with active PLAN/CODE_REVIEW, block it as a task-state contract error instead. - Recognize `## Code Review Result` (with `Overall Verdict: PASS|WARN|FAIL`) or legacy `## 코드리뷰 결과` (with `종합 판정: PASS|WARN|FAIL`) as the review verdict. If both canonical and legacy headings are present in the same file, fail closed. Never parse the same string in implementation evidence, command output, or example text as the runtime verdict. - Locator/raw logs under `.git/agent-task-dispatcher/runs/` are internal recovery state and may not appear in the normal project tree. Include the `locator=` path emitted when the dispatcher starts an attempt and the task-group `WORK_LOG.md` path in status updates. - If a specified `task_group` has neither an observed active task nor a persisted completed task, return state error `unobserved-task-group` with exit code `2`; never treat it as empty completion. diff --git a/agent-ops/skills/project/orchestrate-agent-task-loop/scripts/dispatch.py b/agent-ops/skills/project/orchestrate-agent-task-loop/scripts/dispatch.py index 7fa6010..41a408b 100644 --- a/agent-ops/skills/project/orchestrate-agent-task-loop/scripts/dispatch.py +++ b/agent-ops/skills/project/orchestrate-agent-task-loop/scripts/dispatch.py @@ -2259,13 +2259,19 @@ def user_review_blocker_state(path: Path) -> tuple[bool, str]: if status != "USER_REVIEW": return False, "상태가 USER_REVIEW가 아니다" reason = markdown_section(text, "사유") - if not re.search(r"(?m)^-\s*유형:\s*milestone-lock\s*$", reason): - return False, "milestone-lock 유형이 아니다" + gate_type_matches = re.findall( + r"(?m)^-\s*유형:\s*(milestone-lock|external-execution)\s*$", + reason, + ) + if len(gate_type_matches) != 1: + return False, "지원하는 user-review 유형이 정확히 하나가 아니다" + gate_type = gate_type_matches[0] target = re.search(r"(?m)^-\s*연결 대상:\s*(.+?)\s*$", reason) target_value = target.group(1) if target else "" - if ( - not concrete_user_review_value(target_value) - or "agent-roadmap/" not in target_value + if not concrete_user_review_value(target_value): + return False, "구체적인 연결 대상이 없다" + if gate_type == "milestone-lock" and ( + "agent-roadmap/" not in target_value or "/milestones/" not in target_value or ".md" not in target_value ): @@ -2276,14 +2282,19 @@ def user_review_blocker_state(path: Path) -> tuple[bool, str]: evidence_line.group(1) ): return False, "구체적인 차단 판단 근거가 없다" - decision = markdown_section(text, "연결 결정 필요") + decision = markdown_section(text, "사용자 조치 또는 결정") + legacy_decision = markdown_section(text, "연결 결정 필요") + if decision.strip() and legacy_decision.strip(): + return False, "사용자 조치 또는 결정 섹션이 중복됐다" + if not decision.strip(): + decision = legacy_decision unresolved = [ value for value in re.findall(r"(?m)^-\s*\[\s\]\s+(.+?)\s*$", decision) if concrete_user_review_value(value) ] if not unresolved: - return False, "미해결 연결 결정 항목이 없다" + return False, "미해결 사용자 조치 또는 결정 항목이 없다" resume = markdown_section(text, "재개 조건") resume_conditions = [ line @@ -2292,7 +2303,7 @@ def user_review_blocker_state(path: Path) -> tuple[bool, str]: ] if not resume_conditions: return False, "구체적인 재개 조건이 없다" - return True, "unresolved milestone-lock decision" + return True, f"unresolved {gate_type} user action or decision" def task_stage(task: Task, state: dict[str, Any]) -> str: diff --git a/agent-ops/skills/project/orchestrate-agent-task-loop/tests/test_dispatch.py b/agent-ops/skills/project/orchestrate-agent-task-loop/tests/test_dispatch.py index 0618046..ef9ecc3 100644 --- a/agent-ops/skills/project/orchestrate-agent-task-loop/tests/test_dispatch.py +++ b/agent-ops/skills/project/orchestrate-agent-task-loop/tests/test_dispatch.py @@ -163,6 +163,22 @@ class TaskStageTest(unittest.TestCase): "- Milestone 결정 반영 후 재개\n" ) + @staticmethod + def blocking_external_user_review_text(): + return ( + "# User Review Required - test\n\n" + "## 상태\n\nUSER_REVIEW\n\n" + "## 사유\n\n" + "- 유형: external-execution\n" + "- 연결 대상: ssh toki@toki-labs.com:/Users/toki/agent-work/iop-dev\n\n" + "## 차단 근거\n\n" + "- 차단 판단 근거: Required dev smoke needs a user-controlled runner and no authorized SSH credential is available.\n\n" + "## 사용자 조치 또는 결정\n\n" + "- [ ] Grant runner access or provide the required sanitized smoke evidence.\n\n" + "## 재개 조건\n\n" + "- Verify SSH access or the supplied evidence before resuming review.\n" + ) + def test_default_or_arbitrary_status_text_does_not_start_review(self): with tempfile.TemporaryDirectory() as temporary: root = Path(temporary) @@ -206,6 +222,53 @@ class TaskStageTest(unittest.TestCase): task.recovery = True self.assertEqual(dispatch.task_stage(task, {}), "user-review") + def test_external_execution_user_review_stops_the_loop(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + task = self.make_task(root) + task.user_review = root / "USER_REVIEW.md" + task.user_review.write_text( + self.blocking_external_user_review_text(), encoding="utf-8" + ) + task.plan = None + task.review = None + task.recovery = True + self.assertEqual(dispatch.task_stage(task, {}), "user-review") + + def test_external_execution_user_review_requires_concrete_target(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + task = self.make_task(root) + task.user_review = root / "USER_REVIEW.md" + task.user_review.write_text( + self.blocking_external_user_review_text().replace( + "ssh toki@toki-labs.com:/Users/toki/agent-work/iop-dev", + "없음", + ), + encoding="utf-8", + ) + task.plan = None + task.review = None + task.recovery = True + self.assertEqual(dispatch.task_stage(task, {}), "blocked") + + def test_user_review_rejects_multiple_gate_types(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + task = self.make_task(root) + task.user_review = root / "USER_REVIEW.md" + task.user_review.write_text( + self.blocking_external_user_review_text().replace( + "- 유형: external-execution\n", + "- 유형: external-execution\n- 유형: milestone-lock\n", + ), + encoding="utf-8", + ) + task.plan = None + task.review = None + task.recovery = True + self.assertEqual(dispatch.task_stage(task, {}), "blocked") + def test_user_review_without_blocking_contract_is_state_blocked(self): with tempfile.TemporaryDirectory() as temporary: root = Path(temporary) @@ -11025,6 +11088,30 @@ class ArtifactLanguageContractTest(unittest.TestCase): } return {name: path.read_text(encoding="utf-8") for name, path in paths.items()} + def test_external_execution_user_review_contract_is_shared(self): + documents = self.contract_documents() + skills_root = Path(__file__).resolve().parents[3] + user_review_template = ( + skills_root + / "common" + / "code-review" + / "templates" + / "user-review-template.md" + ).read_text(encoding="utf-8") + + self.assertIn("`external-execution`", documents["plan_skill"]) + self.assertIn("`external-execution`", documents["review_skill"]) + self.assertIn("For `external-execution`", documents["orchestrator_skill"]) + self.assertIn( + "Do not create another follow-up PLAN that repeats the same inaccessible preflight.", + documents["review_skill"], + ) + self.assertIn( + "{milestone-lock | external-execution}", + user_review_template, + ) + self.assertIn("## 사용자 조치 또는 결정", user_review_template) + def test_templates_and_prompts_separate_artifact_and_final_languages(self): documents = self.contract_documents() template = documents["review_template"]