feat(single_request): implement bounded Plan and Review Markdown templates
This commit is contained in:
parent
8a8ea29abd
commit
b6e0080fe5
18 changed files with 1337 additions and 192 deletions
|
|
@ -65,7 +65,7 @@ tracked config에는 public 예시와 기본 구조만 두고, 실제 endpoint/c
|
|||
- `models[].providers`와 `models[].execution_preset`는 상호 배타(one-of)다. 한 `models[]` entry는 정확히 하나만 설정해야 하며, 둘 다 설정하거나 둘 다 비우면 load에서 거부한다. `execution_preset`가 설정된 entry는 provider pool을 갖지 않는 virtual(preset-only) model이며 named execution preset shape에 실행을 위임한다. provider-only budget/token-counter validation은 virtual entry에 적용하지 않는다.
|
||||
- `models[].execution_preset` 값은 앞뒤 공백을 제거해 정규화한다. 공백만 있는 값은 unset으로 처리해 provider-only one-of 규칙을 적용하고, 정규화된 non-empty id는 `execution_presets[]` catalog의 entry로 resolve되어야 한다. dangling reference는 fail-closed로 거부한다. resolve에 성공한 non-empty id는 canonical(trimmed) 형태로 저장되어 downstream lookup이 admission 시점 값과 정확히 일치한다.
|
||||
- `execution_presets[]`는 top-level frozen execution shape catalog이며 `models[].execution_preset`가 참조하는 대상이다. 각 preset의 `selector.model`과 route stage `model`은 기존 `models[].id` catalog를 참조해야 한다. `execution_presets[]` catalog 변경과 `models[].execution_preset` mapping 변경은 모두 live-apply로 분류되며 refresh 이후 새로 시작되는 logical request에만 적용되고 in-flight request에는 영향을 주지 않는다.
|
||||
- `execution_presets[].single_request`는 operator-owned fixed single-request policy다. 설정 시 preset은 `allowed_modes=["light"]`, `stages=[plan, work, review]`의 승인된 plan→work→review 경로를 고수한다. 절대 상한은 `wall_clock_ms ≤ 1800000`, `timeout_ms ≤ 600000`, `max_tool_iterations ≤ 64`, `max_output_bytes ≤ 16777216`이며 `timeout_ms`는 `wall_clock_ms`를 초과할 수 없다. selector와 plan/review stage는 `reasoning_effort=high`를 강제하고 work stage는 `reasoning_effort`를 선언할 수 없다. `workspace_ref`는 비어있을 수 없으며 raw path, credential, Node id, endpoint를 포함하지 않는다. single_request preset은 `workspace_tools`를 선언할 수 없다. catalog 변경과 mapping 변경은 live-apply로 분류되며 refresh 이후 새로 시작되는 logical request에만 적용된다. admitted single-request binding은 refresh 이후에도 frozen public model, stage binding, workspace reference, limits를 유지한다.
|
||||
- `execution_presets[].single_request`는 operator-owned fixed single-request policy다. 설정 시 preset은 `allowed_modes=["light"]`, `stages=[plan, work, review]`의 승인된 plan→work→review 경로를 고수한다. 절대 상한은 `wall_clock_ms ≤ 1800000`, `timeout_ms ≤ 600000`, `max_tool_iterations ≤ 64`, `max_output_bytes ≤ 16777216`이며 `timeout_ms`는 `wall_clock_ms`를 초과할 수 없다. selector와 plan/review stage는 `reasoning_effort=high`를 강제하고 work stage는 `reasoning_effort`를 선언할 수 없다. `workspace_ref`는 비어있을 수 없으며 raw path, credential, Node id, endpoint를 포함하지 않는다. `templates` 섹션을 통해 optional `plan_file` 및 `review_file` (edge.yaml 상대 경로) 커스텀 Markdown 템플릿을 지정할 수 있으며, load 시점에 8192바이트 상한 및 문법 검증이 수행되고 생략 시 built-in default 템플릿이 적용된다. config refresh diff reporting 시 템플릿 파일 경로나 본문은 노출되지 않고 SHA-256 digest만 보고된다. single_request preset은 `workspace_tools`를 선언할 수 없다. catalog 변경과 mapping 변경은 live-apply로 분류되며 refresh 이후 새로 시작되는 logical request에만 적용된다. admitted single-request binding은 refresh 이후에도 frozen public model, stage binding, workspace reference, limits, effective templates를 유지한다.
|
||||
- `nodes[].providers[]`는 Node 아래 resource/provider catalog다. `category`는 `api`, `cli`, `local_inference` resource kind를 나타낸다.
|
||||
- `nodes[].providers[].type`의 `seulgivibe_claude`와 `seulgivibe_openai`는 runtime type을 `openai_compat`로 정규화한다. Edge가 Node adapter payload를 만들 때 명시 provider label이 없으면 원래 Seulgivibe type alias를 `OpenAICompatAdapterConfig.provider`로 보존한다.
|
||||
- `nodes[].providers[].response_stall_timeout_ms`는 provider-originated response-stall timeout을 밀리초 단위로 선언한다. 양수 값은 그대로 사용되고, 0 또는 생략은 문서화된 기본값 `300000`을 적용한다. 음수 값과 safe duration bound를 초과하는 양수 값은 `NodeProviderConf.Validate()`에서 거부한다. effective 값은 `NodeProviderConf.EffectiveResponseStallTimeoutMS()`에서 계산한다. 이 필드는 config refresh에서 `restart_required`로 분류되며, effective-zero 등가성(생략 vs 명시적 0)은 변경으로 보고되지 않는다. request hard timeout, queue timeout, heartbeat/disconnect, CLI `response_idle_timeout_ms`는 기존 소유권을 유지한다.
|
||||
|
|
|
|||
|
|
@ -216,7 +216,7 @@ The shared `packages/go/execution` package contains provider lifecycle, registry
|
|||
| workspace tool executor | A validated `darwin|linux` Node catalog owns opened root and directory handles only when every entry platform matches the host exactly. Windows, unknown hosts, and cross-platform catalogs fail before root open; empty catalogs remain compatible. Go 1.24-compatible no-follow file primitives provide bounded read, bounded list, structured write, and non-recursive delete. Exact operator-owned command templates run through an inherited-root `fchdir`/`exec` shim with minimal allowlisted environment, shared stdout/stderr bounds, process-group timeout/cancel, and stable typed results. OS is runtime evidence rather than a caller-visible selector. |
|
||||
| internal workspace tool loop | The service decodes only `workspace_read`, `workspace_list`, `workspace_write`, `workspace_delete`, and `workspace_command`, opens the admitted workspace once, dispatches one call at a time on the frozen generation, and delivers one deep-copied typed result to the emitting executor continuation. Unique request/stage/tool correlation, per-stage iteration/output/deadline limits, request wall-clock budget, and typed cancel fail closed without external continuation or reselection. |
|
||||
| request-owned internal artifacts | `SingleRequestController` exposes closed plan/review read/write operations. Artifact calls and model workspace tools share one serialized lazy `WorkspaceOpen`, the exact admitted Node generation, the active stage deadline, the immutable output bound, in-flight work accounting, and one terminal cleanup. Node alone maps selectors to `plan.md` and `review.md`, and inventoried descriptor-relative reads fail closed on identity replacement. |
|
||||
| Plan stage | The Plan runner emits the `planning` envelope, sends the immutable task through the frozen Gemini Chat binding with `reasoning_effort=high` and an Edge-owned OpenAI `json_schema` response format, requires one strict small `plan`/`verification` JSON result, and writes deterministic bounded Markdown through `SingleRequestArtifactPlan`. Stage options cannot replace the schema, and the strict parser still enforces exact nonempty canonical fields. |
|
||||
| Plan stage | The Plan runner emits the `planning` envelope, sends the immutable task through the frozen Gemini Chat binding with `reasoning_effort=high` and the template prompt, parses direct PlanMD output against the frozen effective Plan template, and writes deterministic bounded Markdown through `SingleRequestArtifactPlan`. |
|
||||
| Work stage | The `ornith-fast` Work runner reads the closed PLAN artifact, projects only the admitted workspace tools, and resumes the same frozen provider route after exactly correlated Node results. It rejects any Work `reasoning_effort`, malformed or multiple tool calls, and empty completion or verification evidence. |
|
||||
| request-owned cleanup | Node creates and inventories only `.iop/job/<request_id>` internal state, cancels and waits for all active command groups, validates the exact tree without following entries, and removes matching artifacts deepest-first with non-recursive descriptor operations. Symlinks, special files, foreign devices, identity replacements, and unowned entries fail closed. User results and sibling request state are preserved. Concurrent cleanup callers receive one bounded cached typed result. |
|
||||
| provider raw tunnel | 선택된 provider의 HTTP/SSE를 `ProviderTunnelRequest`/`ProviderTunnelFrame`으로 relay하며 순서와 단일 terminal outcome을 보장한다. |
|
||||
|
|
|
|||
|
|
@ -124,7 +124,7 @@ Edge 설정에서 provider-pool이 어떻게 모델 실행 후보를 고르고,
|
|||
| Stream Evidence Gate config | `openai.stream_evidence_gate` provides runtime activation, request-total/strategy fault recovery caps, ingress snapshot bounds, and per-filter capability/enforcement/Unicode hold policy; it is currently restart-required. |
|
||||
| mutable apply | 적용 가능한 변경은 Edge `Cfg`, `NodeStore`, service/input model catalog, OpenAI long-context threshold를 copy-on-write로 교체한다. |
|
||||
| single-request snapshot isolation | An admitted single-request binding is independent of subsequent model catalog, execution preset, or provider pool changes. Refresh replaces the live catalog and preset snapshots used by future admissions; already-admitted bindings retain their original values. |
|
||||
| fixed single-request policy | `execution_presets[].single_request` declares an operator-owned immutable plan→work→review light path with absolute wall-clock (`≤1800000ms`), stage-timeout (`≤600000ms`), tool-iteration (`≤64`), and output-byte (`≤16MiB`) caps. Selector and plan/review stages require `reasoning_effort=high`; work stage forbids it. `workspace_ref` is opaque (never raw path/credential/Node/endpoint). single_request preset rejects `workspace_tools`. Catalog and mapping changes are live-apply and affect only new request snapshots; admitted bindings retain their frozen values across refresh. |
|
||||
| fixed single-request policy | `execution_presets[].single_request` declares an operator-owned immutable plan→work→review light path with absolute wall-clock (`≤1800000ms`), stage-timeout (`≤600000ms`), tool-iteration (`≤64`), and output-byte (`≤16MiB`) caps. Selector and plan/review stages require `reasoning_effort=high`; work stage forbids it. `workspace_ref` is opaque (never raw path/credential/Node/endpoint). Optional `templates` (`plan_file`/`review_file`) load config-relative Markdown templates up to 8192 bytes, fallback to built-in defaults, and report SHA-256 digests without exposing template paths or contents in refresh diff evidence. single_request preset rejects `workspace_tools`. Catalog and mapping changes are live-apply and affect only new request snapshots; admitted bindings retain their frozen values across refresh. |
|
||||
| operator-owned workspace catalog | `nodes[].workspaces[]` is the operator-owned bounded capability catalog for each node. Each entry is keyed by a globally unique, trimmed `ref`, declares `platform` in the closed `darwin|linux` implementation set, and retains the existing absolute clean root, closed operations, approved commands, environment allowlist, and bounded byte/time limits. Refs remain globally unique and any catalog change is `restart_required`. Empty catalogs are backward-compatible on any host. A non-empty catalog requires a supported Node host and every entry must match that host before any root is opened; Windows, unknown hosts, and cross-platform catalogs fail closed. The catalog is delivered by the Node-private typed config payload and retained as opened immutable runtime authority. Raw roots and command details never enter presets, public responses, provider requests, or metadata; operating system is runtime evidence rather than a caller selector. |
|
||||
| Node config refresh push | 변경이 있으면 Edge가 dispatch-ready Node에 node-specific `NodeConfigRefreshRequest`를 push한다. accepted지만 pending인 Node는 register response config를 적용한 뒤 ready가 될 때까지 push 대상이 아니다. |
|
||||
| Node registry swap | Node는 refresh payload로 새 adapter registry를 만들고 router registry를 swap한다. old registry stop은 active run이 있으면 drain 이후로 지연한다. |
|
||||
|
|
@ -170,7 +170,7 @@ sequenceDiagram
|
|||
- `iop.edge-config-runtime-refresh`: `agent-contract/inner/edge-config-runtime-refresh.md`
|
||||
- `iop.edge-node-runtime-wire`: `agent-contract/inner/edge-node-runtime-wire.md`
|
||||
- proto 원문: `proto/iop/runtime.proto`
|
||||
- `execution_presets[].single_request` is the operator-owned fixed single-request policy. Absolute caps: `wall_clock_ms ∈ [1, 1800000]`, `timeout_ms ∈ [1, 600000]`, `timeout_ms ≤ wall_clock_ms`, `max_tool_iterations ∈ [1, 64]`, `max_output_bytes ∈ [1, 16777216]`. Stages enforce exactly plan→work→review with `reasoning_effort=high` on selector and plan/review, forbidden on work. `workspace_ref` is opaque (never raw path/credential/Node/endpoint). single_request preset rejects `workspace_tools`. Catalog/mapping changes are live-apply; admitted bindings are snapshot-isolated across refresh.
|
||||
- `execution_presets[].single_request` is the operator-owned fixed single-request policy. Absolute caps: `wall_clock_ms ∈ [1, 1800000]`, `timeout_ms ∈ [1, 600000]`, `timeout_ms ≤ wall_clock_ms`, `max_tool_iterations ∈ [1, 64]`, `max_output_bytes ∈ [1, 16777216]`. Stages enforce exactly plan→work→review with `reasoning_effort=high` on selector and plan/review, forbidden on work. `workspace_ref` is opaque (never raw path/credential/Node/endpoint). Optional `templates` (`plan_file`/`review_file`) load config-relative Markdown templates up to 8192 bytes, fallback to built-in defaults, and report SHA-256 digests without exposing template paths or contents in refresh diff evidence. single_request preset rejects `workspace_tools`. Catalog/mapping changes are live-apply; admitted bindings are snapshot-isolated across refresh.
|
||||
|
||||
## 설정/데이터/이벤트
|
||||
|
||||
|
|
|
|||
|
|
@ -11,6 +11,7 @@ import (
|
|||
|
||||
"iop/apps/edge/internal/edgevalidate"
|
||||
"iop/packages/go/config"
|
||||
"iop/packages/go/singlerequesttemplate"
|
||||
)
|
||||
|
||||
// applyRuntimeNormalization normalizes candidate config so that classification
|
||||
|
|
@ -395,7 +396,7 @@ func appendExecutionPresetChanges(changes *[]Change, current, candidate *config.
|
|||
appendDeepIfChanged(changes, fmt.Sprintf("execution_presets[%q].selector", id), StatusApplied, cur.Selector, next.Selector)
|
||||
appendDeepIfChanged(changes, fmt.Sprintf("execution_presets[%q].allowed_modes", id), StatusApplied, cur.AllowedModes, next.AllowedModes)
|
||||
appendDeepIfChanged(changes, fmt.Sprintf("execution_presets[%q].routes", id), StatusApplied, cur.Routes, next.Routes)
|
||||
appendDeepIfChanged(changes, fmt.Sprintf("execution_presets[%q].single_request", id), StatusApplied, cur.SingleRequest, next.SingleRequest)
|
||||
appendSingleRequestChanged(changes, fmt.Sprintf("execution_presets[%q].single_request", id), StatusApplied, cur.SingleRequest, next.SingleRequest)
|
||||
appendDeepIfChanged(changes, fmt.Sprintf("execution_presets[%q].workspace_tools", id), StatusApplied, cur.WorkspaceTools, next.WorkspaceTools)
|
||||
}
|
||||
for id := range candidatePresets {
|
||||
|
|
@ -410,6 +411,28 @@ func appendExecutionPresetChanges(changes *[]Change, current, candidate *config.
|
|||
}
|
||||
}
|
||||
|
||||
func appendSingleRequestChanged(changes *[]Change, path string, class Status, cur, next *config.ExecutionSingleRequestPolicy) {
|
||||
if reflect.DeepEqual(cur, next) {
|
||||
return
|
||||
}
|
||||
*changes = append(*changes, Change{
|
||||
Path: path,
|
||||
Class: class,
|
||||
Previous: redactedSingleRequestSummary(cur),
|
||||
Next: redactedSingleRequestSummary(next),
|
||||
})
|
||||
}
|
||||
|
||||
func redactedSingleRequestSummary(sr *config.ExecutionSingleRequestPolicy) string {
|
||||
if sr == nil {
|
||||
return "<nil>"
|
||||
}
|
||||
planDigest := singlerequesttemplate.Digest(sr.Templates.EffectivePlan)
|
||||
reviewDigest := singlerequesttemplate.Digest(sr.Templates.EffectiveReview)
|
||||
return fmt.Sprintf("&{WorkspaceRef:%s Limits:%v Stages:%v Templates:{PlanDigest:%s ReviewDigest:%s}}",
|
||||
sr.WorkspaceRef, sr.Limits, sr.Stages, planDigest, reviewDigest)
|
||||
}
|
||||
|
||||
func resultFromChanges(changes []Change) Result {
|
||||
sort.SliceStable(changes, func(i, j int) bool {
|
||||
if changes[i].Path == changes[j].Path {
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
package configrefresh_test
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"iop/apps/edge/internal/configrefresh"
|
||||
|
|
@ -128,11 +128,11 @@ func TestClassifyExecutionPresetLiveApply(t *testing.T) {
|
|||
t.Errorf("change[%d] class for %s: got %q, want %q", i, c.Path, c.Class, want[i].class)
|
||||
}
|
||||
if c.Path == `execution_presets["preset-m-mod"].single_request` {
|
||||
if c.Previous != fmt.Sprintf("%v", current.ExecutionPresets[1].SingleRequest) {
|
||||
t.Errorf("single_request previous = %q, want %q", c.Previous, fmt.Sprintf("%v", current.ExecutionPresets[1].SingleRequest))
|
||||
if !strings.Contains(c.Previous, "PlanDigest:") || !strings.Contains(c.Previous, "ReviewDigest:") {
|
||||
t.Errorf("single_request previous = %q, expected redacted template digests", c.Previous)
|
||||
}
|
||||
if c.Next != fmt.Sprintf("%v", candidate.ExecutionPresets[1].SingleRequest) {
|
||||
t.Errorf("single_request next = %q, want %q", c.Next, fmt.Sprintf("%v", candidate.ExecutionPresets[1].SingleRequest))
|
||||
if !strings.Contains(c.Next, "PlanDigest:") || !strings.Contains(c.Next, "ReviewDigest:") {
|
||||
t.Errorf("single_request next = %q, expected redacted template digests", c.Next)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -214,3 +214,105 @@ func TestClassifyModelExecutionPresetLiveApply(t *testing.T) {
|
|||
t.Errorf("ChangedModels = %v, want [virtual-model]", result.ChangedModels)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifySingleRequestTemplateContentChange(t *testing.T) {
|
||||
current := &config.EdgeConfig{
|
||||
ExecutionPresets: []config.ExecutionPreset{
|
||||
{
|
||||
ID: "preset-templates",
|
||||
Selector: config.ExecutionModelBinding{Model: "gpt-4o", Options: map[string]any{"reasoning_effort": "high"}},
|
||||
AllowedModes: []string{config.ModeLight},
|
||||
Routes: map[string]config.ExecutionRoute{
|
||||
config.ModeLight: {
|
||||
Stages: []config.ExecutionRouteStage{
|
||||
{Role: "plan", Model: "gpt-4o", Options: map[string]any{"reasoning_effort": "high"}},
|
||||
{Role: "work", Model: "gpt-4o-mini"},
|
||||
{Role: "review", Model: "gpt-4o", Options: map[string]any{"reasoning_effort": "high"}},
|
||||
},
|
||||
},
|
||||
},
|
||||
SingleRequest: &config.ExecutionSingleRequestPolicy{
|
||||
WorkspaceRef: "ws-1",
|
||||
Limits: config.ExecutionSingleRequestLimits{
|
||||
WallClockMS: 60000,
|
||||
StageTimeoutMS: 30000,
|
||||
MaxToolIterations: 10,
|
||||
MaxOutputBytes: 1024,
|
||||
},
|
||||
Stages: config.ExecutionSingleRequestStages{
|
||||
Plan: config.ExecutionSingleRequestStageConfig{Model: "gpt-4o", Options: map[string]any{"reasoning_effort": "high"}},
|
||||
Work: config.ExecutionSingleRequestStageConfig{Model: "gpt-4o-mini"},
|
||||
Review: config.ExecutionSingleRequestStageConfig{Model: "gpt-4o", Options: map[string]any{"reasoning_effort": "high"}},
|
||||
},
|
||||
Templates: config.ExecutionSingleRequestTemplates{
|
||||
PlanFile: "templates/plan.md",
|
||||
ReviewFile: "templates/review.md",
|
||||
EffectivePlan: "Plan template v1\n# Plan\n## Goal\n{{goal}}\n## Steps\n{{steps}}\n## Verification\n{{verification}}",
|
||||
EffectiveReview: "Review template v1\n# Review\n## Result\nPASS\n## Checks\n{{checks}}\n## Verification\n{{verification}}\n## Summary\n{{summary}}",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
candidate := &config.EdgeConfig{
|
||||
ExecutionPresets: []config.ExecutionPreset{
|
||||
{
|
||||
ID: "preset-templates",
|
||||
Selector: config.ExecutionModelBinding{Model: "gpt-4o", Options: map[string]any{"reasoning_effort": "high"}},
|
||||
AllowedModes: []string{config.ModeLight},
|
||||
Routes: map[string]config.ExecutionRoute{
|
||||
config.ModeLight: {
|
||||
Stages: []config.ExecutionRouteStage{
|
||||
{Role: "plan", Model: "gpt-4o", Options: map[string]any{"reasoning_effort": "high"}},
|
||||
{Role: "work", Model: "gpt-4o-mini"},
|
||||
{Role: "review", Model: "gpt-4o", Options: map[string]any{"reasoning_effort": "high"}},
|
||||
},
|
||||
},
|
||||
},
|
||||
SingleRequest: &config.ExecutionSingleRequestPolicy{
|
||||
WorkspaceRef: "ws-1",
|
||||
Limits: config.ExecutionSingleRequestLimits{
|
||||
WallClockMS: 60000,
|
||||
StageTimeoutMS: 30000,
|
||||
MaxToolIterations: 10,
|
||||
MaxOutputBytes: 1024,
|
||||
},
|
||||
Stages: config.ExecutionSingleRequestStages{
|
||||
Plan: config.ExecutionSingleRequestStageConfig{Model: "gpt-4o", Options: map[string]any{"reasoning_effort": "high"}},
|
||||
Work: config.ExecutionSingleRequestStageConfig{Model: "gpt-4o-mini"},
|
||||
Review: config.ExecutionSingleRequestStageConfig{Model: "gpt-4o", Options: map[string]any{"reasoning_effort": "high"}},
|
||||
},
|
||||
Templates: config.ExecutionSingleRequestTemplates{
|
||||
PlanFile: "templates/plan.md", // same path
|
||||
ReviewFile: "templates/review.md",
|
||||
EffectivePlan: "Plan template v2\n# Plan\n## Goal\n{{goal}}\n## Steps\n{{steps}}\n## Verification\n{{verification}}", // content changed
|
||||
EffectiveReview: "Review template v1\n# Review\n## Result\nPASS\n## Checks\n{{checks}}\n## Verification\n{{verification}}\n## Summary\n{{summary}}",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
result := configrefresh.Classify(current, candidate)
|
||||
if result.Status != configrefresh.StatusApplied {
|
||||
t.Fatalf("expected status=%q, got %q (changes: %+v)", configrefresh.StatusApplied, result.Status, result.Changes)
|
||||
}
|
||||
if len(result.Changes) != 1 {
|
||||
t.Fatalf("expected 1 change, got %d", len(result.Changes))
|
||||
}
|
||||
|
||||
c := result.Changes[0]
|
||||
if c.Path != `execution_presets["preset-templates"].single_request` {
|
||||
t.Errorf("change path = %q, want execution_presets[\"preset-templates\"].single_request", c.Path)
|
||||
}
|
||||
if c.Class != configrefresh.StatusApplied {
|
||||
t.Errorf("change class = %q, want StatusApplied", c.Class)
|
||||
}
|
||||
if strings.Contains(c.Previous, "templates/plan.md") || strings.Contains(c.Next, "templates/plan.md") {
|
||||
t.Errorf("change output exposes file path: prev=%q, next=%q", c.Previous, c.Next)
|
||||
}
|
||||
if strings.Contains(c.Previous, "Plan template v1") || strings.Contains(c.Next, "Plan template v2") {
|
||||
t.Errorf("change output exposes raw template body: prev=%q, next=%q", c.Previous, c.Next)
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -92,12 +92,9 @@ func waitExecutionResult(exec edgeservice.SingleRequestExecution) (edgeservice.S
|
|||
return exec.Wait()
|
||||
}
|
||||
|
||||
func executorPlanBody(plan, verification string) []byte {
|
||||
b, _ := json.Marshal(map[string]any{
|
||||
"plan": plan,
|
||||
"verification": verification,
|
||||
})
|
||||
return successBodyWithThoughtSignature(string(b))
|
||||
func executorPlanBody(goal, verification string) []byte {
|
||||
planMD := fmt.Sprintf("# Plan\n\n## Goal\n%s\n\n## Steps\n- Step one.\n- Step two.\n\n## Verification\n- %s\n", goal, verification)
|
||||
return successBodyWithThoughtSignature(planMD)
|
||||
}
|
||||
|
||||
func executorWorkBody(completion, verification string) []byte {
|
||||
|
|
@ -110,9 +107,11 @@ func executorWorkBody(completion, verification string) []byte {
|
|||
|
||||
func executorReviewPassBody(output, summary string) []byte {
|
||||
b, _ := json.Marshal(map[string]any{
|
||||
"decision": "pass",
|
||||
"output": output,
|
||||
"summary": summary,
|
||||
"decision": "pass",
|
||||
"output": output,
|
||||
"checks": "- Checked requirements",
|
||||
"verification": "- Verified tests pass",
|
||||
"summary": summary,
|
||||
})
|
||||
return successBodyWithThoughtSignature(string(b))
|
||||
}
|
||||
|
|
@ -290,7 +289,7 @@ func TestSingleRequestExecutorConcurrentToolIsolation(t *testing.T) {
|
|||
bodyStr := string(reqBody)
|
||||
|
||||
var resp []byte
|
||||
if strings.Contains(bodyStr, "Produce exactly one JSON object with non-empty string fields plan") {
|
||||
if strings.Contains(bodyStr, "Produce exactly one Markdown document matching this template") {
|
||||
resp = executorPlanBody(fmt.Sprintf("Plan for %s", reqID), fmt.Sprintf("Verify plan for %s", reqID))
|
||||
} else if strings.Contains(bodyStr, "Read the supplied plan") {
|
||||
if !strings.Contains(bodyStr, "typed-result-") {
|
||||
|
|
@ -757,7 +756,7 @@ func TestSingleRequestExecutorTerminalWaiterCleanup(t *testing.T) {
|
|||
bodyStr := string(reqBody)
|
||||
|
||||
var resp []byte
|
||||
if strings.Contains(bodyStr, "Produce exactly one JSON object with non-empty string fields plan") {
|
||||
if strings.Contains(bodyStr, "Produce exactly one Markdown document matching this template") {
|
||||
resp = executorPlanBody("Plan step", "Verify plan")
|
||||
} else if strings.Contains(bodyStr, "Read the supplied plan") {
|
||||
if !strings.Contains(bodyStr, "colliding-tool-id") {
|
||||
|
|
@ -811,7 +810,7 @@ func TestSingleRequestExecutorTerminalWaiterCleanup(t *testing.T) {
|
|||
bodyStr := string(reqBody)
|
||||
|
||||
var resp []byte
|
||||
if strings.Contains(bodyStr, "Produce exactly one JSON object with non-empty string fields plan") {
|
||||
if strings.Contains(bodyStr, "Produce exactly one Markdown document matching this template") {
|
||||
resp = executorPlanBody("Plan step", "Verify plan")
|
||||
} else if strings.Contains(bodyStr, "Read the supplied plan") {
|
||||
if !strings.Contains(bodyStr, "colliding-tool-id") {
|
||||
|
|
@ -873,7 +872,7 @@ func TestSingleRequestExecutorTerminalWaiterCleanup(t *testing.T) {
|
|||
}
|
||||
|
||||
var resp []byte
|
||||
if strings.Contains(bodyStr, "Produce exactly one JSON object with non-empty string fields plan") {
|
||||
if strings.Contains(bodyStr, "Produce exactly one Markdown document matching this template") {
|
||||
resp = executorPlanBody("Plan step for "+reqID, "Verify plan")
|
||||
} else if strings.Contains(bodyStr, "Read the supplied plan") {
|
||||
if !strings.Contains(bodyStr, "colliding-tool-id") {
|
||||
|
|
|
|||
|
|
@ -2,15 +2,15 @@ package openai
|
|||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"strings"
|
||||
|
||||
edgeservice "iop/apps/edge/internal/service"
|
||||
"iop/packages/go/singlerequesttemplate"
|
||||
)
|
||||
|
||||
const singleRequestPlanPrompt = "Produce exactly one JSON object with non-empty string fields plan and verification. Keep both concise."
|
||||
func singleRequestPlanPrompt(template string) string {
|
||||
return "Produce exactly one Markdown document matching this template:\n\n" + template + "\n\nRequirements:\n- Keep the goal to a single non-empty line.\n- Provide between 2 and 6 step lines, each starting with '- '.\n- Provide between 1 and 3 verification lines, each starting with '- '.\n- Match all headings and static text exactly with no unresolved tokens."
|
||||
}
|
||||
|
||||
var errSingleRequestPlanStage = errors.New("single-request plan stage: failed")
|
||||
|
||||
|
|
@ -32,48 +32,29 @@ type singleRequestPlanStageRequest struct {
|
|||
Quality *singleRequestQualityGate
|
||||
}
|
||||
|
||||
func singleRequestPlanResponseFormat() *singleRequestProviderResponseFormat {
|
||||
return &singleRequestProviderResponseFormat{
|
||||
Type: "json_schema",
|
||||
JSONSchema: singleRequestProviderResponseJSONSchema{
|
||||
Name: "single_request_plan",
|
||||
Strict: true,
|
||||
Schema: singleRequestProviderOutputSchema{
|
||||
Type: "object",
|
||||
Properties: map[string]singleRequestProviderOutputProperty{
|
||||
"plan": {
|
||||
Type: "string",
|
||||
Description: "A concise execution plan for the task.",
|
||||
},
|
||||
"verification": {
|
||||
Type: "string",
|
||||
Description: "A concise verification procedure for the plan.",
|
||||
},
|
||||
},
|
||||
Required: []string{"plan", "verification"},
|
||||
AdditionalProperties: false,
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func (s *singleRequestPlanStage) run(ctx context.Context, req singleRequestPlanStageRequest, ctrl edgeservice.SingleRequestController) ([]byte, error) {
|
||||
quality := singleRequestQualityGateOrNew(req.Quality)
|
||||
if s == nil || s.provider == nil || ctrl == nil || req.RequestID == "" || req.Task == "" || req.Sequence == 0 {
|
||||
return nil, quality.validation(errSingleRequestPlanStage)
|
||||
}
|
||||
binding := ctrl.Binding()
|
||||
if binding == nil || binding.Templates.Plan == "" {
|
||||
return nil, quality.validation(errSingleRequestPlanStage)
|
||||
}
|
||||
if err := ctrl.SubmitEnvelope(edgeservice.SingleRequestEnvelope{RequestID: req.RequestID, Sequence: req.Sequence, Stage: edgeservice.SingleRequestStatePlanning}); err != nil {
|
||||
return nil, quality.serviceFailure(ctx, err, errSingleRequestPlanStage)
|
||||
}
|
||||
response, err := s.provider.submit(ctx, singleRequestProviderStageRequest{
|
||||
StageBinding: req.StageBinding, Limits: req.Limits, NodeRef: req.NodeRef, SessionID: req.SessionID, UsageAttribution: req.UsageAttribution, Quality: quality,
|
||||
Messages: []chatMessage{{Role: "system", Content: singleRequestPlanPrompt}, {Role: "user", Content: req.Task}},
|
||||
ResponseFormat: singleRequestPlanResponseFormat(),
|
||||
Messages: []chatMessage{
|
||||
{Role: "system", Content: singleRequestPlanPrompt(binding.Templates.Plan)},
|
||||
{Role: "user", Content: req.Task},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
return nil, quality.reclassify(err, errSingleRequestPlanStage)
|
||||
}
|
||||
content, err := renderSingleRequestPlan(response.Output, req.Limits.MaxOutputBytes)
|
||||
content, err := singlerequesttemplate.ParsePlan(binding.Templates.Plan, response.Output, req.Limits.MaxOutputBytes)
|
||||
if err != nil {
|
||||
return nil, quality.malformed(errSingleRequestPlanStage)
|
||||
}
|
||||
|
|
@ -82,46 +63,3 @@ func (s *singleRequestPlanStage) run(ctx context.Context, req singleRequestPlanS
|
|||
}
|
||||
return content, nil
|
||||
}
|
||||
|
||||
type singleRequestPlanResult struct {
|
||||
Plan string `json:"plan"`
|
||||
Verification string `json:"verification"`
|
||||
}
|
||||
|
||||
type singleRequestPlanResultAlias singleRequestPlanResult
|
||||
|
||||
func (r *singleRequestPlanResult) UnmarshalJSON(data []byte) error {
|
||||
if err := validateSingleRequestObjectFields(data, "plan", "verification"); err != nil {
|
||||
return err
|
||||
}
|
||||
var a singleRequestPlanResultAlias
|
||||
if err := json.Unmarshal(data, &a); err != nil {
|
||||
return err
|
||||
}
|
||||
*r = singleRequestPlanResult(a)
|
||||
return nil
|
||||
}
|
||||
|
||||
func renderSingleRequestPlan(raw string, maximum int) ([]byte, error) {
|
||||
if err := validateSingleRequestJSON([]byte(raw)); err != nil {
|
||||
return nil, errSingleRequestPlanStage
|
||||
}
|
||||
decoder := json.NewDecoder(strings.NewReader(raw))
|
||||
decoder.DisallowUnknownFields()
|
||||
var result singleRequestPlanResult
|
||||
if err := decoder.Decode(&result); err != nil {
|
||||
return nil, errSingleRequestPlanStage
|
||||
}
|
||||
if strings.TrimSpace(result.Plan) == "" || strings.TrimSpace(result.Verification) == "" {
|
||||
return nil, errSingleRequestPlanStage
|
||||
}
|
||||
var extra any
|
||||
if err := decoder.Decode(&extra); err != io.EOF {
|
||||
return nil, errSingleRequestPlanStage
|
||||
}
|
||||
content := []byte("# Plan\n\n" + strings.TrimSpace(result.Plan) + "\n\n## Verification\n\n" + strings.TrimSpace(result.Verification) + "\n")
|
||||
if maximum < 1 || len(content) > maximum {
|
||||
return nil, errSingleRequestPlanStage
|
||||
}
|
||||
return content, nil
|
||||
}
|
||||
|
|
|
|||
|
|
@ -4,14 +4,15 @@ import (
|
|||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
edgeservice "iop/apps/edge/internal/service"
|
||||
"iop/packages/go/singlerequesttemplate"
|
||||
)
|
||||
|
||||
type planController struct {
|
||||
binding *edgeservice.SingleRequestBinding
|
||||
envelopes []edgeservice.SingleRequestEnvelope
|
||||
kind edgeservice.SingleRequestArtifactKind
|
||||
content []byte
|
||||
|
|
@ -20,9 +21,15 @@ type planController struct {
|
|||
writeErr error
|
||||
}
|
||||
|
||||
func (c *planController) RequestID() string { return "request-1" }
|
||||
func (c *planController) Binding() *edgeservice.SingleRequestBinding { return nil }
|
||||
func (c *planController) Context() context.Context { return context.Background() }
|
||||
func (c *planController) RequestID() string { return "request-1" }
|
||||
func (c *planController) Binding() *edgeservice.SingleRequestBinding {
|
||||
if c.binding != nil {
|
||||
return c.binding
|
||||
}
|
||||
b, _ := edgeservice.NewSingleRequestBinding("virtual-model", "ws-ref", validStageBinding(), validStageBinding(), validStageBinding(), validLimits())
|
||||
return b
|
||||
}
|
||||
func (c *planController) Context() context.Context { return context.Background() }
|
||||
func (c *planController) State() edgeservice.SingleRequestState {
|
||||
return edgeservice.SingleRequestStateAccepted
|
||||
}
|
||||
|
|
@ -58,7 +65,8 @@ func validPlanStageRequest() singleRequestPlanStageRequest {
|
|||
|
||||
func TestSingleRequestPlanStageWritesArtifact(t *testing.T) {
|
||||
d := matchingDispatch()
|
||||
tunnel := &mockTunnel{frames: framesFor(successBodyWithThoughtSignature(`{"plan":"Inspect the target.","verification":"Run focused tests."}`))}
|
||||
planMD := "# Plan\n\n## Goal\nInspect the target.\n\n## Steps\n- Step one.\n- Step two.\n\n## Verification\n- Run focused tests.\n"
|
||||
tunnel := &mockTunnel{frames: framesFor(successBodyWithThoughtSignature(planMD))}
|
||||
var captured edgeservice.ProviderPoolDispatchRequest
|
||||
provider := newSingleRequestProviderStage(&mockService{submit: func(_ context.Context, r edgeservice.ProviderPoolDispatchRequest) (*edgeservice.ProviderPoolDispatchResult, error) {
|
||||
captured = r
|
||||
|
|
@ -66,71 +74,74 @@ func TestSingleRequestPlanStageWritesArtifact(t *testing.T) {
|
|||
}})
|
||||
ctrl := &planController{}
|
||||
request := validPlanStageRequest()
|
||||
request.StageBinding.Options["response_format"] = map[string]any{"type": "caller_override_ignored"}
|
||||
content, err := newSingleRequestPlanStage(provider).run(context.Background(), request, ctrl)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
expected := "# Plan\n\nInspect the target.\n\n## Verification\n\nRun focused tests.\n"
|
||||
if got := string(content); got != expected {
|
||||
t.Fatalf("content=%q", got)
|
||||
if got := string(content); got != planMD {
|
||||
t.Fatalf("content=%q, want %q", got, planMD)
|
||||
}
|
||||
if ctrl.kind != edgeservice.SingleRequestArtifactPlan || len(ctrl.envelopes) != 1 || ctrl.envelopes[0].Stage != edgeservice.SingleRequestStatePlanning {
|
||||
t.Fatalf("controller=%+v", ctrl)
|
||||
}
|
||||
body, _ := captured.Tunnel.BuildBody("gemini-3.6-flash")
|
||||
if !containsAll(string(body), "Fix immutable task", "Produce exactly one JSON object", "reasoning_effort", "high") {
|
||||
if !containsAll(string(body), "Fix immutable task", "Produce exactly one Markdown document", "reasoning_effort", "high") {
|
||||
t.Fatalf("body=%s", body)
|
||||
}
|
||||
var decoded map[string]any
|
||||
if err := json.Unmarshal(body, &decoded); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
wantResponseFormat := map[string]any{
|
||||
"type": "json_schema",
|
||||
"json_schema": map[string]any{
|
||||
"name": "single_request_plan",
|
||||
"strict": true,
|
||||
"schema": map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"plan": map[string]any{"type": "string", "description": "A concise execution plan for the task."},
|
||||
"verification": map[string]any{"type": "string", "description": "A concise verification procedure for the plan."},
|
||||
},
|
||||
"required": []any{"plan", "verification"},
|
||||
"additionalProperties": false,
|
||||
},
|
||||
},
|
||||
if decoded["response_format"] != nil {
|
||||
t.Fatalf("response_format should be omitted, got %#v", decoded["response_format"])
|
||||
}
|
||||
if !reflect.DeepEqual(decoded["response_format"], wantResponseFormat) {
|
||||
t.Fatalf("response_format=%#v, want %#v", decoded["response_format"], wantResponseFormat)
|
||||
}
|
||||
|
||||
func TestSingleRequestPlanStageCustomTemplate(t *testing.T) {
|
||||
d := matchingDispatch()
|
||||
customTmpl := "# Plan\n\nCustom Header\n\n## Goal\n{{goal}}\n\n## Steps\n{{steps}}\n\n## Verification\n{{verification}}\n"
|
||||
planMD := "# Plan\n\nCustom Header\n\n## Goal\nInspect custom target.\n\n## Steps\n- Custom step 1.\n- Custom step 2.\n\n## Verification\n- Custom verify.\n"
|
||||
|
||||
binding, err := edgeservice.NewSingleRequestBindingWithTemplates("virtual-model", "ws-ref", validStageBinding(), validStageBinding(), validStageBinding(), validLimits(), edgeservice.SingleRequestTemplateBinding{
|
||||
Plan: customTmpl,
|
||||
Review: singlerequesttemplate.DefaultReviewTemplate,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
tunnel := &mockTunnel{frames: framesFor(successBodyWithThoughtSignature(planMD))}
|
||||
provider := newSingleRequestProviderStage(&mockService{submit: func(_ context.Context, _ edgeservice.ProviderPoolDispatchRequest) (*edgeservice.ProviderPoolDispatchResult, error) {
|
||||
return &edgeservice.ProviderPoolDispatchResult{Path: edgeservice.ProviderPoolPathTunnel, Tunnel: tunnel, DispatchInfo: d}, nil
|
||||
}})
|
||||
ctrl := &planController{binding: binding}
|
||||
content, err := newSingleRequestPlanStage(provider).run(context.Background(), validPlanStageRequest(), ctrl)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if string(content) != planMD {
|
||||
t.Fatalf("content = %q, want %q", string(content), planMD)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSingleRequestPlanStageFailsClosed(t *testing.T) {
|
||||
jsonTests := []struct {
|
||||
malformedTests := []struct {
|
||||
name string
|
||||
raw string
|
||||
}{
|
||||
{"empty-object", "{}"},
|
||||
{"missing-verification", `{"plan":"x"}`},
|
||||
{"missing-plan", `{"verification":"y"}`},
|
||||
{"empty-plan-string", `{"plan":"","verification":"y"}`},
|
||||
{"empty-verification-string", `{"plan":"x","verification":""}`},
|
||||
{"whitespace-plan-string", `{"plan":" ","verification":"y"}`},
|
||||
{"whitespace-verification-string", `{"plan":"x","verification":" "}`},
|
||||
{"unknown-field", `{"plan":"x","verification":"y","unknown":1}`},
|
||||
{"duplicate-plan-key", `{"plan":"A","plan":"B","verification":"V"}`},
|
||||
{"duplicate-verification-key", `{"plan":"P","verification":"V1","verification":"V2"}`},
|
||||
{"case-variant-plan-key", `{"Plan":"Inspect.","verification":"Verify."}`},
|
||||
{"case-folded-duplicate-plan-key", `{"plan":"Inspect.","Plan":"Inspect2.","verification":"Verify."}`},
|
||||
{"case-variant-verification-key", `{"plan":"Inspect.","Verification":"Verify."}`},
|
||||
{"case-folded-duplicate-verification-key", `{"plan":"Inspect.","verification":"Verify.","Verification":"Verify2."}`},
|
||||
{"trailing-json", `{"plan":"x","verification":"y"} {}`},
|
||||
{"not-json", "not json"},
|
||||
{"empty-string", ""},
|
||||
{"step-count-1-too-few", "# Plan\n\n## Goal\nFix bug.\n\n## Steps\n- Step 1\n\n## Verification\n- Verify 1\n"},
|
||||
{"step-count-7-too-many", "# Plan\n\n## Goal\nFix bug.\n\n## Steps\n- S1\n- S2\n- S3\n- S4\n- S5\n- S6\n- S7\n\n## Verification\n- Verify 1\n"},
|
||||
{"verif-count-0-too-few", "# Plan\n\n## Goal\nFix bug.\n\n## Steps\n- Step 1\n- Step 2\n\n## Verification\n"},
|
||||
{"verif-count-4-too-many", "# Plan\n\n## Goal\nFix bug.\n\n## Steps\n- Step 1\n- Step 2\n\n## Verification\n- V1\n- V2\n- V3\n- V4\n"},
|
||||
{"multiline-goal", "# Plan\n\n## Goal\nLine 1\nLine 2\n\n## Steps\n- Step 1\n- Step 2\n\n## Verification\n- Verify 1\n"},
|
||||
{"altered-heading", "# Plan Mismatch\n\n## Goal\nFix bug.\n\n## Steps\n- Step 1\n- Step 2\n\n## Verification\n- Verify 1\n"},
|
||||
{"unresolved-token", "# Plan\n\n## Goal\nFix {{goal}} bug.\n\n## Steps\n- Step 1\n- Step 2\n\n## Verification\n- Verify 1\n"},
|
||||
{"trailing-content", "# Plan\n\n## Goal\nFix bug.\n\n## Steps\n- Step 1\n- Step 2\n\n## Verification\n- Verify 1\n\nExtra Trailing"},
|
||||
{"not-markdown", "just plain text"},
|
||||
}
|
||||
|
||||
for _, tt := range jsonTests {
|
||||
for _, tt := range malformedTests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
d := matchingDispatch()
|
||||
tunnel := &mockTunnel{frames: framesFor(successBody(tt.raw))}
|
||||
|
|
@ -148,27 +159,6 @@ func TestSingleRequestPlanStageFailsClosed(t *testing.T) {
|
|||
})
|
||||
}
|
||||
|
||||
t.Run("render-size-exact-boundary-passes", func(t *testing.T) {
|
||||
raw := `{"plan":"A","verification":"B"}`
|
||||
rendered := "# Plan\n\nA\n\n## Verification\n\nB\n"
|
||||
content, err := renderSingleRequestPlan(raw, len(rendered))
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error on exact render boundary: %v", err)
|
||||
}
|
||||
if string(content) != rendered {
|
||||
t.Fatalf("content mismatch: got %q, want %q", content, rendered)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("render-size-exceeded-boundary-fails", func(t *testing.T) {
|
||||
raw := `{"plan":"A","verification":"B"}`
|
||||
rendered := "# Plan\n\nA\n\n## Verification\n\nB\n"
|
||||
_, err := renderSingleRequestPlan(raw, len(rendered)-1)
|
||||
if !errors.Is(err, errSingleRequestPlanStage) {
|
||||
t.Fatalf("expected errSingleRequestPlanStage, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("provider-failure-rejects", func(t *testing.T) {
|
||||
provider := newSingleRequestProviderStage(&mockService{submit: func(context.Context, edgeservice.ProviderPoolDispatchRequest) (*edgeservice.ProviderPoolDispatchResult, error) {
|
||||
return nil, errors.New("provider failure")
|
||||
|
|
@ -220,7 +210,8 @@ func TestSingleRequestPlanStageFailsClosed(t *testing.T) {
|
|||
|
||||
t.Run("artifact-write-failure-rejects", func(t *testing.T) {
|
||||
d := matchingDispatch()
|
||||
tunnel := &mockTunnel{frames: framesFor(successBody(`{"plan":"Plan text","verification":"Verification text"}`))}
|
||||
planMD := "# Plan\n\n## Goal\nGoal text.\n\n## Steps\n- Step 1\n- Step 2\n\n## Verification\n- Verify 1\n"
|
||||
tunnel := &mockTunnel{frames: framesFor(successBody(planMD))}
|
||||
provider := newSingleRequestProviderStage(&mockService{submit: func(context.Context, edgeservice.ProviderPoolDispatchRequest) (*edgeservice.ProviderPoolDispatchResult, error) {
|
||||
return &edgeservice.ProviderPoolDispatchResult{Path: edgeservice.ProviderPoolPathTunnel, Tunnel: tunnel, DispatchInfo: d}, nil
|
||||
}})
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ import (
|
|||
"iop/apps/edge/internal/authprojection"
|
||||
edgeservice "iop/apps/edge/internal/service"
|
||||
"iop/packages/go/config"
|
||||
"iop/packages/go/singlerequesttemplate"
|
||||
)
|
||||
|
||||
var (
|
||||
|
|
@ -72,13 +73,26 @@ func compileSingleRequestBinding(
|
|||
MaxOutputBytes: srLimits.MaxOutputBytes,
|
||||
}
|
||||
|
||||
return edgeservice.NewSingleRequestBinding(
|
||||
effectivePlan := sr.Templates.EffectivePlan
|
||||
if effectivePlan == "" {
|
||||
effectivePlan = singlerequesttemplate.DefaultPlanTemplate
|
||||
}
|
||||
effectiveReview := sr.Templates.EffectiveReview
|
||||
if effectiveReview == "" {
|
||||
effectiveReview = singlerequesttemplate.DefaultReviewTemplate
|
||||
}
|
||||
|
||||
return edgeservice.NewSingleRequestBindingWithTemplates(
|
||||
publicModel,
|
||||
sr.WorkspaceRef,
|
||||
*planBinding,
|
||||
*workBinding,
|
||||
*reviewBinding,
|
||||
limits,
|
||||
edgeservice.SingleRequestTemplateBinding{
|
||||
Plan: effectivePlan,
|
||||
Review: effectiveReview,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -11,10 +11,11 @@ import (
|
|||
|
||||
edgeservice "iop/apps/edge/internal/service"
|
||||
"iop/packages/go/config"
|
||||
"iop/packages/go/singlerequesttemplate"
|
||||
)
|
||||
|
||||
const (
|
||||
singleRequestReviewPrompt = "Review the task, plan, completed work, and verification evidence. Return exactly one JSON object with decision=pass, non-empty output, and non-empty summary when approved. Otherwise make exactly one approved workspace tool call to inspect or repair, with no text content. After a tool result with error_code=not_found, do not pass or inspect again; make one repair tool call."
|
||||
singleRequestReviewPrompt = "Review the task, plan, completed work, and verification evidence. Return exactly one JSON object with decision=pass, non-empty output, non-empty checks, non-empty verification, and non-empty summary when approved. Otherwise make exactly one approved workspace tool call to inspect or repair, with no text content. After a tool result with error_code=not_found, do not pass or inspect again; make one repair tool call."
|
||||
singleRequestReviewStageID = "review"
|
||||
)
|
||||
|
||||
|
|
@ -48,13 +49,15 @@ type singleRequestReviewResult struct {
|
|||
}
|
||||
|
||||
type singleRequestReviewDecision struct {
|
||||
Decision string `json:"decision"`
|
||||
Output string `json:"output"`
|
||||
Summary string `json:"summary"`
|
||||
Decision string `json:"decision"`
|
||||
Output string `json:"output"`
|
||||
Checks string `json:"checks"`
|
||||
Verification string `json:"verification"`
|
||||
Summary string `json:"summary"`
|
||||
}
|
||||
|
||||
func (v *singleRequestReviewDecision) UnmarshalJSON(data []byte) error {
|
||||
if err := validateSingleRequestObjectFields(data, "decision", "output", "summary"); err != nil {
|
||||
if err := validateSingleRequestObjectFields(data, "decision", "output", "checks", "verification", "summary"); err != nil {
|
||||
return err
|
||||
}
|
||||
type alias singleRequestReviewDecision
|
||||
|
|
@ -80,7 +83,7 @@ func (s *singleRequestReviewStage) run(ctx context.Context, req singleRequestRev
|
|||
return nil, quality.validation(errSingleRequestReviewStage)
|
||||
}
|
||||
binding := ctrl.Binding()
|
||||
if binding == nil || binding.Workspace == nil || binding.Workspace.NodeID == "" || binding.Workspace.NodeID != req.NodeRef {
|
||||
if binding == nil || binding.Workspace == nil || binding.Workspace.NodeID == "" || binding.Workspace.NodeID != req.NodeRef || binding.Templates.Review == "" {
|
||||
return nil, quality.validation(errSingleRequestReviewStage)
|
||||
}
|
||||
plan, err := ctrl.ReadInternalArtifact(ctx, edgeservice.SingleRequestArtifactPlan)
|
||||
|
|
@ -116,7 +119,7 @@ func (s *singleRequestReviewStage) run(ctx context.Context, req singleRequestRev
|
|||
if repairRequired {
|
||||
return nil, quality.malformed(errSingleRequestReviewStage)
|
||||
}
|
||||
artifact, result, err := renderSingleRequestReview(*response.pass, req.Limits.MaxOutputBytes)
|
||||
artifact, result, err := renderSingleRequestReview(binding.Templates.Review, *response.pass, req.Limits.MaxOutputBytes)
|
||||
if err != nil {
|
||||
return nil, quality.malformed(errSingleRequestReviewStage)
|
||||
}
|
||||
|
|
@ -416,20 +419,31 @@ func decodeSingleRequestReviewDecision(raw string, maximum int) (*singleRequestR
|
|||
return nil, errSingleRequestReviewStage
|
||||
}
|
||||
var extra any
|
||||
if err := decoder.Decode(&extra); err != io.EOF || decision.Decision != "pass" || strings.TrimSpace(decision.Output) == "" || strings.TrimSpace(decision.Summary) == "" {
|
||||
if err := decoder.Decode(&extra); err != io.EOF || decision.Decision != "pass" || strings.TrimSpace(decision.Output) == "" || strings.TrimSpace(decision.Checks) == "" || strings.TrimSpace(decision.Verification) == "" || strings.TrimSpace(decision.Summary) == "" {
|
||||
return nil, errSingleRequestReviewStage
|
||||
}
|
||||
return &decision, nil
|
||||
}
|
||||
|
||||
func renderSingleRequestReview(decision singleRequestReviewDecision, maximum int) ([]byte, *singleRequestReviewResult, error) {
|
||||
func renderSingleRequestReview(tmpl string, decision singleRequestReviewDecision, maximum int) ([]byte, *singleRequestReviewResult, error) {
|
||||
if decision.Decision != "pass" || maximum < 1 {
|
||||
return nil, nil, errSingleRequestReviewStage
|
||||
}
|
||||
output, summary := strings.TrimSpace(decision.Output), strings.TrimSpace(decision.Summary)
|
||||
artifact := []byte("# Review\n\n" + summary + "\n")
|
||||
if output == "" || summary == "" || len(output) > maximum || len(artifact) > maximum {
|
||||
output := strings.TrimSpace(decision.Output)
|
||||
if output == "" {
|
||||
return nil, nil, errSingleRequestReviewStage
|
||||
}
|
||||
return artifact, &singleRequestReviewResult{Output: append([]byte(nil), []byte(output)...), Summary: summary}, nil
|
||||
artifact, err := singlerequesttemplate.RenderReview(
|
||||
tmpl,
|
||||
singlerequesttemplate.ReviewFields{
|
||||
Checks: decision.Checks,
|
||||
Verification: decision.Verification,
|
||||
Summary: decision.Summary,
|
||||
},
|
||||
maximum,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, nil, errSingleRequestReviewStage
|
||||
}
|
||||
return artifact, &singleRequestReviewResult{Output: append([]byte(nil), []byte(output)...), Summary: strings.TrimSpace(decision.Summary)}, nil
|
||||
}
|
||||
|
|
|
|||
|
|
@ -17,6 +17,7 @@ import (
|
|||
edgenode "iop/apps/edge/internal/node"
|
||||
edgeservice "iop/apps/edge/internal/service"
|
||||
"iop/packages/go/config"
|
||||
"iop/packages/go/singlerequesttemplate"
|
||||
iop "iop/proto/gen/iop"
|
||||
)
|
||||
|
||||
|
|
@ -83,7 +84,13 @@ func reviewRequest(t *testing.T) singleRequestReviewStageRequest {
|
|||
}
|
||||
|
||||
func reviewPassBody(output, summary string) []byte {
|
||||
decision, _ := json.Marshal(map[string]string{"decision": "pass", "output": output, "summary": summary})
|
||||
decision, _ := json.Marshal(map[string]string{
|
||||
"decision": "pass",
|
||||
"output": output,
|
||||
"checks": "- Checked requirements",
|
||||
"verification": "- Verified tests pass",
|
||||
"summary": summary,
|
||||
})
|
||||
b, _ := json.Marshal(map[string]any{"id": "id", "object": "chat.completion", "created": 1, "model": "gemini-3.6-flash", "choices": []any{map[string]any{"index": 0, "finish_reason": "stop", "message": map[string]any{"role": "assistant", "content": string(decision), "reasoning_content": "provider-private-review-reasoning", "extra_content": map[string]any{"google": map[string]any{"thought_signature": "provider-private-final-signature"}}}}}})
|
||||
return b
|
||||
}
|
||||
|
|
@ -305,7 +312,8 @@ func TestSingleRequestReviewStagePassPersistsBeforeFinalizing(t *testing.T) {
|
|||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if string(result.Output) != "Approved output." || result.Summary != "All checks passed." || string(ctrl.artifact) != "# Review\n\nAll checks passed.\n" || len(ctrl.writes) != 1 || ctrl.writes[0] != edgeservice.SingleRequestArtifactReview {
|
||||
expectedArtifact := "# Review\n\n## Result\nPASS\n\n## Checks\n- Checked requirements\n\n## Verification\n- Verified tests pass\n\n## Summary\nAll checks passed.\n"
|
||||
if string(result.Output) != "Approved output." || result.Summary != "All checks passed." || string(ctrl.artifact) != expectedArtifact || len(ctrl.writes) != 1 || ctrl.writes[0] != edgeservice.SingleRequestArtifactReview {
|
||||
t.Fatalf("result=%+v artifact=%q writes=%v", result, ctrl.artifact, ctrl.writes)
|
||||
}
|
||||
if len(ctrl.envelopes) != 2 || ctrl.envelopes[0].Stage != edgeservice.SingleRequestStateReviewing || ctrl.envelopes[1].Stage != edgeservice.SingleRequestStateFinalizing || ctrl.envelopes[1].Result == nil || ctrl.envelopes[1].Result.Output != "Approved output." {
|
||||
|
|
@ -377,10 +385,14 @@ func TestSingleRequestReviewStageInspectionAndRepairRemainInLegalStates(t *testi
|
|||
|
||||
func TestSingleRequestReviewStageFailsClosed(t *testing.T) {
|
||||
for _, raw := range []string{
|
||||
`{"decision":"pass","output":"x","summary":"y","extra":1}`,
|
||||
`{"decision":"pass","output":"","summary":"y"}`,
|
||||
`{"decision":"repair","output":"x","summary":"y"}`,
|
||||
`{"decision":"pass","decision":"pass","output":"x","summary":"y"}`,
|
||||
`{"decision":"pass","output":"x","checks":"c","verification":"v","summary":"y","extra":1}`,
|
||||
`{"decision":"pass","output":"","checks":"c","verification":"v","summary":"y"}`,
|
||||
`{"decision":"pass","output":"x","checks":"","verification":"v","summary":"y"}`,
|
||||
`{"decision":"pass","output":"x","checks":"c","verification":"","summary":"y"}`,
|
||||
`{"decision":"pass","output":"x","checks":"c","verification":"v","summary":""}`,
|
||||
`{"decision":"repair","output":"x","checks":"c","verification":"v","summary":"y"}`,
|
||||
`{"decision":"pass","decision":"pass","output":"x","checks":"c","verification":"v","summary":"y"}`,
|
||||
`{"decision":"pass","output":"x","summary":"y"}`, // missing checks & verification
|
||||
`not-json`,
|
||||
} {
|
||||
if _, err := decodeSingleRequestReviewDecision(raw, 4096); !errors.Is(err, errSingleRequestReviewStage) {
|
||||
|
|
@ -390,9 +402,9 @@ func TestSingleRequestReviewStageFailsClosed(t *testing.T) {
|
|||
for _, raw := range []string{
|
||||
`{"id":"x","object":"chat.completion","created":1,"model":"m","choices":[]}`,
|
||||
`{"id":"x","object":"chat.completion","created":1,"model":"m","choices":[{"index":0,"finish_reason":"tool_calls","message":{"role":"assistant","content":null,"tool_calls":[{"id":"a","type":"function","function":{"name":"workspace_read","arguments":"{}"}},{"id":"b","type":"function","function":{"name":"workspace_read","arguments":"{}"}}]}}]}`,
|
||||
`{"id":"x","object":"chat.completion","created":1,"model":"m","choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"{\"decision\":\"pass\",\"output\":\"x\",\"summary\":\"y\"}","reasoning_content":["private"]}}]}`,
|
||||
`{"id":"x","object":"chat.completion","created":1,"model":"m","choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"{\"decision\":\"pass\",\"output\":\"x\",\"summary\":\"y\"}","extra_content":null}}]}`,
|
||||
`{"id":"x","object":"chat.completion","created":1,"model":"m","choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"{\"decision\":\"pass\",\"output\":\"x\",\"summary\":\"y\"}","extra_content":{"google":{"thought_signature":false}}}}]}`,
|
||||
`{"id":"x","object":"chat.completion","created":1,"model":"m","choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"{\"decision\":\"pass\",\"output\":\"x\",\"checks\":\"c\",\"verification\":\"v\",\"summary\":\"y\"}","reasoning_content":["private"]}}]}`,
|
||||
`{"id":"x","object":"chat.completion","created":1,"model":"m","choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"{\"decision\":\"pass\",\"output\":\"x\",\"checks\":\"c\",\"verification\":\"v\",\"summary\":\"y\"}","extra_content":null}}]}`,
|
||||
`{"id":"x","object":"chat.completion","created":1,"model":"m","choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"{\"decision\":\"pass\",\"output\":\"x\",\"checks\":\"c\",\"verification\":\"v\",\"summary\":\"y\"}","extra_content":{"google":{"thought_signature":false}}}}]}`,
|
||||
`{"id":"x","object":"chat.completion","created":1,"model":"m","choices":[{"index":0,"finish_reason":"tool_calls","message":{"role":"assistant","content":null,"tool_calls":[{"id":"a","type":"function","function":{"name":"workspace_read","arguments":"{}"},"extra_content":null}]}}]}`,
|
||||
`{"id":"x","object":"chat.completion","created":1,"model":"m","choices":[{"index":0,"finish_reason":"tool_calls","message":{"role":"assistant","content":null,"tool_calls":[{"id":"a","type":"function","function":{"name":"workspace_read","arguments":"{}"},"extra_content":{"google":{"thought_signature":""}}}]}}]}`,
|
||||
`{"id":"x","object":"chat.completion","created":1,"model":"m","choices":[{"index":0,"finish_reason":"tool_calls","message":{"role":"assistant","content":null,"tool_calls":[{"id":"a","type":"function","function":{"name":"workspace_read","arguments":"{}"},"extra_content":{"google":{"thought_signature":"sig","unknown":1}}}]}}]}`,
|
||||
|
|
@ -459,7 +471,7 @@ func TestSingleRequestReviewBodyRejectsOptionAliases(t *testing.T) {
|
|||
if _, err := buildSingleRequestReviewBody([]chatMessage{{Role: "user", Content: "x"}}, map[string]any{"reasoning_effort": "low"}, []any{singleRequestWorkToolSchema(edgeservice.InternalWorkspaceToolRead, map[string]any{"type": "object"})}, "gemini", false); !errors.Is(err, errSingleRequestReviewStage) {
|
||||
t.Fatalf("err=%v", err)
|
||||
}
|
||||
if _, _, err := renderSingleRequestReview(singleRequestReviewDecision{Decision: "pass", Output: strings.Repeat("x", 10), Summary: "summary"}, 9); !errors.Is(err, errSingleRequestReviewStage) {
|
||||
if _, _, err := renderSingleRequestReview(singlerequesttemplate.DefaultReviewTemplate, singleRequestReviewDecision{Decision: "pass", Output: strings.Repeat("x", 10), Checks: "c", Verification: "v", Summary: "summary"}, 9); !errors.Is(err, errSingleRequestReviewStage) {
|
||||
t.Fatalf("err=%v", err)
|
||||
}
|
||||
}
|
||||
|
|
@ -477,7 +489,7 @@ func expectedSingleRequestReviewBodyAuthority(isResumed bool) map[string]any {
|
|||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"content": "Review the task, plan, completed work, and verification evidence. Return exactly one JSON object with decision=pass, non-empty output, and non-empty summary when approved. Otherwise make exactly one approved workspace tool call to inspect or repair, with no text content. After a tool result with error_code=not_found, do not pass or inspect again; make one repair tool call."
|
||||
"content": "Review the task, plan, completed work, and verification evidence. Return exactly one JSON object with decision=pass, non-empty output, non-empty checks, non-empty verification, and non-empty summary when approved. Otherwise make exactly one approved workspace tool call to inspect or repair, with no text content. After a tool result with error_code=not_found, do not pass or inspect again; make one repair tool call."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
|
|
@ -596,7 +608,7 @@ func expectedSingleRequestReviewBodyAuthority(isResumed bool) map[string]any {
|
|||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"content": "Review the task, plan, completed work, and verification evidence. Return exactly one JSON object with decision=pass, non-empty output, and non-empty summary when approved. Otherwise make exactly one approved workspace tool call to inspect or repair, with no text content. After a tool result with error_code=not_found, do not pass or inspect again; make one repair tool call."
|
||||
"content": "Review the task, plan, completed work, and verification evidence. Return exactly one JSON object with decision=pass, non-empty output, non-empty checks, non-empty verification, and non-empty summary when approved. Otherwise make exactly one approved workspace tool call to inspect or repair, with no text content. After a tool result with error_code=not_found, do not pass or inspect again; make one repair tool call."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@ import (
|
|||
"sort"
|
||||
|
||||
"iop/packages/go/config"
|
||||
"iop/packages/go/singlerequesttemplate"
|
||||
)
|
||||
|
||||
var (
|
||||
|
|
@ -64,6 +65,9 @@ type SingleRequestBinding struct {
|
|||
// exceed wall_clock_ms.
|
||||
Limits SingleRequestLimits
|
||||
|
||||
// Templates contains the frozen effective Plan and Review Markdown templates.
|
||||
Templates SingleRequestTemplateBinding
|
||||
|
||||
// Workspace is populated only by Service workspace admission. It contains
|
||||
// the request-stable, coordinator-safe capability projection; in
|
||||
// particular it intentionally excludes roots, command templates, and
|
||||
|
|
@ -71,6 +75,13 @@ type SingleRequestBinding struct {
|
|||
Workspace *SingleRequestWorkspaceBinding
|
||||
}
|
||||
|
||||
// SingleRequestTemplateBinding carries the content-only frozen effective Plan
|
||||
// and Review templates snapshot compiled at admission time.
|
||||
type SingleRequestTemplateBinding struct {
|
||||
Plan string
|
||||
Review string
|
||||
}
|
||||
|
||||
// SingleRequestWorkspaceBinding is the coordinator-safe result of one exact
|
||||
// workspace capability admission. It freezes a configured Node id and its
|
||||
// dispatch-ready connection generation together with the closed operation and
|
||||
|
|
@ -215,6 +226,15 @@ type SingleRequestLimits struct {
|
|||
// revision fields and deep-cloned so the caller cannot alter an admitted
|
||||
// binding through the original reference.
|
||||
func NewSingleRequestBinding(publicModel, workspaceRef string, plan, work, review SingleRequestStageBinding, limits SingleRequestLimits) (*SingleRequestBinding, error) {
|
||||
return NewSingleRequestBindingWithTemplates(publicModel, workspaceRef, plan, work, review, limits, SingleRequestTemplateBinding{
|
||||
Plan: singlerequesttemplate.DefaultPlanTemplate,
|
||||
Review: singlerequesttemplate.DefaultReviewTemplate,
|
||||
})
|
||||
}
|
||||
|
||||
// NewSingleRequestBindingWithTemplates constructs a validated, defensive-copy admission
|
||||
// value including explicit effective Plan and Review templates.
|
||||
func NewSingleRequestBindingWithTemplates(publicModel, workspaceRef string, plan, work, review SingleRequestStageBinding, limits SingleRequestLimits, templates SingleRequestTemplateBinding) (*SingleRequestBinding, error) {
|
||||
if publicModel == "" {
|
||||
return nil, errSingleRequestMissingPublicModel
|
||||
}
|
||||
|
|
@ -231,6 +251,13 @@ func NewSingleRequestBinding(publicModel, workspaceRef string, plan, work, revie
|
|||
return nil, errSingleRequestMissingReview
|
||||
}
|
||||
|
||||
if err := singlerequesttemplate.ValidatePlanTemplate(templates.Plan); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := singlerequesttemplate.ValidateReviewTemplate(templates.Review); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
limits, err := validateSingleRequestLimits(limits)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
|
|
@ -260,6 +287,10 @@ func NewSingleRequestBinding(publicModel, workspaceRef string, plan, work, revie
|
|||
Work: workCopy,
|
||||
Review: reviewCopy,
|
||||
Limits: limits,
|
||||
Templates: SingleRequestTemplateBinding{
|
||||
Plan: templates.Plan,
|
||||
Review: templates.Review,
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
|
||||
|
|
@ -289,6 +320,7 @@ func (b *SingleRequestBinding) Clone() *SingleRequestBinding {
|
|||
Dispatch: b.Review.Dispatch.Clone(),
|
||||
},
|
||||
Limits: b.Limits,
|
||||
Templates: b.Templates,
|
||||
Workspace: b.Workspace.Clone(),
|
||||
}
|
||||
}
|
||||
|
|
@ -314,13 +346,14 @@ func cloneValidatedSingleRequestBinding(binding *SingleRequestBinding) (*SingleR
|
|||
if binding == nil {
|
||||
return nil, errSingleRequestWorkspaceMalformed
|
||||
}
|
||||
base, err := NewSingleRequestBinding(
|
||||
base, err := NewSingleRequestBindingWithTemplates(
|
||||
binding.PublicModel,
|
||||
binding.WorkspaceRef,
|
||||
binding.Plan,
|
||||
binding.Work,
|
||||
binding.Review,
|
||||
binding.Limits,
|
||||
binding.Templates,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
|
|
|
|||
|
|
@ -591,3 +591,6 @@ nodes:
|
|||
# model: "qwen3.6:35b"
|
||||
# options:
|
||||
# reasoning_effort: "high"
|
||||
# templates:
|
||||
# plan_file: "templates/plan.md" # relative to this edge.yaml
|
||||
# review_file: "templates/review.md" # relative to this edge.yaml
|
||||
|
|
|
|||
|
|
@ -5,6 +5,8 @@ import (
|
|||
"reflect"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"iop/packages/go/singlerequesttemplate"
|
||||
)
|
||||
|
||||
// ExecutionPreset declares one frozen execution shape.
|
||||
|
|
@ -277,6 +279,15 @@ type ExecutionSingleRequestPolicy struct {
|
|||
Limits ExecutionSingleRequestLimits `mapstructure:"limits" yaml:"limits"`
|
||||
// Stages declares the approved fixed stage map: plan, work, review.
|
||||
Stages ExecutionSingleRequestStages `mapstructure:"stages" yaml:"stages"`
|
||||
// Templates declares optional operator template file paths and internal effective templates.
|
||||
Templates ExecutionSingleRequestTemplates `mapstructure:"templates" yaml:"templates,omitempty"`
|
||||
}
|
||||
|
||||
type ExecutionSingleRequestTemplates struct {
|
||||
PlanFile string `mapstructure:"plan_file" yaml:"plan_file,omitempty"`
|
||||
ReviewFile string `mapstructure:"review_file" yaml:"review_file,omitempty"`
|
||||
EffectivePlan string `mapstructure:"-" yaml:"-"`
|
||||
EffectiveReview string `mapstructure:"-" yaml:"-"`
|
||||
}
|
||||
|
||||
// ExecutionSingleRequestLimits carries server-owned absolute resource caps.
|
||||
|
|
@ -325,6 +336,7 @@ func (p *ExecutionSingleRequestPolicy) Clone() *ExecutionSingleRequestPolicy {
|
|||
Work: p.Stages.Work.Clone(),
|
||||
Review: p.Stages.Review.Clone(),
|
||||
},
|
||||
Templates: p.Templates,
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
|
@ -752,6 +764,17 @@ func validateSingleRequestPolicy(presetIndex int, p *ExecutionPreset, allowedMod
|
|||
}
|
||||
}
|
||||
|
||||
if sr.Templates.EffectivePlan != "" {
|
||||
if err := singlerequesttemplate.ValidatePlanTemplate(sr.Templates.EffectivePlan); err != nil {
|
||||
return fmt.Errorf("execution_presets[%d] id=%q: invalid single_request.templates.plan_file: %w", presetIndex, p.ID, err)
|
||||
}
|
||||
}
|
||||
if sr.Templates.EffectiveReview != "" {
|
||||
if err := singlerequesttemplate.ValidateReviewTemplate(sr.Templates.EffectiveReview); err != nil {
|
||||
return fmt.Errorf("execution_presets[%d] id=%q: invalid single_request.templates.review_file: %w", presetIndex, p.ID, err)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -2,9 +2,13 @@ package config
|
|||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"iop/packages/go/singlerequesttemplate"
|
||||
|
||||
"github.com/mitchellh/mapstructure"
|
||||
"github.com/spf13/viper"
|
||||
)
|
||||
|
|
@ -201,6 +205,10 @@ func LoadEdge(cfgFile string) (*EdgeConfig, error) {
|
|||
}
|
||||
}
|
||||
|
||||
if err := resolveSingleRequestTemplates(cfg.ExecutionPresets, cfgFile); err != nil {
|
||||
return nil, fmt.Errorf("execution_presets: %w", err)
|
||||
}
|
||||
|
||||
// Validate and normalize execution presets before model admission. Preset
|
||||
// validation runs early so that invalid preset shapes fail closed before
|
||||
// any runtime dispatch path can observe them.
|
||||
|
|
@ -671,3 +679,71 @@ func validateWorkspaceNumericLimits(ws WorkspaceDefinition, nodeIdx, wsIdx int)
|
|||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func resolveSingleRequestTemplates(presets []ExecutionPreset, configFilePath string) error {
|
||||
baseDir := filepath.Dir(configFilePath)
|
||||
for i := range presets {
|
||||
p := &presets[i]
|
||||
if p.SingleRequest == nil {
|
||||
continue
|
||||
}
|
||||
sr := p.SingleRequest
|
||||
|
||||
if strings.TrimSpace(sr.Templates.PlanFile) == "" {
|
||||
sr.Templates.EffectivePlan = singlerequesttemplate.DefaultPlanTemplate
|
||||
} else {
|
||||
content, err := loadTemplateFile(baseDir, sr.Templates.PlanFile)
|
||||
if err != nil {
|
||||
return fmt.Errorf("presets[%d] id=%q single_request.templates.plan_file: %w", i, p.ID, err)
|
||||
}
|
||||
if err := singlerequesttemplate.ValidatePlanTemplate(content); err != nil {
|
||||
return fmt.Errorf("presets[%d] id=%q single_request.templates.plan_file: %w", i, p.ID, err)
|
||||
}
|
||||
sr.Templates.EffectivePlan = content
|
||||
}
|
||||
|
||||
if strings.TrimSpace(sr.Templates.ReviewFile) == "" {
|
||||
sr.Templates.EffectiveReview = singlerequesttemplate.DefaultReviewTemplate
|
||||
} else {
|
||||
content, err := loadTemplateFile(baseDir, sr.Templates.ReviewFile)
|
||||
if err != nil {
|
||||
return fmt.Errorf("presets[%d] id=%q single_request.templates.review_file: %w", i, p.ID, err)
|
||||
}
|
||||
if err := singlerequesttemplate.ValidateReviewTemplate(content); err != nil {
|
||||
return fmt.Errorf("presets[%d] id=%q single_request.templates.review_file: %w", i, p.ID, err)
|
||||
}
|
||||
sr.Templates.EffectiveReview = content
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func loadTemplateFile(baseDir, relOrAbsPath string) (string, error) {
|
||||
target := strings.TrimSpace(relOrAbsPath)
|
||||
if !filepath.IsAbs(target) {
|
||||
target = filepath.Join(baseDir, target)
|
||||
}
|
||||
|
||||
st, err := os.Lstat(target)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if !st.Mode().IsRegular() {
|
||||
return "", fmt.Errorf("template file %q must be a regular file (mode %v)", target, st.Mode())
|
||||
}
|
||||
|
||||
f, err := os.Open(target)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
data, err := io.ReadAll(io.LimitReader(f, 8193))
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if len(data) > singlerequesttemplate.MaxTemplateBytes {
|
||||
return "", fmt.Errorf("template file %q size %d exceeds max %d bytes", target, len(data), singlerequesttemplate.MaxTemplateBytes)
|
||||
}
|
||||
return string(data), nil
|
||||
}
|
||||
|
|
|
|||
|
|
@ -514,3 +514,180 @@ func TestModelCatalogEntry_ValidateVirtualEntryUnit(t *testing.T) {
|
|||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestLoadEdgeSingleRequestTemplates(t *testing.T) {
|
||||
validPresetYAML := func(planFile, reviewFile string) string {
|
||||
tmplSection := ""
|
||||
if planFile != "" || reviewFile != "" {
|
||||
tmplSection = " templates:\n"
|
||||
if planFile != "" {
|
||||
tmplSection += " plan_file: " + planFile + "\n"
|
||||
}
|
||||
if reviewFile != "" {
|
||||
tmplSection += " review_file: " + reviewFile + "\n"
|
||||
}
|
||||
}
|
||||
return `
|
||||
server:
|
||||
listen: "0.0.0.0:9090"
|
||||
models:
|
||||
- id: "qwen3.6:35b"
|
||||
providers:
|
||||
vllm-gpu: "nvidia/Qwen3.6-35B"
|
||||
- id: "virtual-sr"
|
||||
execution_preset: "sr-preset"
|
||||
execution_presets:
|
||||
- id: "sr-preset"
|
||||
selector:
|
||||
model: "qwen3.6:35b"
|
||||
options:
|
||||
reasoning_effort: "high"
|
||||
allowed_modes:
|
||||
- "light"
|
||||
routes:
|
||||
light:
|
||||
stages:
|
||||
- role: "plan"
|
||||
model: "qwen3.6:35b"
|
||||
options:
|
||||
reasoning_effort: "high"
|
||||
- role: "work"
|
||||
model: "qwen3.6:35b"
|
||||
- role: "review"
|
||||
model: "qwen3.6:35b"
|
||||
options:
|
||||
reasoning_effort: "high"
|
||||
single_request:
|
||||
workspace_ref: "ws-1"
|
||||
limits:
|
||||
wall_clock_ms: 60000
|
||||
timeout_ms: 30000
|
||||
max_tool_iterations: 10
|
||||
max_output_bytes: 1024
|
||||
stages:
|
||||
plan:
|
||||
model: "qwen3.6:35b"
|
||||
options:
|
||||
reasoning_effort: "high"
|
||||
work:
|
||||
model: "qwen3.6:35b"
|
||||
review:
|
||||
model: "qwen3.6:35b"
|
||||
options:
|
||||
reasoning_effort: "high"
|
||||
` + tmplSection + `
|
||||
nodes:
|
||||
- id: "node-gpu-01"
|
||||
providers:
|
||||
- id: "vllm-gpu"
|
||||
type: "vllm"
|
||||
category: "api"
|
||||
models:
|
||||
- "nvidia/Qwen3.6-35B"
|
||||
capacity: 4
|
||||
`
|
||||
}
|
||||
|
||||
t.Run("built-in template fallback when templates omitted", func(t *testing.T) {
|
||||
rootDir := t.TempDir()
|
||||
cfgSubdir := filepath.Join(rootDir, "sub", "config")
|
||||
if err := os.MkdirAll(cfgSubdir, 0o755); err != nil {
|
||||
t.Fatalf("mkdir: %v", err)
|
||||
}
|
||||
cfgPath := filepath.Join(cfgSubdir, "edge.yaml")
|
||||
if err := os.WriteFile(cfgPath, []byte(validPresetYAML("", "")), 0o600); err != nil {
|
||||
t.Fatalf("write yaml: %v", err)
|
||||
}
|
||||
|
||||
cfg, err := config.LoadEdge(cfgPath)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadEdge failed: %v", err)
|
||||
}
|
||||
sr := cfg.ExecutionPresets[0].SingleRequest
|
||||
if sr.Templates.EffectivePlan == "" || sr.Templates.EffectiveReview == "" {
|
||||
t.Errorf("expected effective templates to be set to built-ins")
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("relative path template resolution config-relative", func(t *testing.T) {
|
||||
rootDir := t.TempDir()
|
||||
cfgSubdir := filepath.Join(rootDir, "sub", "config")
|
||||
tmplSubdir := filepath.Join(cfgSubdir, "tmpl")
|
||||
if err := os.MkdirAll(tmplSubdir, 0o755); err != nil {
|
||||
t.Fatalf("mkdir: %v", err)
|
||||
}
|
||||
cfgPath := filepath.Join(cfgSubdir, "edge.yaml")
|
||||
|
||||
customPlan := "# Plan\n\n## Goal\n{{goal}}\n\n## Steps\n{{steps}}\n\n## Verification\n{{verification}}\n"
|
||||
customReview := "# Review\n\n## Result\nPASS\n\n## Checks\n{{checks}}\n\n## Verification\n{{verification}}\n\n## Summary\n{{summary}}\n"
|
||||
|
||||
if err := os.WriteFile(filepath.Join(tmplSubdir, "custom_plan.md"), []byte(customPlan), 0o600); err != nil {
|
||||
t.Fatalf("write custom plan: %v", err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(tmplSubdir, "custom_review.md"), []byte(customReview), 0o600); err != nil {
|
||||
t.Fatalf("write custom review: %v", err)
|
||||
}
|
||||
if err := os.WriteFile(cfgPath, []byte(validPresetYAML("tmpl/custom_plan.md", "tmpl/custom_review.md")), 0o600); err != nil {
|
||||
t.Fatalf("write yaml: %v", err)
|
||||
}
|
||||
|
||||
cfg, err := config.LoadEdge(cfgPath)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadEdge failed: %v", err)
|
||||
}
|
||||
sr := cfg.ExecutionPresets[0].SingleRequest
|
||||
if sr.Templates.EffectivePlan != customPlan {
|
||||
t.Errorf("EffectivePlan = %q, want %q", sr.Templates.EffectivePlan, customPlan)
|
||||
}
|
||||
if sr.Templates.EffectiveReview != customReview {
|
||||
t.Errorf("EffectiveReview = %q, want %q", sr.Templates.EffectiveReview, customReview)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("missing configured template file fails closed", func(t *testing.T) {
|
||||
rootDir := t.TempDir()
|
||||
cfgPath := filepath.Join(rootDir, "edge.yaml")
|
||||
if err := os.WriteFile(cfgPath, []byte(validPresetYAML("nonexistent_plan.md", "")), 0o600); err != nil {
|
||||
t.Fatalf("write yaml: %v", err)
|
||||
}
|
||||
|
||||
_, err := config.LoadEdge(cfgPath)
|
||||
if err == nil {
|
||||
t.Fatalf("expected error for missing plan template file")
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("oversized template file 8193 bytes fails closed", func(t *testing.T) {
|
||||
rootDir := t.TempDir()
|
||||
cfgPath := filepath.Join(rootDir, "edge.yaml")
|
||||
oversizedPlan := "# Plan\n\n## Goal\n{{goal}}\n\n## Steps\n{{steps}}\n\n## Verification\n{{verification}}\n" + strings.Repeat(" ", 8193)
|
||||
if err := os.WriteFile(filepath.Join(rootDir, "oversized.md"), []byte(oversizedPlan), 0o600); err != nil {
|
||||
t.Fatalf("write oversized plan: %v", err)
|
||||
}
|
||||
if err := os.WriteFile(cfgPath, []byte(validPresetYAML("oversized.md", "")), 0o600); err != nil {
|
||||
t.Fatalf("write yaml: %v", err)
|
||||
}
|
||||
|
||||
_, err := config.LoadEdge(cfgPath)
|
||||
if err == nil {
|
||||
t.Fatalf("expected error for oversized template file")
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("invalid template content missing tokens fails closed", func(t *testing.T) {
|
||||
rootDir := t.TempDir()
|
||||
cfgPath := filepath.Join(rootDir, "edge.yaml")
|
||||
badPlan := "# Plan\n\n## Goal\nNo placeholders here\n"
|
||||
if err := os.WriteFile(filepath.Join(rootDir, "bad_plan.md"), []byte(badPlan), 0o600); err != nil {
|
||||
t.Fatalf("write bad plan: %v", err)
|
||||
}
|
||||
if err := os.WriteFile(cfgPath, []byte(validPresetYAML("bad_plan.md", "")), 0o600); err != nil {
|
||||
t.Fatalf("write yaml: %v", err)
|
||||
}
|
||||
|
||||
_, err := config.LoadEdge(cfgPath)
|
||||
if err == nil {
|
||||
t.Fatalf("expected error for invalid template content")
|
||||
}
|
||||
})
|
||||
}
|
||||
|
|
|
|||
273
packages/go/singlerequesttemplate/template.go
Normal file
273
packages/go/singlerequesttemplate/template.go
Normal file
|
|
@ -0,0 +1,273 @@
|
|||
package singlerequesttemplate
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const MaxTemplateBytes = 8192
|
||||
|
||||
const DefaultPlanTemplate = `# Plan
|
||||
|
||||
## Goal
|
||||
{{goal}}
|
||||
|
||||
## Steps
|
||||
{{steps}}
|
||||
|
||||
## Verification
|
||||
{{verification}}
|
||||
`
|
||||
|
||||
const DefaultReviewTemplate = `# Review
|
||||
|
||||
## Result
|
||||
PASS
|
||||
|
||||
## Checks
|
||||
{{checks}}
|
||||
|
||||
## Verification
|
||||
{{verification}}
|
||||
|
||||
## Summary
|
||||
{{summary}}
|
||||
`
|
||||
|
||||
var (
|
||||
ErrInvalidTemplate = errors.New("single-request template: invalid template")
|
||||
ErrTemplateTooLarge = errors.New("single-request template: exceeds maximum size")
|
||||
ErrMalformedPlan = errors.New("single-request template: malformed plan output")
|
||||
ErrMalformedReview = errors.New("single-request template: malformed review fields")
|
||||
)
|
||||
|
||||
var placeholderRegex = regexp.MustCompile(`\{\{[^}]*\}\}`)
|
||||
|
||||
type ReviewFields struct {
|
||||
Checks string
|
||||
Verification string
|
||||
Summary string
|
||||
}
|
||||
|
||||
func Digest(content string) string {
|
||||
h := sha256.Sum256([]byte(content))
|
||||
return hex.EncodeToString(h[:])
|
||||
}
|
||||
|
||||
func ValidatePlanTemplate(tmpl string) error {
|
||||
if len(tmpl) == 0 {
|
||||
return fmt.Errorf("%w: template is empty", ErrInvalidTemplate)
|
||||
}
|
||||
if len(tmpl) > MaxTemplateBytes {
|
||||
return fmt.Errorf("%w: template size %d exceeds max %d", ErrTemplateTooLarge, len(tmpl), MaxTemplateBytes)
|
||||
}
|
||||
|
||||
if strings.Count(tmpl, "{{goal}}") != 1 {
|
||||
return fmt.Errorf("%w: must contain {{goal}} exactly once", ErrInvalidTemplate)
|
||||
}
|
||||
if strings.Count(tmpl, "{{steps}}") != 1 {
|
||||
return fmt.Errorf("%w: must contain {{steps}} exactly once", ErrInvalidTemplate)
|
||||
}
|
||||
if strings.Count(tmpl, "{{verification}}") != 1 {
|
||||
return fmt.Errorf("%w: must contain {{verification}} exactly once", ErrInvalidTemplate)
|
||||
}
|
||||
|
||||
matches := placeholderRegex.FindAllString(tmpl, -1)
|
||||
for _, m := range matches {
|
||||
if m != "{{goal}}" && m != "{{steps}}" && m != "{{verification}}" {
|
||||
return fmt.Errorf("%w: unknown placeholder %q", ErrInvalidTemplate, m)
|
||||
}
|
||||
}
|
||||
|
||||
idxGoal := strings.Index(tmpl, "{{goal}}")
|
||||
idxSteps := strings.Index(tmpl, "{{steps}}")
|
||||
idxVerif := strings.Index(tmpl, "{{verification}}")
|
||||
if !(idxGoal < idxSteps && idxSteps < idxVerif) {
|
||||
return fmt.Errorf("%w: placeholders must appear in order {{goal}}, {{steps}}, {{verification}}", ErrInvalidTemplate)
|
||||
}
|
||||
|
||||
idxPlanH := strings.Index(tmpl, "# Plan")
|
||||
idxGoalH := strings.Index(tmpl, "## Goal")
|
||||
idxStepsH := strings.Index(tmpl, "## Steps")
|
||||
idxVerifH := strings.Index(tmpl, "## Verification")
|
||||
|
||||
if idxPlanH < 0 || idxGoalH < 0 || idxStepsH < 0 || idxVerifH < 0 {
|
||||
return fmt.Errorf("%w: missing required headings (# Plan, ## Goal, ## Steps, ## Verification)", ErrInvalidTemplate)
|
||||
}
|
||||
if !(idxPlanH < idxGoalH && idxGoalH < idxGoal && idxGoal < idxStepsH && idxStepsH < idxSteps && idxSteps < idxVerifH && idxVerifH < idxVerif) {
|
||||
return fmt.Errorf("%w: headings and placeholders must follow exact structural order", ErrInvalidTemplate)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func ValidateReviewTemplate(tmpl string) error {
|
||||
if len(tmpl) == 0 {
|
||||
return fmt.Errorf("%w: template is empty", ErrInvalidTemplate)
|
||||
}
|
||||
if len(tmpl) > MaxTemplateBytes {
|
||||
return fmt.Errorf("%w: template size %d exceeds max %d", ErrTemplateTooLarge, len(tmpl), MaxTemplateBytes)
|
||||
}
|
||||
|
||||
if strings.Count(tmpl, "{{checks}}") != 1 {
|
||||
return fmt.Errorf("%w: must contain {{checks}} exactly once", ErrInvalidTemplate)
|
||||
}
|
||||
if strings.Count(tmpl, "{{verification}}") != 1 {
|
||||
return fmt.Errorf("%w: must contain {{verification}} exactly once", ErrInvalidTemplate)
|
||||
}
|
||||
if strings.Count(tmpl, "{{summary}}") != 1 {
|
||||
return fmt.Errorf("%w: must contain {{summary}} exactly once", ErrInvalidTemplate)
|
||||
}
|
||||
|
||||
matches := placeholderRegex.FindAllString(tmpl, -1)
|
||||
for _, m := range matches {
|
||||
if m != "{{checks}}" && m != "{{verification}}" && m != "{{summary}}" {
|
||||
return fmt.Errorf("%w: unknown placeholder %q", ErrInvalidTemplate, m)
|
||||
}
|
||||
}
|
||||
|
||||
idxChecks := strings.Index(tmpl, "{{checks}}")
|
||||
idxVerif := strings.Index(tmpl, "{{verification}}")
|
||||
idxSumm := strings.Index(tmpl, "{{summary}}")
|
||||
if !(idxChecks < idxVerif && idxVerif < idxSumm) {
|
||||
return fmt.Errorf("%w: placeholders must appear in order {{checks}}, {{verification}}, {{summary}}", ErrInvalidTemplate)
|
||||
}
|
||||
|
||||
idxReviewH := strings.Index(tmpl, "# Review")
|
||||
idxResultH := strings.Index(tmpl, "## Result")
|
||||
idxPass := strings.Index(tmpl, "PASS")
|
||||
idxChecksH := strings.Index(tmpl, "## Checks")
|
||||
idxVerifH := strings.Index(tmpl, "## Verification")
|
||||
idxSummH := strings.Index(tmpl, "## Summary")
|
||||
|
||||
if idxReviewH < 0 || idxResultH < 0 || idxPass < 0 || idxChecksH < 0 || idxVerifH < 0 || idxSummH < 0 {
|
||||
return fmt.Errorf("%w: missing required headings or PASS result", ErrInvalidTemplate)
|
||||
}
|
||||
if !(idxReviewH < idxResultH && idxResultH < idxPass && idxPass < idxChecksH && idxChecksH < idxChecks && idxChecks < idxVerifH && idxVerifH < idxVerif && idxVerif < idxSummH && idxSummH < idxSumm) {
|
||||
return fmt.Errorf("%w: headings and placeholders must follow exact structural order", ErrInvalidTemplate)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func ParsePlan(tmpl string, rawOutput string, maxOutputBytes int) ([]byte, error) {
|
||||
if maxOutputBytes < 1 || len(rawOutput) > maxOutputBytes {
|
||||
return nil, ErrMalformedPlan
|
||||
}
|
||||
if err := ValidatePlanTemplate(tmpl); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if strings.Contains(rawOutput, "{{") || strings.Contains(rawOutput, "}}") {
|
||||
return nil, ErrMalformedPlan
|
||||
}
|
||||
|
||||
idxGoalPlaceholder := strings.Index(tmpl, "{{goal}}")
|
||||
idxStepsPlaceholder := strings.Index(tmpl, "{{steps}}")
|
||||
idxVerifPlaceholder := strings.Index(tmpl, "{{verification}}")
|
||||
|
||||
f0 := tmpl[:idxGoalPlaceholder]
|
||||
f1 := tmpl[idxGoalPlaceholder+len("{{goal}}") : idxStepsPlaceholder]
|
||||
f2 := tmpl[idxStepsPlaceholder+len("{{steps}}") : idxVerifPlaceholder]
|
||||
f3 := tmpl[idxVerifPlaceholder+len("{{verification}}"):]
|
||||
|
||||
if !strings.HasPrefix(rawOutput, f0) {
|
||||
return nil, ErrMalformedPlan
|
||||
}
|
||||
rem := rawOutput[len(f0):]
|
||||
|
||||
i1 := strings.Index(rem, f1)
|
||||
if i1 < 0 {
|
||||
return nil, ErrMalformedPlan
|
||||
}
|
||||
vGoal := rem[:i1]
|
||||
rem = rem[i1+len(f1):]
|
||||
|
||||
i2 := strings.Index(rem, f2)
|
||||
if i2 < 0 {
|
||||
return nil, ErrMalformedPlan
|
||||
}
|
||||
vSteps := rem[:i2]
|
||||
rem = rem[i2+len(f2):]
|
||||
|
||||
var vVerif string
|
||||
if f3 == "" {
|
||||
vVerif = rem
|
||||
} else {
|
||||
if !strings.HasSuffix(rem, f3) {
|
||||
return nil, ErrMalformedPlan
|
||||
}
|
||||
vVerif = rem[:len(rem)-len(f3)]
|
||||
}
|
||||
|
||||
trimmedGoal := strings.TrimSpace(vGoal)
|
||||
if trimmedGoal == "" || strings.Contains(trimmedGoal, "\n") {
|
||||
return nil, ErrMalformedPlan
|
||||
}
|
||||
|
||||
trimmedSteps := strings.TrimSpace(vSteps)
|
||||
if trimmedSteps == "" {
|
||||
return nil, ErrMalformedPlan
|
||||
}
|
||||
stepLines := strings.Split(trimmedSteps, "\n")
|
||||
if len(stepLines) < 2 || len(stepLines) > 6 {
|
||||
return nil, ErrMalformedPlan
|
||||
}
|
||||
for _, l := range stepLines {
|
||||
trimmedLine := strings.TrimSpace(l)
|
||||
if !strings.HasPrefix(trimmedLine, "- ") || strings.TrimSpace(trimmedLine[2:]) == "" {
|
||||
return nil, ErrMalformedPlan
|
||||
}
|
||||
}
|
||||
|
||||
trimmedVerif := strings.TrimSpace(vVerif)
|
||||
if trimmedVerif == "" {
|
||||
return nil, ErrMalformedPlan
|
||||
}
|
||||
verifLines := strings.Split(trimmedVerif, "\n")
|
||||
if len(verifLines) < 1 || len(verifLines) > 3 {
|
||||
return nil, ErrMalformedPlan
|
||||
}
|
||||
for _, l := range verifLines {
|
||||
trimmedLine := strings.TrimSpace(l)
|
||||
if !strings.HasPrefix(trimmedLine, "- ") || strings.TrimSpace(trimmedLine[2:]) == "" {
|
||||
return nil, ErrMalformedPlan
|
||||
}
|
||||
}
|
||||
|
||||
return []byte(rawOutput), nil
|
||||
}
|
||||
|
||||
func RenderReview(tmpl string, fields ReviewFields, maxOutputBytes int) ([]byte, error) {
|
||||
if maxOutputBytes < 1 {
|
||||
return nil, ErrMalformedReview
|
||||
}
|
||||
if err := ValidateReviewTemplate(tmpl); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
c := strings.TrimSpace(fields.Checks)
|
||||
v := strings.TrimSpace(fields.Verification)
|
||||
s := strings.TrimSpace(fields.Summary)
|
||||
if c == "" || v == "" || s == "" {
|
||||
return nil, ErrMalformedReview
|
||||
}
|
||||
|
||||
res := strings.ReplaceAll(tmpl, "{{checks}}", c)
|
||||
res = strings.ReplaceAll(res, "{{verification}}", v)
|
||||
res = strings.ReplaceAll(res, "{{summary}}", s)
|
||||
|
||||
if strings.Contains(res, "{{") || strings.Contains(res, "}}") {
|
||||
return nil, ErrMalformedReview
|
||||
}
|
||||
|
||||
if len(res) > maxOutputBytes {
|
||||
return nil, ErrMalformedReview
|
||||
}
|
||||
|
||||
return []byte(res), nil
|
||||
}
|
||||
467
packages/go/singlerequesttemplate/template_test.go
Normal file
467
packages/go/singlerequesttemplate/template_test.go
Normal file
|
|
@ -0,0 +1,467 @@
|
|||
package singlerequesttemplate_test
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"iop/packages/go/singlerequesttemplate"
|
||||
)
|
||||
|
||||
func TestValidatePlanTemplate(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
tmpl string
|
||||
wantErr bool
|
||||
}{
|
||||
{
|
||||
name: "default plan template is valid",
|
||||
tmpl: singlerequesttemplate.DefaultPlanTemplate,
|
||||
wantErr: false,
|
||||
},
|
||||
{
|
||||
name: "custom plan template with extra static text",
|
||||
tmpl: `# Plan
|
||||
|
||||
Custom header notes.
|
||||
|
||||
## Goal
|
||||
{{goal}}
|
||||
|
||||
## Steps
|
||||
{{steps}}
|
||||
|
||||
## Verification
|
||||
{{verification}}
|
||||
|
||||
Footer notes.
|
||||
`,
|
||||
wantErr: false,
|
||||
},
|
||||
{
|
||||
name: "empty template",
|
||||
tmpl: "",
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "oversized template 8193 bytes",
|
||||
tmpl: singlerequesttemplate.DefaultPlanTemplate + strings.Repeat(" ", 8193-len(singlerequesttemplate.DefaultPlanTemplate)),
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "missing {{goal}}",
|
||||
tmpl: `# Plan
|
||||
|
||||
## Goal
|
||||
|
||||
## Steps
|
||||
{{steps}}
|
||||
|
||||
## Verification
|
||||
{{verification}}
|
||||
`,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "duplicate {{goal}}",
|
||||
tmpl: `# Plan
|
||||
|
||||
## Goal
|
||||
{{goal}} {{goal}}
|
||||
|
||||
## Steps
|
||||
{{steps}}
|
||||
|
||||
## Verification
|
||||
{{verification}}
|
||||
`,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "unknown token",
|
||||
tmpl: `# Plan
|
||||
|
||||
## Goal
|
||||
{{goal}} {{foo}}
|
||||
|
||||
## Steps
|
||||
{{steps}}
|
||||
|
||||
## Verification
|
||||
{{verification}}
|
||||
`,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "wrong token order",
|
||||
tmpl: `# Plan
|
||||
|
||||
## Goal
|
||||
{{steps}}
|
||||
|
||||
## Steps
|
||||
{{goal}}
|
||||
|
||||
## Verification
|
||||
{{verification}}
|
||||
`,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "missing required heading # Plan",
|
||||
tmpl: `## Goal
|
||||
{{goal}}
|
||||
|
||||
## Steps
|
||||
{{steps}}
|
||||
|
||||
## Verification
|
||||
{{verification}}
|
||||
`,
|
||||
wantErr: true,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
err := singlerequesttemplate.ValidatePlanTemplate(tt.tmpl)
|
||||
if (err != nil) != tt.wantErr {
|
||||
t.Errorf("ValidatePlanTemplate() err = %v, wantErr %v", err, tt.wantErr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestParsePlan(t *testing.T) {
|
||||
validOutput := `# Plan
|
||||
|
||||
## Goal
|
||||
Fix single-request template handling bug.
|
||||
|
||||
## Steps
|
||||
- Inspect template file resolution.
|
||||
- Verify template validation logic.
|
||||
|
||||
## Verification
|
||||
- Run go test on singlerequesttemplate package.
|
||||
`
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
tmpl string
|
||||
raw string
|
||||
maxOutputBytes int
|
||||
wantErr bool
|
||||
}{
|
||||
{
|
||||
name: "valid plan output default template",
|
||||
tmpl: singlerequesttemplate.DefaultPlanTemplate,
|
||||
raw: validOutput,
|
||||
maxOutputBytes: 1024,
|
||||
wantErr: false,
|
||||
},
|
||||
{
|
||||
name: "boundary steps = 2, verif = 1",
|
||||
tmpl: singlerequesttemplate.DefaultPlanTemplate,
|
||||
raw: validOutput,
|
||||
maxOutputBytes: 1024,
|
||||
wantErr: false,
|
||||
},
|
||||
{
|
||||
name: "boundary steps = 6, verif = 3",
|
||||
tmpl: singlerequesttemplate.DefaultPlanTemplate,
|
||||
raw: `# Plan
|
||||
|
||||
## Goal
|
||||
Implement feature end to end.
|
||||
|
||||
## Steps
|
||||
- Step one
|
||||
- Step two
|
||||
- Step three
|
||||
- Step four
|
||||
- Step five
|
||||
- Step six
|
||||
|
||||
## Verification
|
||||
- Verify 1
|
||||
- Verify 2
|
||||
- Verify 3
|
||||
`,
|
||||
maxOutputBytes: 2048,
|
||||
wantErr: false,
|
||||
},
|
||||
{
|
||||
name: "invalid step count = 1 (too few)",
|
||||
tmpl: singlerequesttemplate.DefaultPlanTemplate,
|
||||
raw: `# Plan
|
||||
|
||||
## Goal
|
||||
Implement feature.
|
||||
|
||||
## Steps
|
||||
- Step one
|
||||
|
||||
## Verification
|
||||
- Verify 1
|
||||
`,
|
||||
maxOutputBytes: 1024,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "invalid step count = 7 (too many)",
|
||||
tmpl: singlerequesttemplate.DefaultPlanTemplate,
|
||||
raw: `# Plan
|
||||
|
||||
## Goal
|
||||
Implement feature.
|
||||
|
||||
## Steps
|
||||
- Step 1
|
||||
- Step 2
|
||||
- Step 3
|
||||
- Step 4
|
||||
- Step 5
|
||||
- Step 6
|
||||
- Step 7
|
||||
|
||||
## Verification
|
||||
- Verify 1
|
||||
`,
|
||||
maxOutputBytes: 1024,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "invalid verif count = 0 (too few)",
|
||||
tmpl: singlerequesttemplate.DefaultPlanTemplate,
|
||||
raw: `# Plan
|
||||
|
||||
## Goal
|
||||
Implement feature.
|
||||
|
||||
## Steps
|
||||
- Step 1
|
||||
- Step 2
|
||||
|
||||
## Verification
|
||||
|
||||
`,
|
||||
maxOutputBytes: 1024,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "invalid verif count = 4 (too many)",
|
||||
tmpl: singlerequesttemplate.DefaultPlanTemplate,
|
||||
raw: `# Plan
|
||||
|
||||
## Goal
|
||||
Implement feature.
|
||||
|
||||
## Steps
|
||||
- Step 1
|
||||
- Step 2
|
||||
|
||||
## Verification
|
||||
- Verify 1
|
||||
- Verify 2
|
||||
- Verify 3
|
||||
- Verify 4
|
||||
`,
|
||||
maxOutputBytes: 1024,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "multiline goal",
|
||||
tmpl: singlerequesttemplate.DefaultPlanTemplate,
|
||||
raw: `# Plan
|
||||
|
||||
## Goal
|
||||
First line of goal.
|
||||
Second line of goal.
|
||||
|
||||
## Steps
|
||||
- Step 1
|
||||
- Step 2
|
||||
|
||||
## Verification
|
||||
- Verify 1
|
||||
`,
|
||||
maxOutputBytes: 1024,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "altered heading static text",
|
||||
tmpl: singlerequesttemplate.DefaultPlanTemplate,
|
||||
raw: `# Plan Mismatch
|
||||
|
||||
## Goal
|
||||
Fix bug.
|
||||
|
||||
## Steps
|
||||
- Step 1
|
||||
- Step 2
|
||||
|
||||
## Verification
|
||||
- Verify 1
|
||||
`,
|
||||
maxOutputBytes: 1024,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "unresolved token in output",
|
||||
tmpl: singlerequesttemplate.DefaultPlanTemplate,
|
||||
raw: `# Plan
|
||||
|
||||
## Goal
|
||||
Fix {{goal}} bug.
|
||||
|
||||
## Steps
|
||||
- Step 1
|
||||
- Step 2
|
||||
|
||||
## Verification
|
||||
- Verify 1
|
||||
`,
|
||||
maxOutputBytes: 1024,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "output exceeds maxOutputBytes",
|
||||
tmpl: singlerequesttemplate.DefaultPlanTemplate,
|
||||
raw: validOutput,
|
||||
maxOutputBytes: 10,
|
||||
wantErr: true,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got, err := singlerequesttemplate.ParsePlan(tt.tmpl, tt.raw, tt.maxOutputBytes)
|
||||
if (err != nil) != tt.wantErr {
|
||||
t.Errorf("ParsePlan() err = %v, wantErr %v", err, tt.wantErr)
|
||||
}
|
||||
if !tt.wantErr && string(got) != tt.raw {
|
||||
t.Errorf("ParsePlan() got = %q, want %q", string(got), tt.raw)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateReviewTemplate(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
tmpl string
|
||||
wantErr bool
|
||||
}{
|
||||
{
|
||||
name: "default review template is valid",
|
||||
tmpl: singlerequesttemplate.DefaultReviewTemplate,
|
||||
wantErr: false,
|
||||
},
|
||||
{
|
||||
name: "empty template",
|
||||
tmpl: "",
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "missing PASS",
|
||||
tmpl: `# Review
|
||||
|
||||
## Result
|
||||
FAIL
|
||||
|
||||
## Checks
|
||||
{{checks}}
|
||||
|
||||
## Verification
|
||||
{{verification}}
|
||||
|
||||
## Summary
|
||||
{{summary}}
|
||||
`,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "wrong order",
|
||||
tmpl: `# Review
|
||||
|
||||
## Result
|
||||
PASS
|
||||
|
||||
## Verification
|
||||
{{verification}}
|
||||
|
||||
## Checks
|
||||
{{checks}}
|
||||
|
||||
## Summary
|
||||
{{summary}}
|
||||
`,
|
||||
wantErr: true,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
err := singlerequesttemplate.ValidateReviewTemplate(tt.tmpl)
|
||||
if (err != nil) != tt.wantErr {
|
||||
t.Errorf("ValidateReviewTemplate() err = %v, wantErr %v", err, tt.wantErr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRenderReview(t *testing.T) {
|
||||
fields := singlerequesttemplate.ReviewFields{
|
||||
Checks: "- Checked file permissions\n- Verified build pass",
|
||||
Verification: "- Executed unit test suite",
|
||||
Summary: "All requirements met successfully.",
|
||||
}
|
||||
|
||||
got, err := singlerequesttemplate.RenderReview(singlerequesttemplate.DefaultReviewTemplate, fields, 1024)
|
||||
if err != nil {
|
||||
t.Fatalf("RenderReview() unexpected err = %v", err)
|
||||
}
|
||||
|
||||
want := `# Review
|
||||
|
||||
## Result
|
||||
PASS
|
||||
|
||||
## Checks
|
||||
- Checked file permissions
|
||||
- Verified build pass
|
||||
|
||||
## Verification
|
||||
- Executed unit test suite
|
||||
|
||||
## Summary
|
||||
All requirements met successfully.
|
||||
`
|
||||
if string(got) != want {
|
||||
t.Errorf("RenderReview() got:\n%s\nwant:\n%s", string(got), want)
|
||||
}
|
||||
|
||||
// Missing field test
|
||||
badFields := fields
|
||||
badFields.Summary = ""
|
||||
_, err = singlerequesttemplate.RenderReview(singlerequesttemplate.DefaultReviewTemplate, badFields, 1024)
|
||||
if err == nil {
|
||||
t.Errorf("RenderReview() expected error for empty Summary, got nil")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDigest(t *testing.T) {
|
||||
d1 := singlerequesttemplate.Digest(singlerequesttemplate.DefaultPlanTemplate)
|
||||
d2 := singlerequesttemplate.Digest(singlerequesttemplate.DefaultPlanTemplate)
|
||||
d3 := singlerequesttemplate.Digest("other content")
|
||||
|
||||
if d1 == "" {
|
||||
t.Errorf("Digest() returned empty string")
|
||||
}
|
||||
if d1 != d2 {
|
||||
t.Errorf("Digest() not deterministic: %q != %q", d1, d2)
|
||||
}
|
||||
if d1 == d3 {
|
||||
t.Errorf("Digest() collision: %q == %q", d1, d3)
|
||||
}
|
||||
}
|
||||
Loading…
Reference in a new issue