fix(edge): single request provider 정규화를 적용한다

This commit is contained in:
toki 2026-08-14 00:23:45 +09:00
parent 862fe702a3
commit 73d9b51432
12 changed files with 455 additions and 26 deletions

View file

@ -17,8 +17,11 @@
- `apps/edge/internal/service/single_request_tool_loop.go`
- `apps/edge/internal/service/single_request_types.go`
- `apps/edge/internal/openai/single_request_preset_binding.go`
- `apps/edge/internal/openai/single_request_provider_stage.go`
- `apps/edge/internal/openai/single_request_plan_stage.go`
- `apps/edge/internal/openai/single_request_work_stage.go`
- `apps/edge/internal/openai/single_request_review_stage.go`
- `apps/edge/internal/openai/provider_normalization.go`
- `packages/go/singlerequesttemplate/template.go`
- `apps/edge/internal/openai/anthropic_types.go`
- `apps/edge/internal/openai/routes.go`
@ -97,6 +100,17 @@ binding (route ID, credential slot, provider ID, endpoint, or raw workspace data
echoed to the caller. The admission is owned by the service package; the OpenAI and
Anthropic surfaces read only the public model identity and the frozen limits.
Each private Plan/Work/Review call derives caller-neutral requirements from the
frozen stage options, tool presence, and structured-output contract. The selected
protocol profile then chooses Chat Completions or Responses and maps effort to the
exact or nearest supported lower grade; the caller product or SDK name never enters
that choice. A managed route with an explicit resource selector still freezes and
verifies the exact provider ID. A `default` resource selector intentionally leaves
the provider ID open for the provider pool while continuing to freeze and verify the
model group, profile, upstream model, credential slot/revision, and tunnel execution
path. The concrete provider selected by that pool is therefore valid dispatch
evidence, not a stage-binding mismatch.
The admission also freezes the effective Plan and Review Markdown templates resolved
from the operator's `execution_presets[].single_request.templates` configuration (see
`agent-contract/inner/edge-config-runtime-refresh.md`). These templates are

View file

@ -74,6 +74,6 @@ IOP 전체 안정성을 처음부터 재검증하지 않고, 실패가 재현된
- TLS 환경 경계: 개발 Edge용 사설 CA는 Edge leaf 인증서가 아니라 managed CA bundle을 해당 벤치 caller process에만 전달한다. Codex에는 공식 변수 `CODEX_CA_CERTIFICATE`, Node 기반 caller에는 필요한 경우 `NODE_EXTRA_CA_CERTS`를 사용하며 Codex/IDE 시작 환경이나 셸 전역에 `export`하지 않는다. 그렇지 않으면 공개 TLS 연결에도 같은 CA override가 적용될 수 있다.
- evidence 위치: `agent-test/dev/iop-benchmark-route-minimal-html-smoke.md`
- 현재 사전 확인: 2026-08-13 실제 원격 실행기에서 Claude Code 2.1.177, agy 1.1.12, Codex 0.146.0을 확인했고, 원격 SOPS에 보관된 기존 IOP principal token으로 token 원문을 출력하지 않은 `/v1/models`가 HTTP 200임을 확인했다. 새 벤치 전용 token은 발급하거나 사용하지 않는다.
- 현재 경로 결과: Claude Code → Claude direct와 Claude Code → Gemini direct는 최소 HTML 1회 호출을 통과했다. Claude Code → GPT direct는 Chat Completions의 tools+reasoning 조합 미지원으로 실패했고, IOP provider operation normalization 결함으로 귀속했다. caller-neutral operation 선택, Messages↔Responses 변환, nearest-lower effort mapping의 focused regression은 통과했으며 개발 런타임 재검증이 남아 있다.
- 추가 분리 결과: agy → Gemini direct는 누락된 caller `modelProvider=gemini`를 보정한 뒤 공식 URL-encoded model label을 IOP Gemini path parser가 거부하는 제품 결함까지 좁혔다. Codex → GPT direct의 최초 실패는 CA bundle 대신 Edge leaf 인증서를 전달한 측정 환경 결함이었고, 공식 Responses custom provider와 command-scoped managed CA bundle으로 바꾼 재검증은 10초 안에 `turn.completed``index.html` 생성을 통과했다. 같은 경계의 Codex → GPT execution preset도 16초 안에 `turn.completed`와 terminal marker 1회를 통과했다. Claude Code → Gemini preset의 caller workspace 파일 부재는 Edge-owned 비공개 workspace 정리 계약상 정상이라 측정 판정을 바로잡았고, 184초로 120초 상한을 넘은 지연과 최초 terminal marker 미수집만 별도 실패로 남겼다.
- 현재 경로 결과: Claude Code → Claude direct와 Claude Code → Gemini direct는 최소 HTML 1회 호출을 통과했다. Claude Code → GPT direct는 Chat Completions의 tools+reasoning 조합 미지원으로 실패했으나 caller-neutral operation normalization 반영 뒤 9초 안에 caller success와 `index.html` marker 1회를 확인했다. Codex → GPT direct와 Codex → GPT execution preset도 각각 10초/16초에 통과했다.
- 추가 분리 결과: agy → Gemini direct는 누락된 caller `modelProvider=gemini`를 보정한 뒤 공식 URL-encoded model label을 IOP Gemini path parser가 거부하는 제품 결함까지 좁혔다. Codex → GPT direct의 최초 실패는 CA bundle 대신 Edge leaf 인증서를 전달한 측정 환경 결함이었다. Claude Code → Gemini/GPT preset은 모두 provider 호출 뒤 Plan `validation`으로 닫혔고, 원인은 `default` selector가 비워 둔 provider ID와 pool이 선택한 실제 provider ID를 사후에 무조건 비교한 IOP 결함으로 확인했다. explicit selector와 profile/model/credential/path fence는 유지하면서 default selector의 pool 선택만 인정하도록 수정했고, private Plan/Work/Review에도 profile별 Chat/Responses 선택과 nearest-lower effort mapping을 적용한 focused regression 및 Edge 전체 테스트가 통과했다. 개발 런타임에는 아직 배포되지 않았다.
- 후속 측정: [초경량 Agent 모델 비교](thin-agent-model-comparison-benchmark.md)

View file

@ -83,7 +83,13 @@ source_evidence:
notes: Anthropic Messages ↔ Chat Completions request/response bridge
- type: code
path: apps/edge/internal/openai/provider_normalization.go
notes: Caller-neutral provider operation selection, effort fallback, Messages ↔ Responses conversion
notes: Caller-neutral provider operation selection, effort fallback, Messages/Chat ↔ Responses conversion for public bridges and private single-request stages
- type: code
path: apps/edge/internal/openai/single_request_provider_stage.go
notes: Frozen stage dispatch verification with explicit-provider fencing and default-selector provider-pool ownership
- type: test
path: apps/edge/internal/openai/single_request_provider_stage_test.go
notes: Private stage Chat/Responses selection, max-to-xhigh fallback, response conversion, and default-selector dispatch regression
- type: code
path: apps/edge/internal/openai/anthropic_stream.go
notes: Chat/Responses provider output을 Anthropic Messages JSON/SSE로 변환
@ -184,6 +190,7 @@ Edge가 OpenAI-compatible HTTP 요청을 받아 내부 `adapter + target` 실행
| managed projection auth | `credential_plane.enabled=true` uses the fresh Control Plane projection for inbound token auth and principal route discovery. Static principal/bearer fallback is disabled. |
| managed slot route | Public model id/alias resolves to one projected route, exact slot/profile/upstream model/resource selector, and immutable revisions/generation. Unknown, cross-principal, stale, revoked, or ambiguous bindings fail closed. |
| marked preset single-request admission | An authorized fixed single-request preset compiles one service-owned admission value at request start: requested public model, canonical plan/work/review bindings resolved through managed authorization, opaque workspace capability, and absolute resource caps. Later refresh cannot mutate the admitted shape. No private binding is echoed to the caller. Compiled only after every canonical reference is verified through its catalog binding for the authenticated principal; missing, duplicate, unauthorized, dynamically selected, or option-inconsistent inputs are rejected without fallback. |
| marked single-request provider normalization | Plan/Work/Review derive caller-neutral effort/tool/structured-output requirements and let the selected protocol profile choose Chat Completions or Responses. Effort exact misses fall only to the nearest declared lower grade (`max` → `xhigh` when `max` is absent). Explicit resource selectors keep exact provider-ID verification; a `default` selector leaves provider choice to the pool while model group, profile, upstream model, credential slot/revision, and tunnel path remain frozen. Responses results are converted back to the private common Chat-shaped stage codec before Plan/Work/Review validation. |
| marked single-request internal templates | The admission also freezes the operator-configured effective Plan/Review Markdown templates. They are internal artifact shapes only: the Plan stage obtains a strict one-line `goal` plus bounded one-line `steps`/`verification` arrays, and Edge owns the bullet formatting and renders `plan.md`, while the Review template shapes the private `review.md` artifact. Callers cannot supply, name, or select a template, and template paths, contents, and digests never appear in a response, error, log, or metric label. The caller-visible Messages request/response schema is unchanged and the final text stays the model's `decision.output`. |
| marked single-request ingress | One validated and authorized Messages POST enters the separate service coordinator capability before legacy provider/caller continuation and increments `iop_anthropic_single_request_ingress_total` once. Non-streaming returns one buffered final-only message. Streaming keeps one envelope across the coordinator lifetime, exposes only fixed plan/work/review/repair text blocks plus `event: ping`, and commits one final text/error terminal. Internal reasoning/tool wire never becomes caller `tool_use`; success is acknowledged only after the complete terminal write succeeds. |
| marked single-request S11 terminal policy | The service freezes one closed `end_turn`, `length`, `error`, or `cancelled` disposition. `error` classes are provider, validation, timeout, budget, repetition, malformed, context, internal-tool, and workspace-cleanup. Buffered and SSE share one projection: `end_turn`; `max_tokens` with no private partial output; `400 invalid_request_error` for validation/context; `502 api_error` for other failures; and silent cancellation after caller disconnect. No terminal classification retries, falls back, opens a second request, or later writes success. |
@ -373,6 +380,7 @@ sequenceDiagram
- 2026-08-05: Added Claude Code adaptive-effort/structured-output/cache-control bridge compatibility, stateless Gemini thought-signature tool round trips, and generic Chat replay handling for unsigned private thinking blocks.
- 2026-08-09: Extended `output_config.effort` to accept `low`, `medium`, `high`, `xhigh`, and `max` across Anthropic native and Chat bridge routes without substitution or normalization. Unknown effort values remain `400 invalid_request_error` before provider dispatch. Deterministic Go coverage added for exact bridge mapping, native `max` preservation, and invalid-value rejection. (`apps/edge/internal/openai/anthropic_types.go`, `apps/edge/internal/openai/anthropic_bridge_test.go`, `apps/edge/internal/openai/anthropic_native_test.go`)
- 2026-08-13: Added caller-neutral provider operation normalization for Messages/Responses routes. Tool-bearing adaptive effort can select Responses when Chat cannot preserve the combination, and unsupported effort grades fall only to the nearest declared lower grade (for example `max` to `xhigh`).
- 2026-08-14: Applied the same provider-normalization boundary to private single-request Plan/Work/Review stages. Removed pre-selection Chat operation pinning, added Chat↔Responses stage conversion and nearest-lower effort mapping, and corrected default-selector dispatch validation so the pool-selected provider is accepted without weakening explicit provider, profile, credential, target, or execution-path fences.
- 2026-08-13: Gemini-native path parsing now keeps route IDs strict while accepting the bounded URL-encoded official caller model label `Gemini 3.6 Flash`.
- 2026-08-06: Synchronized always-owned Chat/Responses typed-stall recovery, provider avoidance/fallback admission, and closed-label liveness operational evidence with the current runtime, contracts, and deterministic recovery tests.
- 2026-08-06: Added marked single-request Messages admission through the separate service coordinator capability, one unlabeled runtime ingress counter, buffered sanitized terminal acknowledgement, and deterministic real-POST compatibility evidence.

View file

@ -229,6 +229,7 @@ The shared `packages/go/execution` package contains provider lifecycle, registry
| internal workspace tool loop | The service decodes only `workspace_read`, `workspace_list`, `workspace_write`, `workspace_delete`, and `workspace_command`, opens the admitted workspace once, dispatches one call at a time on the frozen generation, and delivers one deep-copied typed result to the emitting executor continuation. Unique request/stage/tool correlation, per-stage iteration/output/deadline limits, request wall-clock budget, and typed cancel fail closed without external continuation or reselection. |
| request-owned internal artifacts | `SingleRequestController` exposes closed plan/review read/write operations. Artifact calls and model workspace tools share one serialized lazy `WorkspaceOpen`, the exact admitted Node generation, the active stage deadline, the immutable output bound, in-flight work accounting, and one terminal cleanup. Node alone maps selectors to `plan.md` and `review.md`, and inventoried descriptor-relative reads fail closed on identity replacement. |
| Plan stage | The Plan runner validates the frozen effective template, emits the `planning` envelope, sends the immutable task through the frozen Plan binding with `reasoning_effort=high` and a stage-owned strict JSON schema for one-line `goal` plus bounded one-line `steps`/`verification` arrays, validates the fields, adds Markdown bullets, renders the template deterministically inside Edge, and writes the resulting Markdown through `SingleRequestArtifactPlan`. |
| single-request provider normalization | Private Plan/Work/Review calls pass caller-neutral effort/tool/structured-output requirements to the selected protocol profile. The profile chooses Chat Completions or Responses and maps unsupported effort only downward. An explicit managed selector freezes the exact provider ID; `default` freezes no provider ID and accepts the provider pool's concrete choice while retaining exact model-group/profile/target/credential/tunnel fences. No new Edge-Node field is added: the selected operation continues through the existing provider tunnel operation field. |
| single-request effective templates | `execution_presets[].single_request.templates` optionally loads `plan_file`/`review_file` as bounded Markdown relative to the directory containing `edge.yaml`; absolute and empty paths, non-regular files, oversize (`>8192` bytes), non-UTF-8, and invalid grammar fail closed at load, and each file falls back to its built-in default independently. Admission freezes the effective Plan/Review pair into the binding, so a later refresh reaches only newly admitted requests. Templates select internal stage input and internal artifact shape only; caller request/response schemas are unchanged. |
| Work stage | The `ornith-fast` Work runner reads the closed PLAN artifact, projects only the admitted workspace tools, and resumes the same frozen provider route after exactly correlated Node results. It rejects any Work `reasoning_effort`, malformed or multiple tool calls, and empty completion or verification evidence. |
| request-owned cleanup | Node creates and inventories only `.iop/job/<request_id>` internal state, cancels and waits for all active command groups, validates the exact tree without following entries, and removes matching artifacts deepest-first with non-recursive descriptor operations. Symlinks, special files, foreign devices, identity replacements, and unowned entries fail closed. User results and sibling request state are preserved. Concurrent cleanup callers receive one bounded cached typed result. |
@ -345,6 +346,7 @@ Heartbeat interval/wait는 protobuf field가 아닌 양쪽 transport 구현의 l
- `go test -count=1 ./apps/edge/internal/openai -run '^(TestOpenAILivenessObservationSink|TestOpenAILivenessRecoveryObservability)$'` — deterministic OpenAI recovery eligibility/results with closed label values and identifier exclusion.
- `go test -count=1 ./apps/edge/internal/openai -run 'TestAnthropicSingleRequestObservation'` — deterministic single-request observation evidence: ingress=1, request-total=1, terminal=1, stage/tool/cleanup counts, raw-free correlation, and unlabeled metric assertion.
- `go test -count=1 ./apps/edge/internal/openai -run 'TestSingleRequest(ProviderStage|PlanStage)'` — deterministic frozen provider codec and Plan stage evidence, including high reasoning, the closed structured response schema, exact JSON field admission, ordered tunnel frames, deterministic frozen-template rendering, planning envelope, and `plan.md` artifact selection.
- `go test -count=1 ./apps/edge/internal/openai -run 'TestSingleRequest(ProviderStage|ProviderNormalization|PlanStage|WorkStage|ReviewStage|Executor)'` — private stage operation selection, Chat/Responses conversion, nearest-lower effort (`max` → `xhigh`), default-selector pool-provider acceptance, explicit dispatch fences, and composite regression evidence.
- `go test -count=1 ./packages/go/singlerequesttemplate ./packages/go/config` — exact standalone heading/`PASS` grammar, closed placeholder inventory, the `8192`-byte boundary, and relative-only `plan_file`/`review_file` loading with per-file fallback and filesystem-kind rejection.
- `go test -count=1 ./apps/edge/internal/service -run 'TestSingleRequestBinding'` and `go test -count=1 ./apps/edge/internal/openai -run 'TestSingleRequest(PresetBinding|ReviewStage)'` — admitted effective-template snapshot through clone and workspace revalidation, refresh isolation for admitted versus newly admitted requests, and the custom-template REVIEW artifact with unchanged caller-visible output.
- `go test -race -count=1 ./apps/edge/internal/openai -run 'TestSingleRequestWork(Stage|ToolBridge)'` — deterministic ornith-fast Work tool loop, correlation isolation, cancellation cleanup, strict completion evidence, and Work reasoning-option absence.
@ -365,6 +367,7 @@ Heartbeat interval/wait는 protobuf field가 아닌 양쪽 transport 구현의 l
## 변경 기록
- 2026-08-14: Moved private Plan/Work/Review provider calls onto the shared provider-normalization boundary. Stage requirements now select Chat or Responses without caller identity, unsupported effort maps only downward, and default-selector provider-pool choices no longer fail the post-dispatch validation that still fences profile, target, credential revision, model group, and tunnel path.
- 2026-08-12: Replaced nondeterministic free-form PlanMD generation with a stage-owned strict `goal`/`steps`/`verification` JSON response. Edge rejects unknown, duplicate, missing, mistyped, or out-of-bound fields and deterministically renders the already-frozen operator Plan template, preserving template customization and every caller-visible schema (`apps/edge/internal/openai/single_request_plan_stage.go`, `packages/go/singlerequesttemplate/template.go`).
- 2026-08-12: Replaced Markdown-bearing Plan string fields with bounded one-line string arrays and moved all bullet/newline rendering into Edge, removing the remaining provider-specific formatting dependency without changing the public Messages surface (`apps/edge/internal/openai/single_request_plan_stage.go`, `packages/go/singlerequesttemplate/template.go`).
- 2026-08-09: Replaced the Plan stage's structured JSON response contract with operator-configurable bounded Markdown templates. `execution_presets[].single_request.templates` resolves `plan_file`/`review_file` relative to the directory containing `edge.yaml` only, admission freezes the effective Plan/Review pair so a refresh reaches only newly admitted requests, the Plan stage prompts with and parses direct PlanMD against the frozen template, and the Review stage renders the internal REVIEW artifact from the frozen Review template while the caller-visible final response stays the model's `decision.output` (`packages/go/singlerequesttemplate/template.go`, `packages/go/config/load.go`, `apps/edge/internal/openai/single_request_preset_binding.go`, `apps/edge/internal/openai/single_request_plan_stage.go`, `apps/edge/internal/openai/single_request_review_stage.go`).

View file

@ -27,17 +27,19 @@
| Claude Code → Claude direct | 통과 | 없음 | 기존 성공 |
| Claude Code → Gemini direct | 통과 | 없음 | 기존 성공 |
| agy → Gemini direct | 1초, caller login 요구 | `modelProvider=gemini` 보정 뒤 8초, HTTP 400 | IOP Gemini path가 공식 URL-encoded model label을 거부 |
| Claude Code → GPT direct | provider HTTP 400 | 새 normalization 배포 대기 | IOP가 tools+effort를 Chat으로 보낸 결함 |
| Claude Code → GPT direct | provider HTTP 400 | operation normalization 반영 뒤 9초, caller success, `index.html` marker 1회 | IOP가 tools+effort를 Chat으로 보낸 결함 수정 확인 |
| Codex → GPT direct | 30초, `turn.failed`, 파일 없음 | 공식 설정대로 임시 `CODEX_HOME`, Responses 전용 provider, `CODEX_CA_CERTIFICATE`에 CA bundle을 사용해 10초 통과 | 측정 환경 결함: 첫 호출은 CA bundle 대신 Edge leaf 인증서를 사용 |
| Claude Code → Gemini execution preset | 184초, caller terminal success, caller workspace 파일 없음 | 판정 정정: caller 파일 부재는 정상, 120초 초과는 실패 | 측정 판정 결함과 preset 지연을 분리; terminal marker는 최초 결과에서 미수집 |
| Claude Code → Gemini execution preset | 184초, caller terminal success, caller workspace 파일 없음 | 파일 판정은 정상으로 정정; metric에서 Plan `validation` 종료 확인 | 측정 판정 결함과 제품 dispatch 검증 결함을 분리; default selector의 실제 provider ID를 오판 |
| agy → Gemini execution preset | 미실행 | direct parser 수정 배포 대기 | 선행 결함 |
| Claude Code → GPT execution preset | 미실행 | normalization 배포 대기 | 선행 결함 |
| Claude Code → GPT execution preset | 미실행 | 10초, request terminal `validation`, marker 0회 | provider 호출 뒤 default selector의 실제 provider ID를 고정값 불일치로 오판; 국소 수정·회귀 통과, 배포 대기 |
| Codex → GPT execution preset | 16초, `turn.completed`, terminal marker 1회 | 없음 | 통과 |
추가 API 분리에서는 동일 principal의 최소 `/v1/responses`가 HTTP 200이었다. Codex direct도 사용자 설정과 로그인 상태를 배제한 임시 `CODEX_HOME`, Responses 전용 custom provider, 원격 SOPS의 기존 token, command-scoped managed CA bundle으로 통과했다. Gemini-native 최소 요청은 canonical caller model id에서 HTTP 200, 공식 표시 label `Gemini 3.6 Flash`에서 HTTP 400으로 갈려 path parser 결함을 재현했다.
두 preset 실패는 provider 자체 실패가 아니었다. GPT/Gemini 모두 구조화 Plan provider 호출을 마친 뒤 같은 `validation` terminal로 닫혔고, 코드 대조에서 `default` resource selector가 의도적으로 빈 provider ID를 동결하는 반면 사후 검증은 pool이 정상 선택한 실제 provider ID와 무조건 같아야 한다고 요구한 결함을 확인했다. explicit selector의 provider ID와 profile/model/credential/path fence는 유지하고, default selector만 pool 선택을 인정하는 회귀 테스트가 통과했다. 같은 private stage 경계에서 operation을 선고정하지 않고 profile별 Chat/Responses 선택과 nearest-lower effort mapping을 적용했다.
## 재개 조건
원격에 남은 미완료 release head를 운영 절차로 먼저 정리한 뒤 병합된 `dev`를 새 release로 배포한다. 그 뒤 변경된 원인에 연결된 agy direct와 Claude Code GPT direct만 1회 재검증하고, 선행 결함이 해소된 미실행 행을 각 1회 수행한다.
원격에 남은 미완료 release head를 운영 절차로 먼저 정리한 뒤 병합된 `dev`를 새 release로 배포한다. 그 뒤 parser 수정에 연결된 agy direct와 dispatch 검증 수정에 연결된 두 Claude Code preset만 1회 재검증하고, 선행 결함이 해소된 agy preset을 1회 수행한다. Claude Code GPT direct는 이미 수정 배포 뒤 통과했으므로 반복하지 않는다.
성공한 경로는 반복하지 않는다. 실패한 경로는 원인이 변경된 경우에만 해당 경로를 1회 재검증한다.

View file

@ -2,6 +2,7 @@ package openai
import (
"encoding/json"
"errors"
"fmt"
"strings"
@ -56,6 +57,226 @@ type openAIResponsesBridgeResponse struct {
} `json:"incomplete_details"`
}
func singleRequestProviderRequirements(options map[string]any, tools []any, responseFormat *singleRequestProviderResponseFormat) providerRequestRequirements {
requirements := providerRequestRequirements{HasTools: len(tools) > 0, StructuredOutput: responseFormat != nil}
if effort, ok := options["reasoning_effort"].(string); ok {
requirements.Effort = strings.TrimSpace(effort)
}
return requirements
}
func singleRequestProviderCandidatePredicate(frozen *edgeservice.SingleRequestStageDispatchBinding, requirements providerRequestRequirements) edgeservice.ProviderPoolCandidatePredicate {
return func(candidate edgeservice.ProviderPoolCandidate) bool {
if frozen == nil || candidate.ProtocolProfile == nil || candidate.ExecutionPath != string(edgeservice.ProviderPoolPathTunnel) {
return false
}
if frozen.CandidatePredicate != nil && !frozen.CandidatePredicate(candidate) {
return false
}
_, err := selectProviderOperation(*candidate.ProtocolProfile, config.OperationChatCompletions, requirements)
return err == nil
}
}
func singleRequestProviderTunnelPreparer(requirements providerRequestRequirements, buildChat singleRequestProviderBodyBuilder) func(edgeservice.SubmitProviderTunnelRequest, edgeservice.ProviderPoolCandidate) (edgeservice.SubmitProviderTunnelRequest, error) {
return func(tunnel edgeservice.SubmitProviderTunnelRequest, candidate edgeservice.ProviderPoolCandidate) (edgeservice.SubmitProviderTunnelRequest, error) {
if candidate.ProtocolProfile == nil || buildChat == nil {
return tunnel, errProviderStageMissingBinding
}
plan, err := selectProviderOperation(*candidate.ProtocolProfile, config.OperationChatCompletions, requirements)
if err != nil {
return tunnel, err
}
tunnel.Operation = string(plan.Operation)
switch plan.Operation {
case config.OperationChatCompletions:
tunnel.Path = "/v1/chat/completions"
case config.OperationResponses:
tunnel.Path = "/v1/responses"
default:
return tunnel, errProviderStageMissingBinding
}
tunnel.BuildBody = func(target string) ([]byte, error) {
body, err := buildChat(target)
if err != nil {
return nil, err
}
return normalizeSingleRequestProviderRequest(body, plan)
}
return tunnel, nil
}
}
func normalizeSingleRequestProviderRequest(chatBody []byte, plan providerOperationPlan) ([]byte, error) {
var body map[string]any
if err := json.Unmarshal(chatBody, &body); err != nil {
return nil, errProviderStageMalformed
}
if plan.Operation == config.OperationChatCompletions {
if plan.Effort != "" {
body["reasoning_effort"] = plan.Effort
}
return json.Marshal(body)
}
if plan.Operation != config.OperationResponses {
return nil, errProviderStageMalformed
}
return singleRequestChatToResponses(body, plan)
}
func singleRequestChatToResponses(chat map[string]any, plan providerOperationPlan) ([]byte, error) {
responses := map[string]any{"model": chat["model"], "stream": false}
for _, key := range []string{"temperature", "top_p", "service_tier"} {
if value, exists := chat[key]; exists {
responses[key] = value
}
}
for _, key := range []string{"max_completion_tokens", "max_tokens"} {
if value, exists := chat[key]; exists {
responses["max_output_tokens"] = value
break
}
}
messages, ok := chat["messages"].([]any)
if !ok {
return nil, errProviderStageMalformed
}
input := make([]any, 0, len(messages))
instructions := make([]string, 0, 1)
for _, raw := range messages {
message, ok := raw.(map[string]any)
if !ok {
return nil, errProviderStageMalformed
}
role, _ := message["role"].(string)
content, _ := message["content"].(string)
switch role {
case "system":
if content != "" {
instructions = append(instructions, content)
}
case "user":
input = append(input, map[string]any{"type": "message", "role": "user", "content": []any{map[string]any{"type": "input_text", "text": content}}})
case "assistant":
if content != "" {
input = append(input, map[string]any{"type": "message", "role": "assistant", "content": []any{map[string]any{"type": "output_text", "text": content}}})
}
for _, rawCall := range anySlice(message["tool_calls"]) {
call, ok := rawCall.(map[string]any)
function, functionOK := call["function"].(map[string]any)
if !ok || !functionOK {
return nil, errProviderStageMalformed
}
input = append(input, map[string]any{"type": "function_call", "call_id": call["id"], "name": function["name"], "arguments": function["arguments"]})
}
case "tool":
input = append(input, map[string]any{"type": "function_call_output", "call_id": message["tool_call_id"], "output": content})
default:
return nil, errProviderStageMalformed
}
}
if len(instructions) > 0 {
responses["instructions"] = strings.Join(instructions, "\n\n")
}
responses["input"] = input
if plan.Effort != "" {
responses["reasoning"] = map[string]any{"effort": plan.Effort}
}
if rawTools := anySlice(chat["tools"]); len(rawTools) > 0 {
tools := make([]any, 0, len(rawTools))
for _, rawTool := range rawTools {
tool, ok := rawTool.(map[string]any)
function, functionOK := tool["function"].(map[string]any)
if !ok || !functionOK {
return nil, errProviderStageMalformed
}
converted := map[string]any{"type": "function", "name": function["name"], "parameters": function["parameters"]}
for _, key := range []string{"description", "strict"} {
if value, exists := function[key]; exists {
converted[key] = value
}
}
tools = append(tools, converted)
}
responses["tools"] = tools
}
if choice, ok := chat["tool_choice"]; ok {
responses["tool_choice"] = singleRequestResponsesToolChoice(choice)
}
if parallel, ok := chat["parallel_tool_calls"]; ok {
responses["parallel_tool_calls"] = parallel
}
if format, ok := chat["response_format"].(map[string]any); ok {
if schema, schemaOK := format["json_schema"].(map[string]any); schemaOK {
converted := map[string]any{"type": "json_schema"}
for _, key := range []string{"name", "strict", "schema"} {
converted[key] = schema[key]
}
responses["text"] = map[string]any{"format": converted}
}
}
return json.Marshal(responses)
}
func anySlice(value any) []any {
if value == nil {
return nil
}
items, _ := value.([]any)
return items
}
func singleRequestResponsesToolChoice(choice any) any {
if object, ok := choice.(map[string]any); ok {
if function, functionOK := object["function"].(map[string]any); functionOK {
return map[string]any{"type": "function", "name": function["name"]}
}
}
return choice
}
func normalizeSingleRequestProviderResponse(body []byte, dispatch edgeservice.RunDispatch) ([]byte, error) {
if dispatch.ProfileOperation != string(config.OperationResponses) {
return body, nil
}
var response openAIResponsesBridgeResponse
if err := json.Unmarshal(body, &response); err != nil || strings.TrimSpace(response.ID) == "" {
return nil, errors.Join(errProviderStageGeneric, errProviderStageMalformed)
}
content := ""
toolCalls := make([]any, 0)
for _, item := range response.Output {
switch item.Type {
case "message":
for _, part := range item.Content {
if part.Type == "output_text" {
content += part.Text
}
}
case "function_call":
if item.CallID == "" || item.Name == "" || !json.Valid([]byte(item.Arguments)) {
return nil, errors.Join(errProviderStageGeneric, errProviderStageMalformed)
}
toolCalls = append(toolCalls, map[string]any{"id": item.CallID, "type": "function", "function": map[string]any{"name": item.Name, "arguments": item.Arguments}})
}
}
finishReason := "stop"
if len(toolCalls) > 0 {
finishReason = "tool_calls"
} else if response.Status == "incomplete" && response.IncompleteDetails.Reason == "max_output_tokens" {
finishReason = "length"
}
var contentValue any = content
if len(toolCalls) > 0 && content == "" {
contentValue = nil
}
chat := map[string]any{
"id": response.ID, "object": "chat.completion", "created": int64(0), "model": response.Model,
"choices": []any{map[string]any{"index": 0, "finish_reason": finishReason, "message": map[string]any{"role": "assistant", "content": contentValue, "tool_calls": toolCalls}}},
}
return json.Marshal(chat)
}
func anthropicProviderRequirements(req anthropicMessageRequest) providerRequestRequirements {
requirements := providerRequestRequirements{
HasTools: len(req.Tools) > 0,
@ -163,7 +384,8 @@ func rewriteResponsesProviderControls(body []byte, target string, plan providerO
// requirement is eligible.
func selectProviderOperation(profile config.ConcreteProtocolProfile, ingress config.ProtocolOperation, requirements providerRequestRequirements) (providerOperationPlan, error) {
operations := []config.ProtocolOperation{ingress}
if ingress == config.OperationMessages {
switch ingress {
case config.OperationMessages:
switch profile.Driver {
case config.ProtocolDriverAnthropicMessages:
operations = []config.ProtocolOperation{config.OperationMessages}
@ -174,6 +396,15 @@ func selectProviderOperation(profile config.ConcreteProtocolProfile, ingress con
default:
operations = nil
}
case config.OperationChatCompletions:
switch profile.Driver {
case config.ProtocolDriverOpenAIChat:
operations = []config.ProtocolOperation{config.OperationChatCompletions, config.OperationResponses}
case config.ProtocolDriverOpenAIResponses:
operations = []config.ProtocolOperation{config.OperationResponses}
default:
operations = nil
}
}
for _, operation := range operations {

View file

@ -281,12 +281,25 @@ func TestSingleRequestExecutorConcurrentToolIsolation(t *testing.T) {
mockSvc := &mockService{
submit: func(_ context.Context, req edgeservice.ProviderPoolDispatchRequest) (*edgeservice.ProviderPoolDispatchResult, error) {
reqID := req.Tunnel.SessionID
profile, err := config.ResolveProtocolProfile("gemini", "", config.BuiltInProtocolProfileCatalog())
if err != nil {
return nil, err
}
profile.ID = "profile-1"
candidate := edgeservice.ProviderPoolCandidate{
ActualModel: "gemini-3.6-flash", ProviderID: "gemini", ExecutionPath: string(edgeservice.ProviderPoolPathTunnel),
ProfileID: "profile-1", ProfileDriver: string(profile.Driver), ProfileCapabilities: append([]string(nil), profile.Capabilities...), ProtocolProfile: &profile,
}
tunnelReq, err := prepareSingleRequestTestTunnel(req, candidate)
if err != nil {
return nil, err
}
reqID := tunnelReq.SessionID
if reqID == "" {
return nil, errors.New("missing session ID in tunnel request")
}
reqBody, err := req.Tunnel.BuildBody("gemini-3.6-flash")
reqBody, err := tunnelReq.BuildBody("gemini-3.6-flash")
if err != nil {
return nil, err
}
@ -324,10 +337,12 @@ func TestSingleRequestExecutorConcurrentToolIsolation(t *testing.T) {
resp = executorReviewPassBody(fmt.Sprintf("Reviewer Approved for %s", reqID), fmt.Sprintf("Review pass for %s", reqID))
}
dispatch := matchingDispatch()
dispatch.ProfileOperation = tunnelReq.Operation
return &edgeservice.ProviderPoolDispatchResult{
Path: edgeservice.ProviderPoolPathTunnel,
Tunnel: &mockTunnel{frames: framesFor(resp)},
DispatchInfo: matchingDispatch(),
DispatchInfo: dispatch,
}, nil
},
}

View file

@ -43,6 +43,8 @@ type singleRequestProviderStageResponse struct {
Dispatch edgeservice.RunDispatch
}
type singleRequestProviderBodyBuilder func(string) ([]byte, error)
var (
errProviderStageMissingBinding = errors.New("provider stage: missing stage binding")
errProviderStageMissingInput = errors.New("provider stage: missing messages")
@ -64,18 +66,22 @@ func (s *singleRequestProviderStage) submit(ctx context.Context, req singleReque
stageCtx, cancel := providerStageContext(ctx, req.Limits.StageTimeoutMS)
defer cancel()
requirements := singleRequestProviderRequirements(req.StageBinding.Options, nil, req.ResponseFormat)
poolReq := edgeservice.ProviderPoolDispatchRequest{
Run: edgeservice.SubmitRunRequest{NodeRef: req.NodeRef, ModelGroupKey: dispatch.ModelGroupKey, ProviderID: dispatch.ProviderID, UsageAttribution: req.UsageAttribution, SessionID: req.SessionID, TimeoutSec: dispatch.TimeoutSec, MaxQueue: dispatch.MaxQueue, QueueTimeoutMS: dispatch.QueueTimeoutMS, ProviderPool: true},
Tunnel: edgeservice.SubmitProviderTunnelRequest{
CredentialBinding: dispatch.CredentialBindingSnapshot(), NodeRef: req.NodeRef, ModelGroupKey: dispatch.ModelGroupKey, ProviderID: dispatch.ProviderID, UsageAttribution: req.UsageAttribution,
Adapter: "openai_compat", Target: dispatch.UpstreamModel, SessionID: req.SessionID, Method: http.MethodPost, Path: "/v1/chat/completions", Operation: string(config.OperationChatCompletions), Stream: false,
Adapter: "openai_compat", Target: dispatch.UpstreamModel, SessionID: req.SessionID, Method: http.MethodPost, Path: "/v1/chat/completions", Stream: false,
TimeoutSec: dispatch.TimeoutSec, MaxQueue: dispatch.MaxQueue, QueueTimeoutMS: dispatch.QueueTimeoutMS, ProviderPool: true,
BuildBody: func(target string) ([]byte, error) {
return buildSingleRequestChatBody(req.Messages, req.StageBinding.Options, req.ResponseFormat, target)
},
},
AcceptCandidate: dispatch.CandidatePredicate,
AcceptCandidate: singleRequestProviderCandidatePredicate(dispatch, requirements),
}
poolReq.PrepareProtocolTunnel = singleRequestProviderTunnelPreparer(requirements, func(target string) ([]byte, error) {
return buildSingleRequestChatBody(req.Messages, req.StageBinding.Options, req.ResponseFormat, target)
})
result, err := s.service.SubmitProviderPool(stageCtx, poolReq)
if err != nil || result == nil {
return nil, quality.providerFailure(stageCtx, err, errProviderStageGeneric)
@ -92,6 +98,10 @@ func (s *singleRequestProviderStage) submit(ctx context.Context, req singleReque
if err != nil {
return nil, quality.providerFailure(stageCtx, err, errProviderStageGeneric)
}
body, err = normalizeSingleRequestProviderResponse(body, result.DispatchInfo)
if err != nil {
return nil, quality.providerFailure(stageCtx, err, errProviderStageGeneric)
}
response, err := decodeSingleRequestChatResponse(body, result.DispatchInfo)
if err != nil {
return nil, quality.providerFailure(stageCtx, err, errProviderStageGeneric)
@ -109,7 +119,20 @@ func providerStageContext(parent context.Context, timeoutMS int) (context.Contex
func providerStageDispatchMatches(result *edgeservice.ProviderPoolDispatchResult, frozen *edgeservice.SingleRequestStageDispatchBinding) bool {
d := result.DispatchInfo
return result.Path == edgeservice.ProviderPoolPathTunnel && d.ProfileDriver == string(config.ProtocolDriverOpenAIChat) && d.ModelGroupKey == frozen.ModelGroupKey && d.ProviderID == frozen.ProviderID && d.Target == frozen.UpstreamModel && d.ProfileID == frozen.ProfileID && d.CredentialSlotRef == frozen.CredentialSlotRef && d.CredentialRevision == frozen.CredentialRevision && d.ExecutionPath == string(edgeservice.ProviderPoolPathTunnel)
operation := config.ProtocolOperation(d.ProfileOperation)
driver := config.ProtocolDriver(d.ProfileDriver)
// A default managed resource selector intentionally freezes no provider ID:
// the provider pool owns the concrete provider choice while profile, target,
// credential, and execution-path fences remain fixed. Explicit selectors
// continue to require the exact admitted provider.
providerMatches := frozen.ProviderID == "" || d.ProviderID == frozen.ProviderID
return result.Path == edgeservice.ProviderPoolPathTunnel &&
(operation == config.OperationChatCompletions || operation == config.OperationResponses || operation == "") &&
(driver == config.ProtocolDriverOpenAIChat || driver == config.ProtocolDriverOpenAIResponses) &&
d.ModelGroupKey == frozen.ModelGroupKey && providerMatches &&
d.Target == frozen.UpstreamModel && d.ProfileID == frozen.ProfileID &&
d.CredentialSlotRef == frozen.CredentialSlotRef && d.CredentialRevision == frozen.CredentialRevision &&
d.ExecutionPath == string(edgeservice.ProviderPoolPathTunnel)
}
func collectProviderStageFrames(ctx context.Context, frames <-chan *iop.ProviderTunnelFrame, maximum int) ([]byte, error) {

View file

@ -125,6 +125,16 @@ func matchingDispatch() edgeservice.RunDispatch {
}
}
func prepareSingleRequestTestTunnel(req edgeservice.ProviderPoolDispatchRequest, candidate edgeservice.ProviderPoolCandidate) (edgeservice.SubmitProviderTunnelRequest, error) {
if req.PrepareProtocolTunnel != nil {
return req.PrepareProtocolTunnel(req.Tunnel, candidate)
}
if req.PrepareTunnel != nil {
return req.PrepareTunnel(req.Tunnel)
}
return req.Tunnel, nil
}
func framesFor(body []byte) chan *iop.ProviderTunnelFrame {
c := make(chan *iop.ProviderTunnelFrame, 3)
c <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_RESPONSE_START, StatusCode: 200}
@ -169,7 +179,7 @@ func TestSingleRequestProviderStageUsesFrozenOptionsAndDispatch(t *testing.T) {
t.Fatalf("unexpected Run dispatch: %+v", captured.Run)
}
if captured.Tunnel.NodeRef != "node" || captured.Tunnel.ModelGroupKey != "plan-model" || captured.Tunnel.ProviderID != "gemini" || captured.Tunnel.UsageAttribution != "principal" || captured.Tunnel.Adapter != "openai_compat" || captured.Tunnel.Target != "gemini-3.6-flash" || captured.Tunnel.SessionID != "session" || captured.Tunnel.Method != http.MethodPost || captured.Tunnel.Path != "/v1/chat/completions" || captured.Tunnel.Operation != string(config.OperationChatCompletions) || captured.Tunnel.Stream || captured.Tunnel.TimeoutSec != 60 || captured.Tunnel.MaxQueue != 10 || captured.Tunnel.QueueTimeoutMS != 5000 || !captured.Tunnel.ProviderPool || captured.Tunnel.RunID != "" || captured.Tunnel.Headers != nil || captured.Tunnel.Body != nil || captured.Tunnel.Metadata != nil || captured.Tunnel.EstimatedInputTokens != 0 || captured.Tunnel.ContextClass != "" || captured.Tunnel.ResponseStallTimeoutMS != 0 {
if captured.Tunnel.NodeRef != "node" || captured.Tunnel.ModelGroupKey != "plan-model" || captured.Tunnel.ProviderID != "gemini" || captured.Tunnel.UsageAttribution != "principal" || captured.Tunnel.Adapter != "openai_compat" || captured.Tunnel.Target != "gemini-3.6-flash" || captured.Tunnel.SessionID != "session" || captured.Tunnel.Method != http.MethodPost || captured.Tunnel.Path != "/v1/chat/completions" || captured.Tunnel.Operation != "" || captured.Tunnel.Stream || captured.Tunnel.TimeoutSec != 60 || captured.Tunnel.MaxQueue != 10 || captured.Tunnel.QueueTimeoutMS != 5000 || !captured.Tunnel.ProviderPool || captured.Tunnel.RunID != "" || captured.Tunnel.Headers != nil || captured.Tunnel.Body != nil || captured.Tunnel.Metadata != nil || captured.Tunnel.EstimatedInputTokens != 0 || captured.Tunnel.ContextClass != "" || captured.Tunnel.ResponseStallTimeoutMS != 0 {
t.Fatalf("unexpected Tunnel request: %+v", captured.Tunnel)
}
@ -190,12 +200,112 @@ func TestSingleRequestProviderStageUsesFrozenOptionsAndDispatch(t *testing.T) {
t.Fatalf("unexpected frozen body: %s", body)
}
// Check candidate predicate function matches dispatch with both accepted and rejected candidates
if captured.AcceptCandidate == nil || !captured.AcceptCandidate(edgeservice.ProviderPoolCandidate{ProviderID: "gemini"}) || captured.AcceptCandidate(edgeservice.ProviderPoolCandidate{ProviderID: "other"}) {
profile, err := config.ResolveProtocolProfile("gemini", "", config.BuiltInProtocolProfileCatalog())
if err != nil {
t.Fatal(err)
}
accepted := edgeservice.ProviderPoolCandidate{ProviderID: "gemini", ExecutionPath: string(edgeservice.ProviderPoolPathTunnel), ProtocolProfile: &profile}
rejected := accepted
rejected.ProviderID = "other"
// Check candidate predicate against the same concrete profile facts supplied
// by the real provider-pool service.
if captured.AcceptCandidate == nil || !captured.AcceptCandidate(accepted) || captured.AcceptCandidate(rejected) {
t.Fatalf("AcceptCandidate predicate missing or returned unexpected result")
}
}
func TestSingleRequestProviderNormalizationSelectsResponsesAndFallsBackEffort(t *testing.T) {
catalog := config.BuiltInProtocolProfileCatalog()
openAI := catalog["openai"]
responses := openAI.Normalization.Effort[string(config.OperationResponses)]
delete(responses.Levels, "max")
openAI.Normalization.Effort[string(config.OperationResponses)] = responses
catalog["openai-without-max"] = openAI
profile, err := config.ResolveProtocolProfile("openai-without-max", "", catalog)
if err != nil {
t.Fatal(err)
}
tools := []any{singleRequestWorkToolSchema("workspace_read", map[string]any{
"type": "object", "properties": map[string]any{}, "additionalProperties": false,
})}
requirements := singleRequestProviderRequirements(map[string]any{"reasoning_effort": "max"}, tools, singleRequestPlanResponseFormat())
plan, err := selectProviderOperation(profile, config.OperationChatCompletions, requirements)
if err != nil {
t.Fatal(err)
}
if plan.Operation != config.OperationResponses || plan.Effort != "xhigh" || plan.EffortWire != config.ProtocolEffortWireOpenAIResponses {
t.Fatalf("plan=%+v, want Responses max->xhigh", plan)
}
chatBody := func(target string) ([]byte, error) {
return json.Marshal(map[string]any{
"model": target, "stream": false, "reasoning_effort": "max", "max_completion_tokens": 256,
"messages": []any{
map[string]any{"role": "system", "content": "system"},
map[string]any{"role": "user", "content": "task"},
},
"tools": tools, "tool_choice": "auto", "response_format": singleRequestPlanResponseFormat(),
})
}
prepared, err := singleRequestProviderTunnelPreparer(requirements, chatBody)(
edgeservice.SubmitProviderTunnelRequest{Path: "/v1/chat/completions"},
edgeservice.ProviderPoolCandidate{ExecutionPath: string(edgeservice.ProviderPoolPathTunnel), ProtocolProfile: &profile},
)
if err != nil {
t.Fatal(err)
}
if prepared.Operation != string(config.OperationResponses) || prepared.Path != "/v1/responses" {
t.Fatalf("prepared operation/path=%q/%q", prepared.Operation, prepared.Path)
}
body, err := prepared.BuildBody("gpt-model")
if err != nil {
t.Fatal(err)
}
var decoded map[string]any
if err := json.Unmarshal(body, &decoded); err != nil {
t.Fatal(err)
}
reasoning, _ := decoded["reasoning"].(map[string]any)
text, _ := decoded["text"].(map[string]any)
format, _ := text["format"].(map[string]any)
if decoded["model"] != "gpt-model" || decoded["max_output_tokens"] != float64(256) || reasoning["effort"] != "xhigh" || format["type"] != "json_schema" || len(anySlice(decoded["tools"])) != 1 {
t.Fatalf("normalized Responses body=%s", body)
}
}
func TestSingleRequestProviderNormalizationConvertsResponsesResult(t *testing.T) {
raw := []byte(`{"id":"resp-1","model":"gpt-model","status":"completed","output":[{"type":"message","content":[{"type":"output_text","text":"{\"goal\":\"g\",\"steps\":[\"a\",\"b\"],\"verification\":[\"v\"]}"}]}]}`)
body, err := normalizeSingleRequestProviderResponse(raw, edgeservice.RunDispatch{ProfileOperation: string(config.OperationResponses)})
if err != nil {
t.Fatal(err)
}
response, err := decodeSingleRequestChatResponse(body, edgeservice.RunDispatch{ProfileOperation: string(config.OperationResponses)})
if err != nil {
t.Fatal(err)
}
if response.Output != `{"goal":"g","steps":["a","b"],"verification":["v"]}` {
t.Fatalf("output=%q", response.Output)
}
}
func TestSingleRequestProviderStageAcceptsDefaultSelectorProviderChoice(t *testing.T) {
request := providerRequest()
request.StageBinding.Dispatch.ProviderID = ""
tunnel := &mockTunnel{frames: framesFor(successBody("ok"))}
dispatch := matchingDispatch()
stage := newSingleRequestProviderStage(&mockService{submit: func(context.Context, edgeservice.ProviderPoolDispatchRequest) (*edgeservice.ProviderPoolDispatchResult, error) {
return &edgeservice.ProviderPoolDispatchResult{Path: edgeservice.ProviderPoolPathTunnel, Tunnel: tunnel, DispatchInfo: dispatch}, nil
}})
response, err := stage.submit(context.Background(), request)
if err != nil {
t.Fatal(err)
}
if response.Output != "ok" || response.Dispatch.ProviderID != "gemini" {
t.Fatalf("response=%+v", response)
}
}
func TestSingleRequestProviderStageRejectsResponseEnvelope(t *testing.T) {
tests := []struct {
name string

View file

@ -10,7 +10,6 @@ import (
"strings"
edgeservice "iop/apps/edge/internal/service"
"iop/packages/go/config"
"iop/packages/go/singlerequesttemplate"
)
@ -229,11 +228,15 @@ func (s *singleRequestReviewStage) submit(ctx context.Context, req singleRequest
defer cancel()
poolReq := edgeservice.ProviderPoolDispatchRequest{
Run: edgeservice.SubmitRunRequest{NodeRef: req.NodeRef, ModelGroupKey: dispatch.ModelGroupKey, ProviderID: dispatch.ProviderID, UsageAttribution: req.UsageAttribution, SessionID: req.SessionID, TimeoutSec: dispatch.TimeoutSec, MaxQueue: dispatch.MaxQueue, QueueTimeoutMS: dispatch.QueueTimeoutMS, ProviderPool: true},
Tunnel: edgeservice.SubmitProviderTunnelRequest{CredentialBinding: dispatch.CredentialBindingSnapshot(), NodeRef: req.NodeRef, ModelGroupKey: dispatch.ModelGroupKey, ProviderID: dispatch.ProviderID, UsageAttribution: req.UsageAttribution, Adapter: "openai_compat", Target: dispatch.UpstreamModel, SessionID: req.SessionID, Method: http.MethodPost, Path: "/v1/chat/completions", Operation: string(config.OperationChatCompletions), Stream: false, TimeoutSec: dispatch.TimeoutSec, MaxQueue: dispatch.MaxQueue, QueueTimeoutMS: dispatch.QueueTimeoutMS, ProviderPool: true, BuildBody: func(target string) ([]byte, error) {
Tunnel: edgeservice.SubmitProviderTunnelRequest{CredentialBinding: dispatch.CredentialBindingSnapshot(), NodeRef: req.NodeRef, ModelGroupKey: dispatch.ModelGroupKey, ProviderID: dispatch.ProviderID, UsageAttribution: req.UsageAttribution, Adapter: "openai_compat", Target: dispatch.UpstreamModel, SessionID: req.SessionID, Method: http.MethodPost, Path: "/v1/chat/completions", Stream: false, TimeoutSec: dispatch.TimeoutSec, MaxQueue: dispatch.MaxQueue, QueueTimeoutMS: dispatch.QueueTimeoutMS, ProviderPool: true, BuildBody: func(target string) ([]byte, error) {
return buildSingleRequestReviewBody(messages, req.StageBinding.Options, tools, target, repairRequired)
}},
AcceptCandidate: dispatch.CandidatePredicate,
}
requirements := singleRequestProviderRequirements(req.StageBinding.Options, tools, singleRequestReviewResponseFormat())
poolReq.AcceptCandidate = singleRequestProviderCandidatePredicate(dispatch, requirements)
poolReq.PrepareProtocolTunnel = singleRequestProviderTunnelPreparer(requirements, func(target string) ([]byte, error) {
return buildSingleRequestReviewBody(messages, req.StageBinding.Options, tools, target, repairRequired)
})
result, err := s.provider.service.SubmitProviderPool(stageCtx, poolReq)
if err != nil || result == nil || result.Tunnel == nil {
return nil, quality.providerFailure(stageCtx, err, errSingleRequestReviewStage)
@ -246,6 +249,10 @@ func (s *singleRequestReviewStage) submit(ctx context.Context, req singleRequest
if err != nil {
return nil, quality.providerFailure(stageCtx, err, errSingleRequestReviewStage)
}
body, err = normalizeSingleRequestProviderResponse(body, result.DispatchInfo)
if err != nil {
return nil, quality.providerFailure(stageCtx, err, errSingleRequestReviewStage)
}
response, err := decodeSingleRequestReviewProviderResponse(body, req.Limits.MaxOutputBytes)
if err != nil {
if !errors.Is(err, errProviderStageOutputLimit) && !errors.Is(err, errProviderStageContextLimit) {

View file

@ -1037,7 +1037,7 @@ func assertSingleRequestReviewDispatchAuthority(t *testing.T, got edgeservice.Pr
SessionID: "review-session",
Method: "POST",
Path: "/v1/chat/completions",
Operation: string(config.OperationChatCompletions),
Operation: "",
Stream: false,
TimeoutSec: dispatch.TimeoutSec,
MaxQueue: dispatch.MaxQueue,
@ -1051,10 +1051,16 @@ func assertSingleRequestReviewDispatchAuthority(t *testing.T, got edgeservice.Pr
if got.AcceptCandidate == nil {
t.Fatal("missing AcceptCandidate predicate")
}
if !got.AcceptCandidate(edgeservice.ProviderPoolCandidate{ProviderID: dispatch.ProviderID}) {
profile, err := config.ResolveProtocolProfile("gemini", "", config.BuiltInProtocolProfileCatalog())
if err != nil {
t.Fatal(err)
}
accepted := edgeservice.ProviderPoolCandidate{ProviderID: dispatch.ProviderID, ExecutionPath: string(edgeservice.ProviderPoolPathTunnel), ProtocolProfile: &profile}
if !got.AcceptCandidate(accepted) {
t.Fatalf("AcceptCandidate rejected matching provider %q", dispatch.ProviderID)
}
if got.AcceptCandidate(edgeservice.ProviderPoolCandidate{ProviderID: "rejected-provider"}) {
accepted.ProviderID = "rejected-provider"
if got.AcceptCandidate(accepted) {
t.Fatalf("AcceptCandidate accepted non-matching provider")
}
}

View file

@ -12,7 +12,6 @@ import (
"sync"
edgeservice "iop/apps/edge/internal/service"
"iop/packages/go/config"
)
const (
@ -436,11 +435,18 @@ func (s *singleRequestWorkStage) submit(ctx context.Context, req singleRequestWo
defer cancel()
poolReq := edgeservice.ProviderPoolDispatchRequest{
Run: edgeservice.SubmitRunRequest{NodeRef: req.NodeRef, ModelGroupKey: dispatch.ModelGroupKey, ProviderID: dispatch.ProviderID, UsageAttribution: req.UsageAttribution, SessionID: req.SessionID, TimeoutSec: dispatch.TimeoutSec, MaxQueue: dispatch.MaxQueue, QueueTimeoutMS: dispatch.QueueTimeoutMS, ProviderPool: true},
Tunnel: edgeservice.SubmitProviderTunnelRequest{CredentialBinding: dispatch.CredentialBindingSnapshot(), NodeRef: req.NodeRef, ModelGroupKey: dispatch.ModelGroupKey, ProviderID: dispatch.ProviderID, UsageAttribution: req.UsageAttribution, Adapter: "openai_compat", Target: dispatch.UpstreamModel, SessionID: req.SessionID, Method: http.MethodPost, Path: "/v1/chat/completions", Operation: string(config.OperationChatCompletions), Stream: false, TimeoutSec: dispatch.TimeoutSec, MaxQueue: dispatch.MaxQueue, QueueTimeoutMS: dispatch.QueueTimeoutMS, ProviderPool: true, BuildBody: func(target string) ([]byte, error) {
Tunnel: edgeservice.SubmitProviderTunnelRequest{CredentialBinding: dispatch.CredentialBindingSnapshot(), NodeRef: req.NodeRef, ModelGroupKey: dispatch.ModelGroupKey, ProviderID: dispatch.ProviderID, UsageAttribution: req.UsageAttribution, Adapter: "openai_compat", Target: dispatch.UpstreamModel, SessionID: req.SessionID, Method: http.MethodPost, Path: "/v1/chat/completions", Stream: false, TimeoutSec: dispatch.TimeoutSec, MaxQueue: dispatch.MaxQueue, QueueTimeoutMS: dispatch.QueueTimeoutMS, ProviderPool: true, BuildBody: func(target string) ([]byte, error) {
return buildSingleRequestWorkBody(messages, req.StageBinding.Options, tools, target, completionEligible)
}},
AcceptCandidate: dispatch.CandidatePredicate,
}
requirements := singleRequestProviderRequirements(req.StageBinding.Options, tools, nil)
if completionEligible {
requirements.StructuredOutput = true
}
poolReq.AcceptCandidate = singleRequestProviderCandidatePredicate(dispatch, requirements)
poolReq.PrepareProtocolTunnel = singleRequestProviderTunnelPreparer(requirements, func(target string) ([]byte, error) {
return buildSingleRequestWorkBody(messages, req.StageBinding.Options, tools, target, completionEligible)
})
result, err := s.provider.service.SubmitProviderPool(stageCtx, poolReq)
if err != nil || result == nil || result.Tunnel == nil {
return nil, quality.providerFailure(stageCtx, err, errSingleRequestWorkStage)
@ -453,6 +459,10 @@ func (s *singleRequestWorkStage) submit(ctx context.Context, req singleRequestWo
if err != nil {
return nil, quality.providerFailure(stageCtx, err, errSingleRequestWorkStage)
}
body, err = normalizeSingleRequestProviderResponse(body, result.DispatchInfo)
if err != nil {
return nil, quality.providerFailure(stageCtx, err, errSingleRequestWorkStage)
}
response, err := decodeSingleRequestWorkProviderResponse(body, req.Limits.MaxOutputBytes)
if err != nil {
if !errors.Is(err, errProviderStageOutputLimit) && !errors.Is(err, errProviderStageContextLimit) {