fix(edge): provider 호출 정규화 계층을 추가한다
caller별 예외 대신 요청 의미와 protocol profile capability로 operation을 선택해 tools와 effort 조합을 보존한다. 지원하지 않는 effort는 가장 가까운 하위 등급으로만 내리고 상향 매핑은 거부한다.
This commit is contained in:
parent
4f503eed98
commit
e5869a53ae
22 changed files with 1162 additions and 75 deletions
|
|
@ -8,6 +8,7 @@
|
|||
- 원본 경로:
|
||||
- `packages/go/config/edge_types.go`
|
||||
- `packages/go/config/provider_types.go`
|
||||
- `packages/go/config/protocol_profile.go`
|
||||
- `packages/go/config/execution_preset_types.go`
|
||||
- `packages/go/config/load.go`
|
||||
- `packages/go/config/validate.go`
|
||||
|
|
@ -42,7 +43,8 @@ tracked config에는 public 예시와 기본 구조만 두고, 실제 endpoint/c
|
|||
- TLS certificate/key/CA paths, credential keyring paths, issuer signing keys, and Node recipient keys are file references to operator-mounted material. Raw keys, principal tokens, and provider credentials never belong in tracked YAML. Role/name workload identities and HTTP server names must match their configured peer expectations.
|
||||
- Managed provider credentials are selected only through an authenticated projected route. The effective route binds one principal, slot, profile, upstream model, resource selector, credential revision, route revision, and projection generation; caller metadata and legacy provider-auth headers cannot replace any binding field.
|
||||
- `openai.principal_tokens[]`는 raw token을 저장하지 않고 hash/reference로 principal 매핑을 관리한다. 각 entry는 `token_ref` (non-empty, unique), `token_hash_sha256` (64-char hex, duplicate hash rejection), `principal_ref` (non-empty), optional `principal_alias` 필드를 갖는다. 여러 entry가 같은 `principal_ref`와 `principal_alias`를 공유할 수 있으며, 이때 `token_ref`가 앱/통합/용도별 사용량 분해 기준이 된다. tracked config에는 raw token을 저장하지 않고 hash/reference만 둔다.
|
||||
- `protocol_profiles` is the top-level map of custom profile overlays, keyed by stable profile id. Each `ProtocolProfileConf` can declare `base`, `driver`, `base_url`, an operation-path map, `auth`, `capabilities`, `model_mapping`, and `extensions`. A custom overlay extends one built-in or custom base; cycles, unknown bases, and invalid driver/operation/capability combinations are rejected during config normalization.
|
||||
- `protocol_profiles` is the top-level map of custom profile overlays, keyed by stable profile id. Each `ProtocolProfileConf` can declare `base`, `driver`, `base_url`, an operation-path map, `auth`, `capabilities`, `model_mapping`, `normalization`, and `extensions`. A custom overlay extends one built-in or custom base; cycles, unknown bases, and invalid driver/operation/capability combinations are rejected during config normalization.
|
||||
- `normalization.effort[operation]` declares the provider wire, supported normalized grades, whether the operation preserves effort with caller tools, and whether it preserves an explicit thinking token budget. Every normalization operation must exist in the profile operation map. Grade keys use `none|low|medium|high|xhigh|max`; exact miss falls back only to the nearest declared lower key. A canonical mapped value above its source key is rejected so config cannot silently upgrade requested effort. This Edge-local selection fact is consumed before tunnel dispatch and is not serialized into a new caller or Edge-Node wire field.
|
||||
- `nodes[].providers[].profile` selects a built-in or custom catalog entry. If the selector is empty, legacy provider-type normalization can select a compatibility profile; this is distinct from `base` inheritance. Normalization resolves the selection into the runtime-only `ProviderDefinition.RuntimeProfile` snapshot, which is not serialized back into YAML. The resolved snapshot is copied into the nested OpenAI-compatible adapter config, not into a per-request tunnel message.
|
||||
- `ConcreteProtocolProfile.MapModel(model)`은 provider의 model alias 정규화를 수행한다. provider가 model mapping을 정의하면 IOP external `model` key를 provider served target으로 변환한다. 매핑이 없으면 original model을 그대로 사용한다.
|
||||
- `ConcreteProtocolProfile.HasCapability(cap)`는 provider capability admission에 사용된다. closed vocabulary (`models`, `chat`, `messages`, `responses`, `streaming`, `tool_calling`, `count_tokens`)만 허용한다.
|
||||
|
|
|
|||
|
|
@ -375,7 +375,7 @@ Wrong methods on Anthropic-selected endpoints return `405 invalid_request_error`
|
|||
- `tools`: 각 tool은 `name`, `input_schema`를 필수로 가진다. 선택 boolean `defer_loading`은 Claude Code tool-search 호출 호환성 annotation으로만 수용한다. Native Messages raw tunnel은 원문을 보존하지만, decoded Chat bridge와 marked single-request 경로에서는 route, provider, workspace, tool policy 또는 authorization 권한으로 해석하지 않고 normalized Chat provider body에서 제거한다.
|
||||
- `tool_choice`: `auto`, `any`, `none`, `tool` 타입만 허용한다.
|
||||
- `thinking`: 양수 `budget_tokens`가 있는 `type="enabled"` 또는 budget 없는 `type="adaptive"`를 허용한다. 선택 `display`는 Claude Code thinking-redaction 호환성을 위해 `omitted` 또는 `summarized`만 수용한다. Native Messages raw tunnel은 원문을 보존하지만, decoded Chat bridge와 marked single-request 경로에서는 display를 route, stage, provider, workspace, tool policy 또는 authorization 권한으로 해석하지 않고 normalized Chat provider body에서 제거한다. Chat bridge의 `enabled`는 profile의 thinking/reasoning extension이 필요하고, `adaptive`는 `output_config.effort` 기반 provider 제어를 사용한다.
|
||||
- `output_config.effort`: `low`, `medium`, `high`, `xhigh`, `max`를 허용하며 Chat bridge에서 `reasoning_effort`로 변환한다. 대소문자, 별칭, 캐핑, 다운시프트는 허용하지 않는다.
|
||||
- `output_config.effort`: canonical IOP 등급 `low`, `medium`, `high`, `xhigh`, `max`를 허용한다. Edge는 caller identity가 아니라 요청의 tools/thinking/stream 요구와 selected protocol profile의 operation별 normalization을 사용해 provider wire 값을 정한다. exact 등급이 없으면 선언된 가장 가까운 하위 등급으로만 매핑하고 상향하지 않는다. 예를 들어 `max`가 없고 `xhigh`가 있으면 `xhigh`를 사용한다. 어떤 하위 등급도 없으면 dispatch 전에 `not_supported_error`로 거부한다.
|
||||
- `output_config.format`: `type="json_schema"`와 object `schema`를 허용하며 Chat bridge에서 OpenAI-compatible `response_format.json_schema`로 변환한다.
|
||||
- `cache_control`: text/image/tool/tool-result/thinking block과 tool declaration의 compatibility annotation을 수용하되 Chat bridge에서는 정책으로 해석하거나 provider body에 전달하지 않는다.
|
||||
- `metadata`: caller-defined object이며 IOP identity source로 사용하지 않는다. Native Messages 경로는 원문을 보존하고, Chat bridge는 object 여부만 검증한 뒤 provider body에서는 제거한다.
|
||||
|
|
@ -520,7 +520,7 @@ Top-level `models[]` is the static catalog source for IOP model discovery and pr
|
|||
### Native vs Bridge
|
||||
|
||||
선택된 provider의 `ConcreteProtocolProfile.Driver`가 `anthropic_messages`이면 Edge는 provider raw tunnel을 통해 Anthropic-native request/response를 relay한다. Ordinary native routes preserve provider response model/body bytes, while authorized virtual presets rewrite successful response identity to the requested virtual model.
|
||||
`openai_chat`이면 Edge는 Anthropic Messages request를 Chat Completions request로 bridge하고, Chat bridge 응답을 다시 Anthropic Messages response로 변환한다. Authorized virtual presets retain their requested virtual response model identity through that conversion; ordinary bridge responses use the bridge's converted response model semantics.
|
||||
`openai_chat`이면 Edge는 caller-neutral request requirements를 먼저 만들고, profile이 보존 가능한 operation을 선택한다. Chat Completions가 tools+effort 조합을 보존하지 못하지만 같은 profile의 Responses operation이 보존할 수 있으면 `POST /v1/responses`로 bridge하고 결과를 Anthropic Messages 형식으로 되돌린다. Chat operation이 요구사항을 모두 보존할 때만 기존 Chat bridge를 사용한다. Authorized virtual presets retain their requested virtual response model identity through that conversion; ordinary bridge responses use the selected bridge's converted response model semantics.
|
||||
그 외 driver는 `502 api_error` "selected provider returned an unsupported protocol driver"를 반환한다.
|
||||
|
||||
Chat bridge는 Gemini OpenAI-compatible tool call의 `extra_content.google.thought_signature`를 opaque Anthropic `tool_use.id`에 담아 caller에게 전달한다. Caller는 해당 id를 tool result까지 변경 없이 replay해야 하며, 다음 요청에서 Edge는 원래 tool call id와 signature를 복원한다. Signature가 없는 provider의 tool id는 변경하지 않는다.
|
||||
|
|
@ -548,7 +548,8 @@ before response commitment.
|
|||
Anthropic Messages 요청은 선택된 provider가 다음 capability를 가져야 한다:
|
||||
|
||||
- native: `messages` capability + `messages` operation
|
||||
- Chat bridge: `chat` capability + `chat_completions` operation
|
||||
- Chat bridge: `chat` capability + `chat_completions` operation + 요청 control을 보존하는 operation별 normalization
|
||||
- Responses bridge: `responses` capability + `responses` operation + 요청 control을 보존하는 operation별 normalization
|
||||
- `streaming` capability (streaming 요청인 경우)
|
||||
- `tool_calling` capability (tools가 있는 요청인 경우)
|
||||
- `count_tokens` capability + `count_tokens` operation (count_tokens native fallback 요청인 경우; TokenCounter local count path는 provider selection 및 capability check가 필요 없다)
|
||||
|
|
@ -559,6 +560,7 @@ capability 불만족은 `400 not_supported_error`로 종료한다.
|
|||
|
||||
Chat bridge의 explicit `thinking.type="enabled"`와 assistant thinking block 전달은 provider profile의 `extensions.thinking` 또는 `extensions.reasoning`이 `true`일 때만 지원한다. Claude Code가 이전 응답에서 받은 빈 signature의 thinking block을 generic Chat profile 요청에 replay하면 private reasoning block만 제거하고 visible text/tool history는 유지한다. Signed thinking block은 profile과 관계없이 Chat bridge에서 거부한다.
|
||||
해당 profile extension 없이 explicit enabled thinking으로 bridge하면 `400 invalid_request_error` "selected Chat profile does not support thinking"를 반환한다. `thinking.type="adaptive"`는 별도 budget field를 만들지 않고 `output_config.effort`를 `reasoning_effort`로 변환한다.
|
||||
OpenAI profile의 Responses operation은 adaptive effort와 caller tools를 함께 보존할 수 있지만 explicit `thinking.budget_tokens`를 보존하지 않으므로, 이 조합은 Responses로 조용히 변환하지 않고 fail closed한다.
|
||||
|
||||
## Usage Attribution
|
||||
|
||||
|
|
|
|||
|
|
@ -193,6 +193,7 @@ Normalized provider 실행으로 라우팅되는 요청의 최소 형태:
|
|||
- `max_output_tokens`: 출력 길이 상한이다. 내부 provider option의 `max_tokens`로 전달된다.
|
||||
- `temperature`: 생성 다양성 option이다. 대상 adapter가 지원하지 않으면 무시될 수 있다.
|
||||
- `top_p`: nucleus sampling option이다. 대상 adapter가 지원하지 않으면 무시될 수 있다.
|
||||
- `reasoning.effort`: provider tunnel route에서는 canonical IOP effort 등급을 operation별 protocol normalization으로 매핑한다. exact 등급이 없으면 선언된 가장 가까운 하위 등급만 사용하며 상향하지 않는다. 같은 `reasoning` object의 다른 field는 보존한다.
|
||||
|
||||
Normalized route 금지:
|
||||
|
||||
|
|
@ -207,7 +208,7 @@ Normalized route 금지:
|
|||
현재 구현 메모:
|
||||
|
||||
- normalized(non-provider) `/v1/responses` route는 strict field validation을 유지하며 non-streaming string input만 지원한다.
|
||||
- provider-pool model group route(`models[]`)의 `/v1/responses` 호출은 selected provider가 the Responses operation and capability를 선언한 tunnel candidate이면 raw passthrough로 provider `POST /v1/responses`에 전달한다. This admission is not exclusive to the `openai_responses` driver. caller body는 `model` field만 served target으로 rewrite하고, selected provider가 지원하는 OpenAI-compatible 표준 field와 provider extension field(`max_output_tokens`, `tools`, `store`, provider-specific knobs 등)는 보존한다. `stream:true`는 provider raw SSE로 relay한다. Managed mode injects the selected slot lease at the Node; legacy mode may apply configured provider-auth forwarding. Response model echo rewrite is not applied, and this path never falls back to normalized `SubmitRun`.
|
||||
- provider-pool model group route(`models[]`)의 `/v1/responses` 호출은 selected provider가 Responses operation/capability와 요청의 tool/effort semantics를 선언한 tunnel candidate이면 provider `POST /v1/responses`에 전달한다. This admission is not exclusive to the `openai_responses` driver. caller body는 served target과, 필요한 경우 operation별로 매핑된 `reasoning.effort`만 rewrite하고 `max_output_tokens`, `tools`, `store`, provider-specific knobs 및 `reasoning`의 다른 field는 보존한다. `stream:true`는 provider raw SSE로 relay한다. Managed mode injects the selected slot lease at the Node; legacy mode may apply configured provider-auth forwarding. Response model echo rewrite is not applied, and this path never falls back to normalized `SubmitRun`.
|
||||
- provider-pool model group route는 provider candidate를 먼저 선택한다. 선택된 provider가 OpenAI-compatible 호출 방식을 지원하면 `ProviderTunnelRequest` passthrough를 사용하고, Ollama/native provider이면 normalized `RunRequest`를 사용한다. provider type만으로 Ollama를 candidate set에서 제거하지 않으며, OpenAI-compatible provider의 tunnel 구현이 없으면 normalized fallback이 아니라 unsupported/implementation error다.
|
||||
- provider-pool pending request는 lease 반환, config refresh, provider disable, Node disconnect/reconnect 때 live config와 dispatch-ready registry에서 candidate를 다시 계산한다. 후보가 full인 상태는 queue policy에 따라 계속 대기하지만 live candidate가 모두 사라지면 원래 queue timeout까지 기다리지 않고 terminal unavailable로 끝난다.
|
||||
- provider-pool admission/unavailable 실패는 현재 외부 error envelope를 유지해 HTTP `502`와 `type="node_dispatch_error"`로 반환한다. 별도 public status code나 response field를 추가하지 않으며 error message에는 raw token이나 private endpoint를 포함하지 않는다.
|
||||
|
|
@ -271,7 +272,7 @@ Chat Completions의 실행 경로는 caller가 보낸 `model`의 route/provider
|
|||
IOP 확장 think 제어 field:
|
||||
|
||||
- `think` (bool, optional): thinking/reasoning 생성 활성화 여부를 표현하는 IOP 확장 field다. 생략하면 provider 기본값을 유지한다. `false`는 thinking 생성을 끄도록 요청하고, `true`는 provider가 지원하면 thinking 생성을 명시 활성화한다.
|
||||
- `reasoning_effort` (string, optional): `none`, `low`, `medium`, `high` 중 하나인 IOP 확장 field다. `none`은 `think=false`와 같은 disable 의미로 처리한다. `low`/`medium`/`high`는 provider 또는 normalized backend가 지원하는 경우에만 전달한다.
|
||||
- `reasoning_effort` (string, optional): `none`, `low`, `medium`, `high`, `xhigh`, `max` 중 하나인 IOP 확장 field다. `none`은 `think=false`와 같은 disable 의미로 처리한다. provider profile이 operation별 effort scale을 더 작게 선언하면 exact 또는 가장 가까운 하위 등급으로 매핑하고, 상향 매핑은 config load에서 거부한다.
|
||||
- `thinking_token_budget` (int, optional): IOP 확장 thinking token budget. 0 이상이어야 한다.
|
||||
- `include_reasoning` (bool, optional): OpenAI-compatible 응답에서 `reasoning_content` 노출 여부. non-provider normalized route에서는 생략하거나 `true`이면 provider reasoning delta/message를 노출할 수 있고, `false`이면 provider가 reasoning을 생성해도 response의 `reasoning_content`를 제거한다. provider-pool pure `passthrough`는 provider body 보존이 우선이며, 현재 IOP가 이 field만으로 reasoning field를 제거한다고 보장하지 않는다.
|
||||
|
||||
|
|
@ -331,7 +332,7 @@ Provider pool model catalog의 `models[]` entry가 generation policy를 제공
|
|||
|
||||
Conflict 정책:
|
||||
|
||||
- `reasoning_effort`가 비어 있거나 `none|low|medium|high` 외 값이면 400 에러.
|
||||
- `reasoning_effort`가 비어 있거나 `none|low|medium|high|xhigh|max` 외 값이면 400 에러.
|
||||
- `thinking_token_budget`가 음수이면 400 에러.
|
||||
- `think=false`와 `reasoning_effort=low|medium|high`가 함께 있으면 400 에러.
|
||||
- `think=false`일 때 `thinking_token_budget`를 설정하면 400 에러.
|
||||
|
|
|
|||
|
|
@ -43,7 +43,7 @@ IOP 전체 안정성을 처음부터 재검증하지 않고, 실패가 재현된
|
|||
|
||||
### Epic: [route-smoke] 벤치 경로 초경량 확인
|
||||
|
||||
- [ ] [minimal-html-calls] 9개 조합에 동일한 최소 HTML 요청을 한 번씩 직접 실행하고, 경로별 caller/model/route, terminal, 경과 시간, `index.html` marker 확인 결과를 한 개의 Markdown 표에 기록한다. 검증: 새 runner/manifest 없이 각 행에 실제 호출 결과가 하나만 있어야 한다.
|
||||
- [ ] [minimal-html-calls] 9개 조합에 동일한 최소 HTML 요청을 한 번씩 직접 실행하고, 경로별 caller/model/route, terminal, 경과 시간, `index.html` marker 확인 결과를 한 개의 Markdown 표에 기록한다. 사설 dev CA가 필요한 호출은 `SSL_CERT_FILE`과 `NODE_EXTRA_CA_CERTS`를 해당 caller process에만 command-scoped로 전달한다. 검증: 새 runner/manifest 없이 각 행에 실제 호출 결과가 하나만 있어야 하며, 호출 전후 ambient Codex/IDE/shell 환경에는 두 CA 변수가 없어야 한다.
|
||||
- [ ] [failed-path-fixes] 실패한 조합마다 제품·caller·provider·환경 중 소유 경계를 기록하고, IOP 제품 결함이 재현된 경우에만 국소 수정과 focused regression을 수행한 뒤 해당 조합만 다시 호출한다. 검증: 성공한 조합의 반복 실행이 없고, 재실행 행에는 변경된 원인과 연결된 수정·테스트 근거가 있어야 한다.
|
||||
- [ ] [thin-bench-handoff] 9개 조합의 통과 또는 구체적 외부 차단 상태를 짧게 정리해 `[bench-lite-01]` 실행 가능 여부를 남긴다. 검증: 비교 점수나 순위가 아니라 호출 가능 여부와 남은 소유자만 기록한다.
|
||||
|
||||
|
|
@ -71,7 +71,8 @@ IOP 전체 안정성을 처음부터 재검증하지 않고, 실패가 재현된
|
|||
|
||||
- 계획 범위: 짧은 단일 test plan으로 즉시 실행할 수 있게 유지한다. 별도 설계·SDD·다단계 복구 계획으로 확장하지 않는다.
|
||||
- 실행 방식: 기존 공식 caller 명령을 한 번씩 직접 실행한다. 공통화가 필요해 보여도 이 Milestone에서는 script로 승격하지 않는다.
|
||||
- TLS 환경 경계: 개발 Edge용 사설 CA는 해당 벤치 caller process에만 전달한다. `SSL_CERT_FILE`과 `NODE_EXTRA_CA_CERTS`를 Codex/IDE 시작 환경이나 셸 전역에 `export`하지 않는다. 그렇지 않으면 공개 TLS 연결에도 같은 CA override가 적용될 수 있다.
|
||||
- evidence 위치: `agent-test/dev/iop-benchmark-route-minimal-html-smoke.md`
|
||||
- 현재 사전 확인: 2026-08-13에 Claude Code 2.1.228, agy 1.1.12, Codex 0.147.0 실행 파일과 CA/token 파일의 존재·mode를 확인했으나, 기존 `token/.iop-bench`로 public `/v1/models`가 HTTP 401을 반환했다. 현재 public Edge 활성 config에는 이 token hash 매핑이 0건이며 caller/model 호출은 시작하지 않았다.
|
||||
- 재개 조건: 기존 5개 route를 볼 수 있는 유효한 dev-corp principal token을 operator-private 경로에 준비하거나 기존 test token을 active Edge에 안전하게 매핑·재시작하고, token 원문을 출력하지 않은 `/v1/models` 확인이 200이어야 한다.
|
||||
- 현재 사전 확인: 2026-08-13에 Claude Code 2.1.228, agy 1.1.12, Codex 0.147.0을 확인했고, 원격 SOPS에 보관된 기존 IOP principal token으로 token 원문을 출력하지 않은 `/v1/models`가 HTTP 200임을 확인했다. 새 벤치 전용 token은 발급하거나 사용하지 않는다.
|
||||
- 현재 경로 결과: Claude Code → Claude direct와 Claude Code → Gemini direct는 최소 HTML 1회 호출을 통과했다. Claude Code → GPT direct는 Chat Completions의 tools+reasoning 조합 미지원으로 실패했고, IOP provider operation normalization 결함으로 귀속했다. caller-neutral operation 선택, Messages↔Responses 변환, nearest-lower effort mapping의 focused regression은 통과했으며 개발 런타임 재검증이 남아 있다.
|
||||
- 후속 측정: [초경량 Agent 모델 비교](thin-agent-model-comparison-benchmark.md)
|
||||
|
|
|
|||
|
|
@ -74,13 +74,22 @@ source_evidence:
|
|||
notes: Anthropic native tunnel response relay with header allowlist
|
||||
- type: code
|
||||
path: apps/edge/internal/openai/anthropic_bridge.go
|
||||
notes: Anthropic Messages ↔ Chat Completions bidirectional bridge
|
||||
notes: Anthropic Messages ↔ Chat Completions request/response bridge
|
||||
- type: code
|
||||
path: apps/edge/internal/openai/provider_normalization.go
|
||||
notes: Caller-neutral provider operation selection, effort fallback, Messages ↔ Responses conversion
|
||||
- type: code
|
||||
path: apps/edge/internal/openai/anthropic_stream.go
|
||||
notes: Chat/Responses provider output을 Anthropic Messages JSON/SSE로 변환
|
||||
- type: code
|
||||
path: apps/edge/internal/openai/anthropic_types.go
|
||||
notes: Anthropic request/response types, header validation, content block decode
|
||||
- type: test
|
||||
path: apps/edge/internal/openai/anthropic_bridge_test.go
|
||||
notes: Claude Code beta/request mapping과 Gemini thought signature 왕복 검증
|
||||
notes: Messages compatibility mapping, Responses operation selection, effort fallback, Gemini thought signature 왕복 검증
|
||||
- type: test
|
||||
path: apps/edge/internal/openai/responses_protocol_profile_test.go
|
||||
notes: Responses effort nearest-lower mapping과 나머지 reasoning field 보존 검증
|
||||
- type: code
|
||||
path: apps/edge/internal/openai/principal.go
|
||||
notes: Shared principal token hash auth for both OpenAI and Anthropic surfaces
|
||||
|
|
@ -89,7 +98,7 @@ source_evidence:
|
|||
notes: Shared provider tunnel auth headers and passthrough
|
||||
- type: code
|
||||
path: packages/go/config/protocol_profile.go
|
||||
notes: ConcreteProtocolProfile, ProtocolOperation, ProtocolDriver, capability admission, model mapping
|
||||
notes: ConcreteProtocolProfile, operation capability admission, model/effort normalization
|
||||
- type: code
|
||||
path: apps/edge/internal/openai/run_result.go
|
||||
notes: RunEvent stream을 OpenAI-compatible result로 수집
|
||||
|
|
@ -189,7 +198,7 @@ Edge가 OpenAI-compatible HTTP 요청을 받아 내부 `adapter + target` 실행
|
|||
| Anthropic ingress | `POST /v1/messages` and `POST /anthropic/v1/messages` share one handler; the corresponding count-tokens paths share another. `/anthropic/v1/models`, and `/v1/models` with `anthropic-version`, return the Anthropic model-list shape. Wrong methods return `405 invalid_request_error`. |
|
||||
| Anthropic caller auth | Anthropic ingress accepts `Authorization: Bearer <token>` or `X-Api-Key: <token>`. If both are present they must match; shared principal-token and legacy bearer fallback apply after this validation. |
|
||||
| Anthropic provider-pool dispatch | Messages and count-tokens require a provider-pool model route. Native Messages requires `messages` capability and operation, while the Chat bridge requires `chat` capability and `chat_completions` operation; streaming and tools add their own capability checks. |
|
||||
| Claude Code Chat bridge | Supported Claude Code beta headers are consumed at the bridge, adaptive effort (low/medium/high/xhigh/max) maps to Chat `reasoning_effort`, JSON schema output maps to `response_format`, Anthropic metadata/cache-control annotations are stripped, Gemini tool thought signatures round-trip through opaque tool-use ids, and unsigned private thinking replay is dropped only for generic Chat profiles that cannot represent it. |
|
||||
| provider-normalized Messages bridge | Supported Messages compatibility headers are consumed at the bridge. Edge derives caller-neutral tool/effort/token-budget/stream requirements, selects a profile operation that preserves them, and maps effort to exact or nearest lower provider grade. Chat-compatible providers may therefore use Chat or Responses without caller-name branches. JSON schema and tool shapes are converted for the selected wire; Gemini tool thought signatures still round-trip through opaque tool-use ids. |
|
||||
| bounded ingress and StreamGate ownership | Chat/Responses bodies are limited to 16 MiB before the first read. Every supported path delegates response-start staging, applicable filter arbitration, bounded liveness recovery, and the single terminal to `runtime/stream-evidence-gate`; `enabled` controls configured semantic policy only. |
|
||||
| typed stall terminal | Supported Chat/Responses normalized and tunnel attempts always translate only Edge-confirmed `response_stalled` terminals into a raw-free liveness recovery candidate; post-commit, cancelled, tool-bearing, missing-snapshot, exhausted, unsupported, unconfirmed, generic, and no-owner paths stay terminal. |
|
||||
| liveness operational evidence | Each private liveness cycle emits one closed eligibility counter and at most one closed final-result counter. Constructor-owned generic logs use a safe projection without identifiers or payloads, while application-installed observation sinks retain the original immutable events. |
|
||||
|
|
@ -203,7 +212,7 @@ Edge가 OpenAI-compatible HTTP 요청을 받아 내부 `adapter + target` 실행
|
|||
| reasoning observation metric | provider가 reasoning token을 보고하지 않고 reasoning text만 관측되면 관측 횟수와 character count 보조 metric을 emit하고, 별도 estimated-token counter(`iop_openai_reasoning_estimated_tokens_total`)로 `estimation_method="chars_div_4"` 추정을 제공한다. |
|
||||
| Grafana usage surface | 1차 조회 표면은 Prometheus/Grafana query guide이며 actual `provider_id`·`served_model` 기준 daily/monthly rollup과 `usage_attribution=model_group`으로 승인된 `route_model` query-time rollup, usage origin breakdown, operator-managed cloud price baseline, cloud-equivalent cost, avoided-cost ROI 기준을 문서로 제공한다. Control Plane/Client dashboard와 request-level ledger는 후속 범위다. |
|
||||
| Responses API | normalized(non-provider) `/v1/responses` supports only non-streaming string input. A provider model-group route relays `/v1/responses` to the selected provider when that candidate declares the Responses operation/capability; this is not exclusive to one driver. |
|
||||
| Responses provider passthrough | provider-pool model group route와 direct OpenAI-compatible provider route의 `/v1/responses`는 provider raw tunnel을 사용한다. Edge는 `model`만 served target으로 rewrite하고 unknown/Codex field와 `stream:true` raw SSE를 provider로 relay한다. Usage is recorded with endpoint=`responses`, response_mode=`passthrough`, route_model=request alias, and the selected actual provider/served model. Responses는 선택적 기능이다. |
|
||||
| Responses provider passthrough | provider-pool model group route와 direct OpenAI-compatible provider route의 `/v1/responses`는 provider raw tunnel을 사용한다. Edge는 served `model`과 필요한 operation-normalized `reasoning.effort`만 rewrite하고 unknown/Codex field, 다른 reasoning field와 `stream:true` raw SSE를 보존한다. Usage is recorded with endpoint=`responses`, response_mode=`passthrough`, route_model=request alias, and the selected actual provider/served model. Responses는 선택적 기능이다. |
|
||||
| strict output | strict output이 켜져 있으면 XML completion contract 기반 instruction 또는 prompt prefix를 추가할 수 있다. |
|
||||
| tool call 처리 | Chat Completions `tools`는 provider native metadata 복원 또는 text tool-call synthesis/validation 경로를 사용한다. Anthropic Messages `tools`는 Chat bridge를 통해 OpenAI `tools`로 변환되거나, native Anthropic tunnel로 직접 전달된다. |
|
||||
| cancel 전파 | HTTP caller timeout/cancel이 cancel-worthy error이면 Node `CancelRun`으로 전파한다. |
|
||||
|
|
@ -271,6 +280,7 @@ sequenceDiagram
|
|||
- In legacy mode, `openai.provider_auth` stores only a forwarding rule and reads raw provider material from its request-time header; inbound IOP authorization is never reused. Managed mode rejects that rule and the caller header and uses only the sealed slot lease.
|
||||
- OpenAI request metadata is bounded caller context. Workspace, runtime, and session ownership are outside this input surface.
|
||||
- Chat Completions와 Responses request는 caller metadata로 provider raw tunnel과 normalized response shape를 선택하지 않는다. route/provider capability만 실행 경로를 결정한다.
|
||||
- Provider operation selection never branches on caller/agent identity. It evaluates the selected profile against normalized request requirements. Effort uses `none < low < medium < high < xhigh < max`; an unsupported grade may fall only to the nearest declared lower grade, and no lower grade means fail-closed admission.
|
||||
- run metadata에는 `openai_model`, `openai_stream`, `strict_output`, `estimated_input_tokens`, `context_class`가 들어갈 수 있다.
|
||||
- provider tunnel metadata에는 routing context와 관측 후보가 들어갈 수 있으며, provider body에는 합쳐지지 않는다.
|
||||
- Node complete event metadata의 `openai_tool_calls`와 `openai_text_tool_fallback`은 response tool call 복원에 쓰인다.
|
||||
|
|
@ -354,6 +364,7 @@ sequenceDiagram
|
|||
- 2026-08-02: Removed IOP-owned workspace and Agent/CLI runtime semantics while preserving bounded metadata, managed projection, and credential lease behavior.
|
||||
- 2026-08-05: Added Claude Code adaptive-effort/structured-output/cache-control bridge compatibility, stateless Gemini thought-signature tool round trips, and generic Chat replay handling for unsigned private thinking blocks.
|
||||
- 2026-08-09: Extended `output_config.effort` to accept `low`, `medium`, `high`, `xhigh`, and `max` across Anthropic native and Chat bridge routes without substitution or normalization. Unknown effort values remain `400 invalid_request_error` before provider dispatch. Deterministic Go coverage added for exact bridge mapping, native `max` preservation, and invalid-value rejection. (`apps/edge/internal/openai/anthropic_types.go`, `apps/edge/internal/openai/anthropic_bridge_test.go`, `apps/edge/internal/openai/anthropic_native_test.go`)
|
||||
- 2026-08-13: Added caller-neutral provider operation normalization for Messages/Responses routes. Tool-bearing adaptive effort can select Responses when Chat cannot preserve the combination, and unsupported effort grades fall only to the nearest declared lower grade (for example `max` to `xhigh`).
|
||||
- 2026-08-06: Synchronized always-owned Chat/Responses typed-stall recovery, provider avoidance/fallback admission, and closed-label liveness operational evidence with the current runtime, contracts, and deterministic recovery tests.
|
||||
- 2026-08-06: Added marked single-request Messages admission through the separate service coordinator capability, one unlabeled runtime ingress counter, buffered sanitized terminal acknowledgement, and deterministic real-POST compatibility evidence.
|
||||
- 2026-08-06: Added the marked streaming subset with fixed plan/work/review/repair progress, liveness ping, serialized monotonic text blocks, private-wire exclusion, one success/error terminal, joined ticker shutdown, and post-`message_stop` completion acknowledgement.
|
||||
|
|
|
|||
|
|
@ -50,7 +50,10 @@ source_evidence:
|
|||
notes: lease state와 candidate pressure 기반 online/offline provider snapshot
|
||||
- type: code
|
||||
path: packages/go/config/protocol_profile.go
|
||||
notes: ConcreteProtocolProfile, ProtocolOperation, ProtocolDriver, overlay validation, alias normalization, capability admission, model mapping
|
||||
notes: ConcreteProtocolProfile, overlay validation, capability admission, model/operation/effort normalization
|
||||
- type: test
|
||||
path: packages/go/config/protocol_profile_test.go
|
||||
notes: Effort exact/nearest-lower mapping, tools 조합, 상향 매핑 거부 검증
|
||||
- type: code
|
||||
path: apps/edge/internal/configrefresh/classify.go
|
||||
notes: dry-run/apply classification과 changed path report 생성
|
||||
|
|
@ -124,6 +127,7 @@ Edge 설정에서 provider-pool이 어떻게 모델 실행 후보를 고르고,
|
|||
| provider snapshot | 일반·long in-flight는 provider lease state, queued 값은 Edge queue에서 해당 provider를 후보로 포함하는 고유 pending request pressure에서 계산한다. offline provider는 catalog identity를 유지하고 effective 수치를 0으로 보고한다. |
|
||||
| runtime provider health overlay | A confirmed current bound unavailable stall lowers a separate `(node_id, connection_generation, provider_id)` overlay. The provider is excluded from effective admission and its snapshot projects unavailable with zero effective capacity/counters, while configured health remains unchanged. Only a later exact higher-sequence available CAPABILITIES probe recovers it; inconclusive evidence is a no-op. Post-decision metrics/logs expose only closed source, health, decision, and state-change values; they contain no resource identity or raw request/response data. |
|
||||
| mixed provider execution path | 같은 model group의 OpenAI-compatible provider와 Ollama/native provider를 같은 후보군으로 두며, 선택된 provider capability로 passthrough 또는 normalized 실행 경로를 결정한다. OpenAI-compatible provider는 `openai_chat`, `anthropic_messages`, 또는 `openai_responses` driver로 해석된다. |
|
||||
| provider operation normalization | `protocol_profiles[].normalization.effort` records operation-scoped provider wire, supported normalized grades, tools compatibility, and explicit token-budget compatibility. Request admission uses these facts rather than caller identity; exact grade misses use only the nearest lower declared grade and never upgrade. |
|
||||
| long-context admission | estimated input token이 threshold 이상이면 `context_class=long`으로 분류하고, provider long slot이 있으면 일반 capacity slot과 함께 점유한다. |
|
||||
| config refresh dry-run/apply | loopback admin HTTP `POST /refresh`가 candidate config를 dry-run 또는 apply한다. |
|
||||
| refresh classification | listener, Edge identity, bootstrap path, adapter structural 변경 등은 restart-required로 분류한다. |
|
||||
|
|
@ -186,7 +190,8 @@ sequenceDiagram
|
|||
- `long_context_threshold_tokens` 기본 예시는 `100000`이고 0 이하 값은 config load에서 거부된다.
|
||||
- `credential_plane.enabled=true` requires Edge-Node server TLS, an enabled TLS Control Plane connector, and OpenAI ingress TLS when enabled. The Edge cannot combine managed mode with `openai.bearer_token`, `openai.principal_tokens[]`, `openai.provider_auth`, or static provider credential sources.
|
||||
- Node managed mode requires Edge transport TLS, `recipient_key_id`/recipient private-key path, issuer key id/public-key path, and a bounded replay cache. All cert/key/keyring values are external file references and credential-plane changes are restart-required.
|
||||
- `protocol_profiles` is the top-level catalog of custom overlays. A `ProtocolProfileConf` supplies `base`, `driver`, `base_url`, operation paths, `auth`, `capabilities`, `model_mapping`, and `extensions`; `base` inheritance is separate from legacy provider-type normalization.
|
||||
- `protocol_profiles` is the top-level catalog of custom overlays. A `ProtocolProfileConf` supplies `base`, `driver`, `base_url`, operation paths, `auth`, `capabilities`, `model_mapping`, `normalization`, and `extensions`; `base` inheritance is separate from legacy provider-type normalization.
|
||||
- `normalization.effort[operation]` must reference a declared operation and a recognized wire. Grade keys are the closed `none|low|medium|high|xhigh|max` order. Runtime mapping takes the exact key or nearest lower key; canonical mapped values cannot exceed the source grade. `with_tools` and `token_budget` describe whether that operation preserves the corresponding semantic combination.
|
||||
- `nodes[].providers[].profile` selects a catalog entry. Config normalization resolves that selection (or a legacy type alias) into the runtime-only `RuntimeProfile` snapshot; the source YAML remains a selector plus catalog, not a per-model overlay.
|
||||
- `nodes[].providers[].response_stall_timeout_ms` is validated at config load: zero/omitted resolves to `60000ms`; safe positive values are retained; negative and duration-overflow values are rejected. Its effective value is immutable for the selected provider attempt and survives queue re-resolution for both execution paths.
|
||||
- Profile catalog and provider-selector changes are restart-required. Snapshot immutability describes loaded runtime state and does not make those changes live-applicable.
|
||||
|
|
@ -258,5 +263,6 @@ sequenceDiagram
|
|||
- 2026-08-05: Added the separate generation-scoped runtime provider health overlay, effective admission/snapshot exclusion, config-health immutability, and exact higher-sequence CAPABILITIES recovery.
|
||||
- 2026-08-05: Added post-decision provider-health operational evidence with bounded counters and structured logs, isolated from overlay state and provider identity.
|
||||
- 2026-08-09: Synchronized the single-request effective-template boundary: relative-only `plan_file`/`review_file` resolution against the `edge.yaml` directory with pre-access absolute rejection, independent per-file default fallback, load-time regular-file/size/UTF-8/grammar rejection, digest-only refresh diff evidence, and admission-time freezing so a refresh reaches only newly admitted requests (`packages/go/config/load.go`, `apps/edge/internal/configrefresh/classify.go`, `apps/edge/internal/openai/single_request_preset_binding.go`).
|
||||
- 2026-08-13: Added operation-scoped provider effort normalization facts and nearest-lower, never-upgrade mapping semantics to the protocol profile catalog.
|
||||
- 2026-08-06: Synchronized the fixed single-request policy (`execution_presets[].single_request`) absolute caps, plan→work→review stage shape, opaque `workspace_ref`, live-apply classification, and snapshot-isolation semantics with current code, contract, and classifier implementation.
|
||||
- 2026-08-06: Required effective positive workspace-operation bounds and clarified that the later Node-private typed config/admission transport is deferred; public/preset/provider surfaces retain no raw workspace roots or command templates.
|
||||
|
|
|
|||
|
|
@ -63,6 +63,13 @@ func prepareAnthropicChatBridge(body []byte, target string, profile config.Concr
|
|||
if err != nil {
|
||||
return nil, req, err
|
||||
}
|
||||
plan, err := selectProviderOperation(profile, config.OperationMessages, anthropicProviderRequirements(req))
|
||||
if err != nil {
|
||||
return nil, req, err
|
||||
}
|
||||
if plan.Operation != config.OperationChatCompletions || (plan.Effort != "" && plan.EffortWire != config.ProtocolEffortWireOpenAIChat && plan.EffortWire != config.ProtocolEffortWireGeminiChat) {
|
||||
return nil, req, fmt.Errorf("selected profile does not support the Chat bridge request controls")
|
||||
}
|
||||
if req.TopK != nil {
|
||||
return nil, req, fmt.Errorf("top_k is not supported by the Chat bridge")
|
||||
}
|
||||
|
|
@ -101,10 +108,10 @@ func prepareAnthropicChatBridge(body []byte, target string, profile config.Concr
|
|||
"stream": req.Stream,
|
||||
}
|
||||
maxTokensField := "max_tokens"
|
||||
if profile.ID == "openai" {
|
||||
if mapping, ok := profile.EffortMapping(config.OperationChatCompletions); ok && mapping.Wire == config.ProtocolEffortWireOpenAIChat {
|
||||
// OpenAI's current Chat completion models use the completion-specific
|
||||
// field. Other OpenAI-compatible profiles retain their native legacy
|
||||
// spelling instead of inheriting an OpenAI-only request contract.
|
||||
// field. Profiles on other Chat-compatible wires retain their native
|
||||
// legacy spelling instead of inheriting this request contract.
|
||||
maxTokensField = "max_completion_tokens"
|
||||
}
|
||||
chat[maxTokensField] = *req.MaxTokens
|
||||
|
|
@ -154,8 +161,8 @@ func prepareAnthropicChatBridge(body []byte, target string, profile config.Concr
|
|||
chat["thinking_token_budget"] = req.Thinking.BudgetTokens
|
||||
}
|
||||
if req.OutputConfig != nil {
|
||||
if req.OutputConfig.Effort != "" {
|
||||
chat["reasoning_effort"] = req.OutputConfig.Effort
|
||||
if plan.Effort != "" {
|
||||
chat["reasoning_effort"] = plan.Effort
|
||||
}
|
||||
if req.OutputConfig.Format != nil {
|
||||
var schema map[string]any
|
||||
|
|
|
|||
|
|
@ -197,16 +197,17 @@ func TestAnthropicChatBridgeDropsUnsignedThinkingReplayForGenericProfile(t *test
|
|||
|
||||
func TestAnthropicChatBridgeRejectsUnsupportedBeforeWire(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
body string
|
||||
beta string
|
||||
name string
|
||||
body string
|
||||
beta string
|
||||
errorType string
|
||||
}{
|
||||
{name: "top k", body: `{"model":"claude-route","max_tokens":16,"top_k":4,"messages":[{"role":"user","content":"hello"}]}`},
|
||||
{name: "unknown block", body: `{"model":"claude-route","max_tokens":16,"messages":[{"role":"user","content":[{"type":"search_result","content":"unknown"}]}]}`},
|
||||
{name: "unknown field", body: `{"model":"claude-route","max_tokens":16,"vendor_extension":true,"messages":[{"role":"user","content":"hello"}]}`},
|
||||
{name: "context management scalar", body: `{"model":"claude-route","max_tokens":16,"context_management":"compact","messages":[{"role":"user","content":"hello"}]}`, beta: "context-management-2025-06-27"},
|
||||
{name: "context management array", body: `{"model":"claude-route","max_tokens":16,"context_management":[],"messages":[{"role":"user","content":"hello"}]}`, beta: "context-management-2025-06-27"},
|
||||
{name: "thinking capability", body: `{"model":"claude-route","max_tokens":16,"thinking":{"type":"enabled","budget_tokens":8},"messages":[{"role":"user","content":"hello"}]}`},
|
||||
{name: "thinking capability", body: `{"model":"claude-route","max_tokens":16,"thinking":{"type":"enabled","budget_tokens":8},"messages":[{"role":"user","content":"hello"}]}`, errorType: "not_supported_error"},
|
||||
{name: "tool strict", body: `{"model":"claude-route","max_tokens":16,"messages":[{"role":"user","content":"hello"}],"tools":[{"name":"Read","input_schema":{"type":"object"},"strict":true}]}`, beta: "advanced-tool-use-2025-11-20"},
|
||||
{name: "tool eager input streaming", body: `{"model":"claude-route","max_tokens":16,"messages":[{"role":"user","content":"hello"}],"tools":[{"name":"Read","input_schema":{"type":"object"},"eager_input_streaming":true}]}`, beta: "advanced-tool-use-2025-11-20"},
|
||||
{name: "thinking display value", body: `{"model":"claude-route","max_tokens":16,"thinking":{"type":"adaptive","display":"raw"},"messages":[{"role":"user","content":"hello"}]}`},
|
||||
|
|
@ -228,7 +229,11 @@ func TestAnthropicChatBridgeRejectsUnsupportedBeforeWire(t *testing.T) {
|
|||
req.Header.Set(anthropicBetaHeader, tc.beta)
|
||||
}
|
||||
w := serveAnthropicHTTPRequest(srv, req)
|
||||
if w.Code != http.StatusBadRequest || !strings.Contains(w.Body.String(), `"type":"invalid_request_error"`) {
|
||||
errorType := tc.errorType
|
||||
if errorType == "" {
|
||||
errorType = "invalid_request_error"
|
||||
}
|
||||
if w.Code != http.StatusBadRequest || !strings.Contains(w.Body.String(), `"type":"`+errorType+`"`) {
|
||||
t.Fatalf("status=%d body=%s", w.Code, w.Body.String())
|
||||
}
|
||||
if got := len(fake.tunnelReqsSnapshot()); got != 0 {
|
||||
|
|
@ -341,6 +346,94 @@ func TestAnthropicChatBridgeEffortExactTokenPreservation(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
func TestAnthropicToolsAndEffortSelectResponsesWithLowerFallback(t *testing.T) {
|
||||
candidate := anthropicTestCandidate(t, "openai")
|
||||
candidate.ActualModel = "served-responses"
|
||||
responsesMapping := candidate.ProtocolProfile.Normalization.Effort[string(config.OperationResponses)]
|
||||
delete(responsesMapping.Levels, "max")
|
||||
candidate.ProtocolProfile.Normalization.Effort[string(config.OperationResponses)] = responsesMapping
|
||||
|
||||
providerResponse := []byte(`{"id":"resp_effort","model":"served-responses","status":"completed","output":[{"type":"function_call","id":"fc_1","call_id":"call_1","name":"write_file","arguments":"{\"path\":\"index.html\"}"}],"usage":{"input_tokens":11,"output_tokens":3,"input_tokens_details":{"cached_tokens":2}}}`)
|
||||
fake := &providerFakeRunService{
|
||||
poolDispatchPath: string(edgeservice.ProviderPoolPathTunnel),
|
||||
poolSelectedCandidate: candidate,
|
||||
tunnelFrames: anthropicTunnelFrames(http.StatusOK, "application/json", providerResponse),
|
||||
}
|
||||
srv := NewServer(config.EdgeOpenAIConf{}, fake, nil)
|
||||
srv.SetModelCatalog([]config.ModelCatalogEntry{{ID: "gpt-route", Providers: map[string]string{"openai": "served-responses"}}})
|
||||
body := `{"model":"gpt-route","max_tokens":256,"output_config":{"effort":"max"},"tools":[{"name":"write_file","description":"write a file","input_schema":{"type":"object","properties":{"path":{"type":"string"}},"required":["path"]}}],"messages":[{"role":"user","content":"create index.html"}]}`
|
||||
w := serveAnthropicRequest(srv, "/v1/messages", body)
|
||||
|
||||
if w.Code != http.StatusOK {
|
||||
t.Fatalf("status=%d body=%s", w.Code, w.Body.String())
|
||||
}
|
||||
requests := fake.tunnelReqsSnapshot()
|
||||
if len(requests) != 1 {
|
||||
t.Fatalf("tunnel requests=%d, want 1", len(requests))
|
||||
}
|
||||
if requests[0].Operation != string(config.OperationResponses) || requests[0].Path != "/v1/responses" {
|
||||
t.Fatalf("operation/path=%q/%q, want responses//v1/responses", requests[0].Operation, requests[0].Path)
|
||||
}
|
||||
var upstream map[string]any
|
||||
if err := json.Unmarshal(fake.tunnelBodiesSnapshot()[0], &upstream); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
reasoning, _ := upstream["reasoning"].(map[string]any)
|
||||
if reasoning["effort"] != "xhigh" {
|
||||
t.Fatalf("reasoning effort=%v, want xhigh fallback", reasoning["effort"])
|
||||
}
|
||||
if _, hasChatMessages := upstream["messages"]; hasChatMessages {
|
||||
t.Fatalf("Responses bridge emitted Chat messages: %+v", upstream)
|
||||
}
|
||||
if _, hasResponsesInput := upstream["input"]; !hasResponsesInput {
|
||||
t.Fatalf("Responses bridge omitted input: %+v", upstream)
|
||||
}
|
||||
var response anthropicMessageResponse
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &response); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(response.Content) != 1 || response.Content[0]["type"] != "tool_use" || response.Content[0]["name"] != "write_file" {
|
||||
t.Fatalf("Anthropic tool response=%+v", response.Content)
|
||||
}
|
||||
if response.StopReason == nil || *response.StopReason != "tool_use" {
|
||||
t.Fatalf("stop_reason=%v, want tool_use", response.StopReason)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnthropicStreamingToolsAndEffortUseResponsesBridge(t *testing.T) {
|
||||
candidate := anthropicTestCandidate(t, "openai")
|
||||
candidate.ActualModel = "served-responses"
|
||||
providerResponse := []byte("event: response.completed\n" +
|
||||
`data: {"type":"response.completed","response":{"id":"resp_stream","model":"served-responses","status":"completed","output":[{"type":"function_call","id":"fc_1","call_id":"call_1","name":"write_file","arguments":"{\"path\":\"index.html\"}"}],"usage":{"input_tokens":11,"output_tokens":3}}}` + "\n\n" +
|
||||
"data: [DONE]\n\n")
|
||||
fake := &providerFakeRunService{
|
||||
poolDispatchPath: string(edgeservice.ProviderPoolPathTunnel),
|
||||
poolSelectedCandidate: candidate,
|
||||
tunnelFrames: anthropicTunnelFrames(http.StatusOK, "text/event-stream", providerResponse),
|
||||
}
|
||||
srv := NewServer(config.EdgeOpenAIConf{}, fake, nil)
|
||||
srv.SetModelCatalog([]config.ModelCatalogEntry{{ID: "gpt-route", Providers: map[string]string{"openai": "served-responses"}}})
|
||||
body := `{"model":"gpt-route","max_tokens":256,"stream":true,"output_config":{"effort":"high"},"tools":[{"name":"write_file","input_schema":{"type":"object"}}],"messages":[{"role":"user","content":"create index.html"}]}`
|
||||
w := serveAnthropicRequest(srv, "/v1/messages", body)
|
||||
|
||||
if w.Code != http.StatusOK || w.Header().Get("Content-Type") != "text/event-stream" {
|
||||
t.Fatalf("status=%d content-type=%q body=%s", w.Code, w.Header().Get("Content-Type"), w.Body.String())
|
||||
}
|
||||
requests := fake.tunnelReqsSnapshot()
|
||||
if len(requests) != 1 || requests[0].Operation != string(config.OperationResponses) || !requests[0].Stream {
|
||||
t.Fatalf("Responses stream dispatch mismatch: %+v", requests)
|
||||
}
|
||||
output := w.Body.String()
|
||||
for _, want := range []string{"event: message_start", `"type":"tool_use"`, `"name":"write_file"`, "event: message_stop"} {
|
||||
if !strings.Contains(output, want) {
|
||||
t.Fatalf("Anthropic stream missing %q: %s", want, output)
|
||||
}
|
||||
}
|
||||
if strings.Contains(output, "response.completed") || strings.Count(output, "event: message_stop") != 1 {
|
||||
t.Fatalf("provider event leaked or terminal count mismatched: %s", output)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnthropicChatBridgeEffortRejectsInvalidValue(t *testing.T) {
|
||||
for _, effort := range []string{"HIGH", "XHigh", "maxx", "xhighx", "h"} {
|
||||
t.Run(effort, func(t *testing.T) {
|
||||
|
|
|
|||
|
|
@ -247,8 +247,12 @@ func (s *Server) handleAnthropicMessages(w http.ResponseWriter, r *http.Request)
|
|||
return
|
||||
}
|
||||
|
||||
needsTools := anthropicRequestNeedsTools(body)
|
||||
poolReq, presetIngress, err := s.anthropicPoolRequest(r, dispatch, envelope, body, config.OperationMessages, needsTools)
|
||||
requirements, err := decodeAnthropicProviderRequirements(body)
|
||||
if err != nil {
|
||||
s.writeAnthropicPreIngressError(w, http.StatusBadRequest, "invalid_request_error", err.Error(), anthropicPreIngressInvalidEnvelope)
|
||||
return
|
||||
}
|
||||
poolReq, presetIngress, err := s.anthropicPoolRequest(r, dispatch, envelope, body, config.OperationMessages, requirements)
|
||||
if err != nil {
|
||||
writeAnthropicError(w, http.StatusBadRequest, "invalid_request_error", err.Error())
|
||||
return
|
||||
|
|
@ -311,15 +315,28 @@ func (s *Server) handleAnthropicMessages(w http.ResponseWriter, r *http.Request)
|
|||
}
|
||||
defer result.Tunnel.Close()
|
||||
|
||||
switch result.DispatchInfo.ProfileDriver {
|
||||
case string(config.ProtocolDriverAnthropicMessages):
|
||||
profileOperation := result.DispatchInfo.ProfileOperation
|
||||
if profileOperation == "" {
|
||||
switch result.DispatchInfo.ProfileDriver {
|
||||
case string(config.ProtocolDriverAnthropicMessages):
|
||||
profileOperation = string(config.OperationMessages)
|
||||
case string(config.ProtocolDriverOpenAIChat):
|
||||
profileOperation = string(config.OperationChatCompletions)
|
||||
case string(config.ProtocolDriverOpenAIResponses):
|
||||
profileOperation = string(config.OperationResponses)
|
||||
}
|
||||
}
|
||||
switch profileOperation {
|
||||
case string(config.OperationMessages):
|
||||
publicModelID := ""
|
||||
if dispatch.IsPreset {
|
||||
publicModelID = dispatch.ExternalModelID
|
||||
}
|
||||
s.writeAnthropicNativeTunnelResponse(w, r, result.Tunnel, publicModelID)
|
||||
case string(config.ProtocolDriverOpenAIChat):
|
||||
case string(config.OperationChatCompletions):
|
||||
s.writeAnthropicChatBridgeResponse(w, r, result.Tunnel, envelope)
|
||||
case string(config.OperationResponses):
|
||||
s.writeAnthropicResponsesBridgeResponse(w, r, result.Tunnel, envelope)
|
||||
default:
|
||||
writeAnthropicError(w, http.StatusBadGateway, "api_error", "selected provider returned an unsupported protocol driver")
|
||||
}
|
||||
|
|
@ -547,7 +564,7 @@ func (s *Server) handleAnthropicCountTokens(w http.ResponseWriter, r *http.Reque
|
|||
return
|
||||
}
|
||||
|
||||
poolReq, _, err := s.anthropicPoolRequest(r, dispatch, envelope, body, config.OperationCountTokens, false)
|
||||
poolReq, _, err := s.anthropicPoolRequest(r, dispatch, envelope, body, config.OperationCountTokens, providerRequestRequirements{})
|
||||
if err != nil {
|
||||
writeAnthropicError(w, http.StatusBadRequest, "invalid_request_error", err.Error())
|
||||
return
|
||||
|
|
@ -572,7 +589,7 @@ func (s *Server) anthropicPoolRequest(
|
|||
envelope anthropicRequestEnvelope,
|
||||
body []byte,
|
||||
operation config.ProtocolOperation,
|
||||
needsTools bool,
|
||||
requirements providerRequestRequirements,
|
||||
) (edgeservice.ProviderPoolDispatchRequest, presetIngressResult, error) {
|
||||
metadata := principalMetadata(r.Context())
|
||||
if metadata == nil {
|
||||
|
|
@ -593,9 +610,9 @@ func (s *Server) anthropicPoolRequest(
|
|||
}
|
||||
// Resume-selector and ordinary continuations both construct the same
|
||||
// trusted selector request; only local eligibility bypasses the pool.
|
||||
return s.buildAnthropicPoolRequest(r, dispatch, envelope, body, operation, needsTools, metadata, presetIngress)
|
||||
return s.buildAnthropicPoolRequest(r, dispatch, envelope, body, operation, requirements, metadata, presetIngress)
|
||||
}
|
||||
return s.buildAnthropicPoolRequest(r, dispatch, envelope, body, operation, needsTools, metadata, presetIngressResult{})
|
||||
return s.buildAnthropicPoolRequest(r, dispatch, envelope, body, operation, requirements, metadata, presetIngressResult{})
|
||||
}
|
||||
|
||||
func (s *Server) buildAnthropicPoolRequest(
|
||||
|
|
@ -604,7 +621,7 @@ func (s *Server) buildAnthropicPoolRequest(
|
|||
envelope anthropicRequestEnvelope,
|
||||
body []byte,
|
||||
operation config.ProtocolOperation,
|
||||
needsTools bool,
|
||||
requirements providerRequestRequirements,
|
||||
metadata map[string]string,
|
||||
presetIngress presetIngressResult,
|
||||
) (edgeservice.ProviderPoolDispatchRequest, presetIngressResult, error) {
|
||||
|
|
@ -634,7 +651,7 @@ func (s *Server) buildAnthropicPoolRequest(
|
|||
EstimatedInputTokens: estimate, ContextClass: contextClass, ProviderPool: true,
|
||||
},
|
||||
}
|
||||
poolReq.AcceptCandidate = anthropicCandidatePredicate(operation, envelope.Stream, needsTools)
|
||||
poolReq.AcceptCandidate = anthropicCandidatePredicate(operation, requirements)
|
||||
if dispatch.Managed {
|
||||
poolReq.AcceptCandidate = composeCandidatePredicates(poolReq.AcceptCandidate, dispatch.CandidatePredicate())
|
||||
}
|
||||
|
|
@ -643,18 +660,22 @@ func (s *Server) buildAnthropicPoolRequest(
|
|||
return tunnelReq, newAnthropicClientError("not_supported_error", fmt.Errorf("selected provider has no concrete protocol profile"))
|
||||
}
|
||||
profile := selected.ProtocolProfile.Clone()
|
||||
plan, err := selectProviderOperation(profile, operation, requirements)
|
||||
if err != nil {
|
||||
return tunnelReq, newAnthropicClientError("not_supported_error", err)
|
||||
}
|
||||
headers, err := s.anthropicUpstreamHeaders(r, profile, profile.Driver == config.ProtocolDriverAnthropicMessages)
|
||||
if err != nil {
|
||||
return tunnelReq, newAnthropicClientError("invalid_request_error", err)
|
||||
}
|
||||
tunnelReq.Headers = headers
|
||||
switch profile.Driver {
|
||||
case config.ProtocolDriverAnthropicMessages:
|
||||
tunnelReq.Operation = string(operation)
|
||||
switch plan.Operation {
|
||||
case config.OperationMessages, config.OperationCountTokens:
|
||||
tunnelReq.Operation = string(plan.Operation)
|
||||
tunnelReq.BuildBody = func(target string) ([]byte, error) {
|
||||
return rewriteResponsesModel(body, target)
|
||||
}
|
||||
case config.ProtocolDriverOpenAIChat:
|
||||
case config.OperationChatCompletions:
|
||||
if operation != config.OperationMessages {
|
||||
return tunnelReq, newAnthropicClientError("not_supported_error", fmt.Errorf("selected Chat profile has no native count-tokens operation"))
|
||||
}
|
||||
|
|
@ -668,6 +689,21 @@ func (s *Server) buildAnthropicPoolRequest(
|
|||
tunnelReq.Operation = string(config.OperationChatCompletions)
|
||||
tunnelReq.Body = bridged
|
||||
tunnelReq.BuildBody = nil
|
||||
case config.OperationResponses:
|
||||
if operation != config.OperationMessages {
|
||||
return tunnelReq, newAnthropicClientError("not_supported_error", fmt.Errorf("selected Responses profile has no native count-tokens operation"))
|
||||
}
|
||||
if err := validateAnthropicHeaders(r); err != nil {
|
||||
return tunnelReq, newAnthropicClientError("invalid_request_error", err)
|
||||
}
|
||||
bridged, _, err := prepareAnthropicResponsesBridge(body, selected.ActualModel, profile, plan)
|
||||
if err != nil {
|
||||
return tunnelReq, newAnthropicClientError("invalid_request_error", err)
|
||||
}
|
||||
tunnelReq.Operation = string(config.OperationResponses)
|
||||
tunnelReq.Path = "/v1/responses"
|
||||
tunnelReq.Body = bridged
|
||||
tunnelReq.BuildBody = nil
|
||||
default:
|
||||
return tunnelReq, newAnthropicClientError("not_supported_error", fmt.Errorf("unsupported protocol driver %q", profile.Driver))
|
||||
}
|
||||
|
|
@ -676,28 +712,19 @@ func (s *Server) buildAnthropicPoolRequest(
|
|||
return poolReq, presetIngress, nil
|
||||
}
|
||||
|
||||
func anthropicCandidatePredicate(operation config.ProtocolOperation, stream, needsTools bool) edgeservice.ProviderPoolCandidatePredicate {
|
||||
func anthropicCandidatePredicate(operation config.ProtocolOperation, requirements providerRequestRequirements) edgeservice.ProviderPoolCandidatePredicate {
|
||||
return func(candidate edgeservice.ProviderPoolCandidate) bool {
|
||||
profile := candidate.ProtocolProfile
|
||||
if profile == nil || candidate.ExecutionPath != string(edgeservice.ProviderPoolPathTunnel) {
|
||||
return false
|
||||
}
|
||||
if stream && !profile.HasCapability("streaming") {
|
||||
return false
|
||||
}
|
||||
if needsTools && !profile.HasCapability("tool_calling") {
|
||||
return false
|
||||
}
|
||||
switch operation {
|
||||
case config.OperationCountTokens:
|
||||
return profile.Driver == config.ProtocolDriverAnthropicMessages &&
|
||||
profile.HasCapability("count_tokens") && profileHasOperation(*profile, config.OperationCountTokens)
|
||||
case config.OperationMessages:
|
||||
if profile.Driver == config.ProtocolDriverAnthropicMessages {
|
||||
return profile.HasCapability("messages") && profileHasOperation(*profile, config.OperationMessages)
|
||||
}
|
||||
return profile.Driver == config.ProtocolDriverOpenAIChat &&
|
||||
profile.HasCapability("chat") && profileHasOperation(*profile, config.OperationChatCompletions)
|
||||
_, err := selectProviderOperation(*profile, operation, requirements)
|
||||
return err == nil
|
||||
default:
|
||||
return false
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1203,3 +1203,140 @@ func (s *Server) writeAnthropicChatBridgeResponse(w http.ResponseWriter, r *http
|
|||
}
|
||||
}
|
||||
}
|
||||
|
||||
// writeAnthropicResponsesBridgeResponse translates an OpenAI Responses
|
||||
// provider result back to the Messages surface. Streaming input is currently
|
||||
// buffered until the Responses terminal so the caller still receives one
|
||||
// valid Anthropic SSE lifecycle without exposing provider-specific events.
|
||||
func (s *Server) writeAnthropicResponsesBridgeResponse(w http.ResponseWriter, r *http.Request, handle edgeservice.ProviderTunnelResult, envelope anthropicRequestEnvelope) {
|
||||
frames := handle.Stream().Frames
|
||||
if frames == nil {
|
||||
writeAnthropicError(w, http.StatusBadGateway, "api_error", "provider tunnel is unavailable")
|
||||
return
|
||||
}
|
||||
timer := time.NewTimer(handle.WaitTimeout())
|
||||
defer timer.Stop()
|
||||
status := http.StatusOK
|
||||
headers := make(map[string]string)
|
||||
var body []byte
|
||||
for {
|
||||
select {
|
||||
case <-r.Context().Done():
|
||||
s.cancelRunOnHTTPGiveUp(handle.Dispatch(), r.Context().Err())
|
||||
return
|
||||
case <-timer.C:
|
||||
s.cancelRunOnHTTPGiveUp(handle.Dispatch(), errRunTimedOut)
|
||||
writeAnthropicError(w, http.StatusBadGateway, "api_error", "provider response timed out")
|
||||
return
|
||||
case frame, ok := <-frames:
|
||||
if !ok {
|
||||
writeAnthropicError(w, http.StatusBadGateway, "api_error", "provider tunnel closed before a response")
|
||||
return
|
||||
}
|
||||
switch frame.GetKind() {
|
||||
case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_RESPONSE_START:
|
||||
status = int(frame.GetStatusCode())
|
||||
if status == 0 {
|
||||
status = http.StatusOK
|
||||
}
|
||||
for key, value := range frame.GetHeaders() {
|
||||
headers[key] = value
|
||||
}
|
||||
case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY:
|
||||
body = append(body, frame.GetBody()...)
|
||||
case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_ERROR:
|
||||
writeAnthropicError(w, http.StatusBadGateway, "api_error", "provider tunnel failed")
|
||||
return
|
||||
case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_END:
|
||||
copyAnthropicResponseHeaders(w.Header(), headers)
|
||||
w.Header().Del("Content-Length")
|
||||
if status >= http.StatusBadRequest {
|
||||
writeAnthropicError(w, status, "api_error", "upstream provider rejected the request")
|
||||
return
|
||||
}
|
||||
responseBody := body
|
||||
if envelope.Stream {
|
||||
var terminal struct {
|
||||
Type string `json:"type"`
|
||||
Response json.RawMessage `json:"response"`
|
||||
}
|
||||
for _, event := range splitOpenAIResponsesSSE(body) {
|
||||
if json.Unmarshal(event, &terminal) == nil && terminal.Type == "response.completed" && len(terminal.Response) > 0 {
|
||||
responseBody = terminal.Response
|
||||
}
|
||||
}
|
||||
}
|
||||
converted, err := convertResponsesResponseToAnthropic(responseBody, envelope.Model)
|
||||
if err != nil {
|
||||
writeAnthropicError(w, http.StatusBadGateway, "api_error", "upstream response could not be translated")
|
||||
return
|
||||
}
|
||||
if !envelope.Stream {
|
||||
writeJSON(w, http.StatusOK, converted)
|
||||
return
|
||||
}
|
||||
writeBufferedAnthropicMessageStream(w, converted)
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func splitOpenAIResponsesSSE(body []byte) [][]byte {
|
||||
var payloads [][]byte
|
||||
normalized := bytes.ReplaceAll(body, []byte("\r\n"), []byte("\n"))
|
||||
for _, event := range bytes.Split(normalized, []byte("\n\n")) {
|
||||
var data [][]byte
|
||||
for _, line := range bytes.Split(event, []byte("\n")) {
|
||||
line = bytes.TrimSpace(line)
|
||||
if bytes.HasPrefix(line, []byte("data:")) {
|
||||
part := bytes.TrimSpace(bytes.TrimPrefix(line, []byte("data:")))
|
||||
if !bytes.Equal(part, []byte("[DONE]")) {
|
||||
data = append(data, part)
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(data) > 0 {
|
||||
payloads = append(payloads, bytes.Join(data, []byte("\n")))
|
||||
}
|
||||
}
|
||||
return payloads
|
||||
}
|
||||
|
||||
func writeBufferedAnthropicMessageStream(w http.ResponseWriter, message anthropicMessageResponse) {
|
||||
w.Header().Set("Content-Type", "text/event-stream")
|
||||
w.Header().Set("Cache-Control", "no-cache")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
start := message
|
||||
start.Content = []map[string]any{}
|
||||
start.StopReason = nil
|
||||
start.Usage.OutputTokens = 0
|
||||
_ = writeAnthropicSSEEvent(w, "message_start", map[string]any{"type": "message_start", "message": start})
|
||||
for index, block := range message.Content {
|
||||
blockType, _ := block["type"].(string)
|
||||
opening := map[string]any{"type": blockType}
|
||||
var delta map[string]any
|
||||
switch blockType {
|
||||
case "text":
|
||||
opening["text"] = ""
|
||||
delta = map[string]any{"type": "text_delta", "text": block["text"]}
|
||||
case "thinking":
|
||||
opening["thinking"], opening["signature"] = "", ""
|
||||
delta = map[string]any{"type": "thinking_delta", "thinking": block["thinking"]}
|
||||
case "tool_use":
|
||||
opening["id"], opening["name"], opening["input"] = block["id"], block["name"], map[string]any{}
|
||||
encodedInput, _ := json.Marshal(block["input"])
|
||||
delta = map[string]any{"type": "input_json_delta", "partial_json": string(encodedInput)}
|
||||
default:
|
||||
continue
|
||||
}
|
||||
_ = writeAnthropicSSEEvent(w, "content_block_start", map[string]any{"type": "content_block_start", "index": index, "content_block": opening})
|
||||
_ = writeAnthropicSSEEvent(w, "content_block_delta", map[string]any{"type": "content_block_delta", "index": index, "delta": delta})
|
||||
_ = writeAnthropicSSEEvent(w, "content_block_stop", map[string]any{"type": "content_block_stop", "index": index})
|
||||
}
|
||||
_ = writeAnthropicSSEEvent(w, "message_delta", map[string]any{
|
||||
"type": "message_delta", "delta": map[string]any{"stop_reason": message.StopReason, "stop_sequence": nil},
|
||||
"usage": message.Usage,
|
||||
})
|
||||
_ = writeAnthropicSSEEvent(w, "message_stop", map[string]any{"type": "message_stop"})
|
||||
}
|
||||
|
|
|
|||
|
|
@ -15,8 +15,8 @@ func validateThinkControl(req *chatCompletionRequest) error {
|
|||
if eff == "" {
|
||||
return fmt.Errorf("reasoning_effort cannot be empty when present")
|
||||
}
|
||||
if eff != "none" && eff != "low" && eff != "medium" && eff != "high" {
|
||||
return fmt.Errorf("reasoning_effort must be one of none, low, medium, or high")
|
||||
if eff != "none" && eff != "low" && eff != "medium" && eff != "high" && eff != "xhigh" && eff != "max" {
|
||||
return fmt.Errorf("reasoning_effort must be one of none, low, medium, high, xhigh, or max")
|
||||
}
|
||||
}
|
||||
if req.Think != nil && !*req.Think {
|
||||
|
|
|
|||
|
|
@ -110,6 +110,7 @@ func presetSelectorAdmission(
|
|||
NodeID: selected.NodeID,
|
||||
ExecutionPath: selected.ExecutionPath,
|
||||
ProfileDriver: selected.ProfileDriver,
|
||||
ProfileOperation: selected.ProfileOperation,
|
||||
ProfileCapabilities: append([]string(nil), selected.ProfileCapabilities...),
|
||||
}
|
||||
expectedGroup := presetSelectorModelGroupKey(dispatch, dispatch.ExternalModelID)
|
||||
|
|
@ -118,7 +119,7 @@ func presetSelectorAdmission(
|
|||
strings.TrimSpace(selected.ProviderID) != "" &&
|
||||
strings.TrimSpace(selected.ModelGroupKey) == strings.TrimSpace(expectedGroup) &&
|
||||
strings.TrimSpace(selected.ExecutionPath) == string(result.Path)
|
||||
gate.CapabilitySatisfied = selectedPresetCapability(protocol, selected.ProfileDriver, selected.ProfileCapabilities)
|
||||
gate.CapabilitySatisfied = selectedPresetCapability(protocol, selected.ProfileOperation, selected.ProfileDriver, selected.ProfileCapabilities)
|
||||
return selected, gate, nil
|
||||
}
|
||||
|
||||
|
|
@ -169,10 +170,21 @@ func (s *Server) runLivePresetSelectorResult(
|
|||
}
|
||||
}
|
||||
|
||||
func selectedPresetCapability(protocol, driver string, capabilities []string) bool {
|
||||
func selectedPresetCapability(protocol, operation, driver string, capabilities []string) bool {
|
||||
required := "chat"
|
||||
if protocol == "anthropic" && driver == string(config.ProtocolDriverAnthropicMessages) {
|
||||
required = "messages"
|
||||
if protocol == "anthropic" {
|
||||
switch operation {
|
||||
case string(config.OperationMessages):
|
||||
required = "messages"
|
||||
case string(config.OperationResponses):
|
||||
required = "responses"
|
||||
case string(config.OperationChatCompletions):
|
||||
required = "chat"
|
||||
default:
|
||||
if driver == string(config.ProtocolDriverAnthropicMessages) {
|
||||
required = "messages"
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, capability := range capabilities {
|
||||
if strings.TrimSpace(capability) == required {
|
||||
|
|
@ -440,7 +452,7 @@ func collectPresetTunnelResult(ctx context.Context, handle edgeservice.ProviderT
|
|||
if status < http.StatusOK || status >= http.StatusMultipleChoices {
|
||||
return normalizedStageOutput{}, fmt.Errorf("preset selector provider returned HTTP %d", status)
|
||||
}
|
||||
stage, err := decodePresetTunnelBody(body.Bytes(), contentType, protocol, selected.ProfileDriver)
|
||||
stage, err := decodePresetTunnelBody(body.Bytes(), contentType, protocol, selected.ProfileOperation, selected.ProfileDriver)
|
||||
if err != nil {
|
||||
return normalizedStageOutput{}, err
|
||||
}
|
||||
|
|
@ -453,7 +465,7 @@ func collectPresetTunnelResult(ctx context.Context, handle edgeservice.ProviderT
|
|||
TotalTokens: int(sideUsage.GetInputTokens() + sideUsage.GetOutputTokens()),
|
||||
ReasoningTokens: int(sideUsage.GetReasoningTokens()), CachedInputTokens: int(sideUsage.GetCachedInputTokens()),
|
||||
}
|
||||
if protocol == "anthropic" && selected.ProfileDriver == string(config.ProtocolDriverAnthropicMessages) {
|
||||
if protocol == "anthropic" && selected.ProfileOperation == string(config.OperationMessages) {
|
||||
stage.Usage, _ = json.Marshal(anthropicUsage{
|
||||
InputTokens: int(sideUsage.GetInputTokens()), OutputTokens: int(sideUsage.GetOutputTokens()),
|
||||
CacheReadInputTokens: int(sideUsage.GetCachedInputTokens()),
|
||||
|
|
@ -494,14 +506,17 @@ func unixSeconds(timestamp int64) int64 {
|
|||
return timestamp
|
||||
}
|
||||
|
||||
func decodePresetTunnelBody(body []byte, contentType, protocol, driver string) (normalizedStageOutput, error) {
|
||||
func decodePresetTunnelBody(body []byte, contentType, protocol, operation, driver string) (normalizedStageOutput, error) {
|
||||
streaming := strings.Contains(strings.ToLower(contentType), "text/event-stream") || bytes.Contains(body, []byte("data:"))
|
||||
if protocol == "anthropic" && driver == string(config.ProtocolDriverAnthropicMessages) {
|
||||
if protocol == "anthropic" && (operation == string(config.OperationMessages) || (operation == "" && driver == string(config.ProtocolDriverAnthropicMessages))) {
|
||||
if streaming {
|
||||
return decodeAnthropicPresetSSE(body)
|
||||
}
|
||||
return decodeAnthropicPresetJSON(body)
|
||||
}
|
||||
if protocol == "anthropic" && operation == string(config.OperationResponses) {
|
||||
return decodeResponsesPresetBody(body, streaming)
|
||||
}
|
||||
var stage normalizedStageOutput
|
||||
var err error
|
||||
if streaming {
|
||||
|
|
@ -519,6 +534,30 @@ func decodePresetTunnelBody(body []byte, contentType, protocol, driver string) (
|
|||
return stage, nil
|
||||
}
|
||||
|
||||
func decodeResponsesPresetBody(body []byte, streaming bool) (normalizedStageOutput, error) {
|
||||
responseBody := body
|
||||
if streaming {
|
||||
for _, payload := range splitOpenAIResponsesSSE(body) {
|
||||
var event struct {
|
||||
Type string `json:"type"`
|
||||
Response json.RawMessage `json:"response"`
|
||||
}
|
||||
if json.Unmarshal(payload, &event) == nil && event.Type == "response.completed" && len(event.Response) > 0 {
|
||||
responseBody = event.Response
|
||||
}
|
||||
}
|
||||
}
|
||||
converted, err := convertResponsesResponseToAnthropic(responseBody, "")
|
||||
if err != nil {
|
||||
return normalizedStageOutput{}, err
|
||||
}
|
||||
raw, err := json.Marshal(converted)
|
||||
if err != nil {
|
||||
return normalizedStageOutput{}, err
|
||||
}
|
||||
return decodeAnthropicPresetJSON(raw)
|
||||
}
|
||||
|
||||
func decodeOpenAIPresetJSON(body []byte) (normalizedStageOutput, error) {
|
||||
var response struct {
|
||||
ID string `json:"id"`
|
||||
|
|
@ -1593,7 +1632,8 @@ func stageCorrelation(stageID string, output normalizedStageOutput, dispatch edg
|
|||
// protocol. The tunnel decode and the HTTP-turn stage source both select their
|
||||
// decoder from this single fact rather than the caller endpoint.
|
||||
func hotPathStageWireProtocol(dispatch edgeservice.RunDispatch) string {
|
||||
if dispatch.ProfileDriver == string(config.ProtocolDriverAnthropicMessages) {
|
||||
if dispatch.ProfileOperation == string(config.OperationMessages) ||
|
||||
(dispatch.ProfileOperation == "" && dispatch.ProfileDriver == string(config.ProtocolDriverAnthropicMessages)) {
|
||||
return "anthropic"
|
||||
}
|
||||
return "openai"
|
||||
|
|
|
|||
|
|
@ -105,6 +105,7 @@ type hotPathSelectorGate struct {
|
|||
NodeID string
|
||||
ExecutionPath string
|
||||
ProfileDriver string
|
||||
ProfileOperation string
|
||||
ProfileCapabilities []string
|
||||
Healthy bool
|
||||
CapabilitySatisfied bool
|
||||
|
|
|
|||
457
apps/edge/internal/openai/provider_normalization.go
Normal file
457
apps/edge/internal/openai/provider_normalization.go
Normal file
|
|
@ -0,0 +1,457 @@
|
|||
package openai
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
edgeservice "iop/apps/edge/internal/service"
|
||||
"iop/packages/go/config"
|
||||
)
|
||||
|
||||
// providerRequestRequirements is the caller-neutral request shape used for
|
||||
// provider operation selection. Agent or SDK identity is intentionally absent.
|
||||
type providerRequestRequirements struct {
|
||||
Effort string
|
||||
HasTools bool
|
||||
HasTokenBudget bool
|
||||
Stream bool
|
||||
StructuredOutput bool
|
||||
}
|
||||
|
||||
type providerOperationPlan struct {
|
||||
Operation config.ProtocolOperation
|
||||
Effort string
|
||||
EffortWire string
|
||||
}
|
||||
|
||||
type openAIResponsesBridgeResponse struct {
|
||||
ID string `json:"id"`
|
||||
Model string `json:"model"`
|
||||
Status string `json:"status"`
|
||||
Output []struct {
|
||||
Type string `json:"type"`
|
||||
ID string `json:"id"`
|
||||
CallID string `json:"call_id"`
|
||||
Name string `json:"name"`
|
||||
Arguments string `json:"arguments"`
|
||||
Content []struct {
|
||||
Type string `json:"type"`
|
||||
Text string `json:"text"`
|
||||
} `json:"content"`
|
||||
Summary []struct {
|
||||
Type string `json:"type"`
|
||||
Text string `json:"text"`
|
||||
} `json:"summary"`
|
||||
} `json:"output"`
|
||||
Usage struct {
|
||||
InputTokens int `json:"input_tokens"`
|
||||
OutputTokens int `json:"output_tokens"`
|
||||
InputDetails struct {
|
||||
CachedTokens int `json:"cached_tokens"`
|
||||
} `json:"input_tokens_details"`
|
||||
} `json:"usage"`
|
||||
IncompleteDetails struct {
|
||||
Reason string `json:"reason"`
|
||||
} `json:"incomplete_details"`
|
||||
}
|
||||
|
||||
func anthropicProviderRequirements(req anthropicMessageRequest) providerRequestRequirements {
|
||||
requirements := providerRequestRequirements{
|
||||
HasTools: len(req.Tools) > 0,
|
||||
Stream: req.Stream,
|
||||
HasTokenBudget: req.Thinking != nil && req.Thinking.Type == "enabled",
|
||||
StructuredOutput: req.OutputConfig != nil && req.OutputConfig.Format != nil,
|
||||
}
|
||||
if req.OutputConfig != nil {
|
||||
requirements.Effort = strings.TrimSpace(req.OutputConfig.Effort)
|
||||
}
|
||||
return requirements
|
||||
}
|
||||
|
||||
func decodeAnthropicProviderRequirements(body []byte) (providerRequestRequirements, error) {
|
||||
var request struct {
|
||||
Stream bool `json:"stream"`
|
||||
Tools []json.RawMessage `json:"tools"`
|
||||
Thinking *anthropicThinkingConfig `json:"thinking"`
|
||||
OutputConfig *anthropicOutputConfig `json:"output_config"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &request); err != nil {
|
||||
return providerRequestRequirements{}, fmt.Errorf("decode Messages request")
|
||||
}
|
||||
return anthropicProviderRequirements(anthropicMessageRequest{
|
||||
Stream: request.Stream, Tools: make([]anthropicTool, len(request.Tools)),
|
||||
Thinking: request.Thinking, OutputConfig: request.OutputConfig,
|
||||
}), nil
|
||||
}
|
||||
|
||||
func decodeResponsesProviderRequirements(body []byte) (providerRequestRequirements, error) {
|
||||
var request struct {
|
||||
Tools []json.RawMessage `json:"tools"`
|
||||
Reasoning *struct {
|
||||
Effort string `json:"effort"`
|
||||
} `json:"reasoning"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &request); err != nil {
|
||||
return providerRequestRequirements{}, fmt.Errorf("decode Responses request")
|
||||
}
|
||||
requirements := providerRequestRequirements{HasTools: len(request.Tools) > 0}
|
||||
if request.Reasoning != nil {
|
||||
requirements.Effort = strings.TrimSpace(request.Reasoning.Effort)
|
||||
}
|
||||
return requirements, nil
|
||||
}
|
||||
|
||||
func responsesCandidatePredicate(requirements providerRequestRequirements) edgeservice.ProviderPoolCandidatePredicate {
|
||||
return func(candidate edgeservice.ProviderPoolCandidate) bool {
|
||||
// A nil profile is the legacy tunnel contract: operation resolution is
|
||||
// deferred to the existing dispatch path, which must remain compatible.
|
||||
if candidate.ProtocolProfile == nil {
|
||||
return true
|
||||
}
|
||||
if candidate.ExecutionPath != string(edgeservice.ProviderPoolPathTunnel) {
|
||||
return requirements.Effort == "" && !requirements.HasTools && !requirements.HasTokenBudget
|
||||
}
|
||||
_, err := selectProviderOperation(*candidate.ProtocolProfile, config.OperationResponses, requirements)
|
||||
return err == nil
|
||||
}
|
||||
}
|
||||
|
||||
func rewriteResponsesProviderControls(body []byte, target string, plan providerOperationPlan) ([]byte, error) {
|
||||
patches := make([]topLevelJSONPatch, 0, 2)
|
||||
if strings.TrimSpace(target) != "" {
|
||||
modelJSON, err := json.Marshal(target)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
patches = append(patches, topLevelJSONPatch{name: "model", value: modelJSON})
|
||||
}
|
||||
if plan.Effort != "" {
|
||||
var root map[string]json.RawMessage
|
||||
if err := json.Unmarshal(body, &root); err != nil {
|
||||
return nil, fmt.Errorf("decode Responses request")
|
||||
}
|
||||
var reasoning map[string]any
|
||||
if raw := root["reasoning"]; len(raw) > 0 && string(raw) != "null" {
|
||||
if err := json.Unmarshal(raw, &reasoning); err != nil {
|
||||
return nil, fmt.Errorf("reasoning must be an object")
|
||||
}
|
||||
}
|
||||
if reasoning == nil {
|
||||
reasoning = make(map[string]any)
|
||||
}
|
||||
reasoning["effort"] = plan.Effort
|
||||
reasoningJSON, err := json.Marshal(reasoning)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
patches = append(patches, topLevelJSONPatch{name: "reasoning", value: reasoningJSON})
|
||||
}
|
||||
if len(patches) == 0 {
|
||||
return body, nil
|
||||
}
|
||||
patchPlan, err := planTopLevelJSONPatches(body, patches)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return patchPlan.apply(), nil
|
||||
}
|
||||
|
||||
// selectProviderOperation chooses an operation solely from normalized request
|
||||
// requirements and the selected provider profile. The order prefers the
|
||||
// closest wire surface, but only an operation that preserves every declared
|
||||
// requirement is eligible.
|
||||
func selectProviderOperation(profile config.ConcreteProtocolProfile, ingress config.ProtocolOperation, requirements providerRequestRequirements) (providerOperationPlan, error) {
|
||||
operations := []config.ProtocolOperation{ingress}
|
||||
if ingress == config.OperationMessages {
|
||||
switch profile.Driver {
|
||||
case config.ProtocolDriverAnthropicMessages:
|
||||
operations = []config.ProtocolOperation{config.OperationMessages}
|
||||
case config.ProtocolDriverOpenAIChat:
|
||||
operations = []config.ProtocolOperation{config.OperationChatCompletions, config.OperationResponses}
|
||||
case config.ProtocolDriverOpenAIResponses:
|
||||
operations = []config.ProtocolOperation{config.OperationResponses}
|
||||
default:
|
||||
operations = nil
|
||||
}
|
||||
}
|
||||
|
||||
for _, operation := range operations {
|
||||
if _, ok := profile.Operations[string(operation)]; !ok {
|
||||
continue
|
||||
}
|
||||
if required := operationRequiredCapability(operation); required != "" && !profile.HasCapability(required) {
|
||||
continue
|
||||
}
|
||||
if requirements.Stream && !profile.HasCapability("streaming") {
|
||||
continue
|
||||
}
|
||||
if requirements.HasTools && !profile.HasCapability("tool_calling") {
|
||||
continue
|
||||
}
|
||||
|
||||
plan := providerOperationPlan{Operation: operation}
|
||||
mapping, hasMapping := profile.EffortMapping(operation)
|
||||
supportsTokenBudget := hasMapping && mapping.TokenBudget
|
||||
if operation == config.OperationChatCompletions && profileSupportsAnthropicThinking(profile) {
|
||||
supportsTokenBudget = true
|
||||
}
|
||||
if requirements.HasTokenBudget && !supportsTokenBudget {
|
||||
continue
|
||||
}
|
||||
if requirements.Effort != "" {
|
||||
mapped, ok := profile.MapReasoningEffort(operation, requirements.Effort, requirements.HasTools)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
plan.Effort = mapped
|
||||
plan.EffortWire = mapping.Wire
|
||||
}
|
||||
return plan, nil
|
||||
}
|
||||
|
||||
return providerOperationPlan{}, fmt.Errorf("protocol profile %q cannot preserve the requested operation, tools, and reasoning controls", profile.ID)
|
||||
}
|
||||
|
||||
func operationRequiredCapability(operation config.ProtocolOperation) string {
|
||||
switch operation {
|
||||
case config.OperationMessages:
|
||||
return "messages"
|
||||
case config.OperationChatCompletions:
|
||||
return "chat"
|
||||
case config.OperationResponses:
|
||||
return "responses"
|
||||
case config.OperationCountTokens:
|
||||
return "count_tokens"
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
func prepareAnthropicResponsesBridge(body []byte, target string, profile config.ConcreteProtocolProfile, plan providerOperationPlan) ([]byte, anthropicMessageRequest, error) {
|
||||
req, err := decodeAnthropicMessageRequest(body, true)
|
||||
if err != nil {
|
||||
return nil, req, err
|
||||
}
|
||||
if plan.Operation != config.OperationResponses || (plan.Effort != "" && plan.EffortWire != config.ProtocolEffortWireOpenAIResponses) {
|
||||
return nil, req, fmt.Errorf("selected operation has incompatible Responses effort normalization")
|
||||
}
|
||||
if req.TopK != nil {
|
||||
return nil, req, fmt.Errorf("top_k is not supported by the Responses bridge")
|
||||
}
|
||||
if req.Thinking != nil && req.Thinking.Type == "enabled" {
|
||||
return nil, req, fmt.Errorf("selected Responses profile does not support an explicit thinking token budget")
|
||||
}
|
||||
|
||||
input := make([]map[string]any, 0, len(req.Messages))
|
||||
for index, message := range req.Messages {
|
||||
blocks, err := decodeAnthropicContent(message.Content)
|
||||
if err != nil {
|
||||
return nil, req, fmt.Errorf("messages[%d].content: %w", index, err)
|
||||
}
|
||||
converted, err := anthropicMessageToResponses(message.Role, blocks)
|
||||
if err != nil {
|
||||
return nil, req, fmt.Errorf("messages[%d]: %w", index, err)
|
||||
}
|
||||
input = append(input, converted...)
|
||||
}
|
||||
|
||||
responses := map[string]any{
|
||||
"model": target,
|
||||
"input": input,
|
||||
"max_output_tokens": *req.MaxTokens,
|
||||
"stream": req.Stream,
|
||||
}
|
||||
if system, err := decodeAnthropicSystem(req.System); err != nil {
|
||||
return nil, req, err
|
||||
} else if len(system) > 0 {
|
||||
parts := make([]string, 0, len(system))
|
||||
for _, block := range system {
|
||||
parts = append(parts, block.Text)
|
||||
}
|
||||
responses["instructions"] = strings.Join(parts, "\n")
|
||||
}
|
||||
if plan.Effort != "" {
|
||||
responses["reasoning"] = map[string]any{"effort": plan.Effort}
|
||||
}
|
||||
if req.Temperature != nil {
|
||||
responses["temperature"] = *req.Temperature
|
||||
}
|
||||
if req.TopP != nil {
|
||||
responses["top_p"] = *req.TopP
|
||||
}
|
||||
if len(req.StopSequences) > 0 {
|
||||
return nil, req, fmt.Errorf("stop_sequences is not supported by the Responses bridge")
|
||||
}
|
||||
if len(req.Tools) > 0 {
|
||||
tools := make([]map[string]any, 0, len(req.Tools))
|
||||
for _, tool := range req.Tools {
|
||||
var schema map[string]any
|
||||
if err := json.Unmarshal(tool.InputSchema, &schema); err != nil {
|
||||
return nil, req, fmt.Errorf("tool %q input_schema is invalid", tool.Name)
|
||||
}
|
||||
converted := map[string]any{"type": "function", "name": tool.Name, "parameters": schema}
|
||||
if tool.Description != "" {
|
||||
converted["description"] = tool.Description
|
||||
}
|
||||
tools = append(tools, converted)
|
||||
}
|
||||
responses["tools"] = tools
|
||||
}
|
||||
if req.ToolChoice != nil {
|
||||
choice, parallel := anthropicToolChoiceToResponses(*req.ToolChoice)
|
||||
responses["tool_choice"] = choice
|
||||
if parallel != nil {
|
||||
responses["parallel_tool_calls"] = *parallel
|
||||
}
|
||||
}
|
||||
if req.OutputConfig != nil && req.OutputConfig.Format != nil {
|
||||
var schema map[string]any
|
||||
if err := json.Unmarshal(req.OutputConfig.Format.Schema, &schema); err != nil {
|
||||
return nil, req, fmt.Errorf("decode output_config.format.schema: %w", err)
|
||||
}
|
||||
responses["text"] = map[string]any{"format": map[string]any{
|
||||
"type": "json_schema", "name": "response", "strict": true, "schema": schema,
|
||||
}}
|
||||
}
|
||||
|
||||
encoded, err := json.Marshal(responses)
|
||||
if err != nil {
|
||||
return nil, req, fmt.Errorf("encode Responses bridge request: %w", err)
|
||||
}
|
||||
return encoded, req, nil
|
||||
}
|
||||
|
||||
func anthropicMessageToResponses(role string, blocks []anthropicContentBlock) ([]map[string]any, error) {
|
||||
if role == "assistant" {
|
||||
out := make([]map[string]any, 0, len(blocks))
|
||||
for _, block := range blocks {
|
||||
switch block.Type {
|
||||
case "text":
|
||||
out = append(out, map[string]any{"type": "message", "role": "assistant", "content": []map[string]any{{"type": "output_text", "text": block.Text}}})
|
||||
case "thinking":
|
||||
if block.Signature != "" {
|
||||
return nil, fmt.Errorf("signed thinking blocks cannot be represented by the Responses bridge")
|
||||
}
|
||||
case "tool_use":
|
||||
callID, _, _ := decodeAnthropicBridgeToolID(block.ID)
|
||||
out = append(out, map[string]any{"type": "function_call", "call_id": callID, "name": block.Name, "arguments": string(block.Input)})
|
||||
default:
|
||||
return nil, fmt.Errorf("content block %q is invalid for an assistant message", block.Type)
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
out := make([]map[string]any, 0, len(blocks))
|
||||
content := make([]map[string]any, 0, len(blocks))
|
||||
flushContent := func() {
|
||||
if len(content) > 0 {
|
||||
out = append(out, map[string]any{"type": "message", "role": "user", "content": content})
|
||||
content = nil
|
||||
}
|
||||
}
|
||||
for _, block := range blocks {
|
||||
switch block.Type {
|
||||
case "text":
|
||||
content = append(content, map[string]any{"type": "input_text", "text": block.Text})
|
||||
case "image":
|
||||
imageURL := block.Source.URL
|
||||
if block.Source.Type == "base64" {
|
||||
imageURL = "data:" + block.Source.MediaType + ";base64," + block.Source.Data
|
||||
}
|
||||
content = append(content, map[string]any{"type": "input_image", "image_url": imageURL})
|
||||
case "tool_result":
|
||||
flushContent()
|
||||
result, err := anthropicToolResultText(block.Content)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if block.IsError {
|
||||
result = "Error: " + result
|
||||
}
|
||||
callID, _, _ := decodeAnthropicBridgeToolID(block.ToolUseID)
|
||||
out = append(out, map[string]any{"type": "function_call_output", "call_id": callID, "output": result})
|
||||
default:
|
||||
return nil, fmt.Errorf("content block %q is invalid for a user message", block.Type)
|
||||
}
|
||||
}
|
||||
flushContent()
|
||||
if len(out) == 0 {
|
||||
return nil, fmt.Errorf("user message content is empty")
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func anthropicToolChoiceToResponses(choice anthropicToolChoice) (any, *bool) {
|
||||
parallel := !choice.DisableParallelToolUse
|
||||
switch choice.Type {
|
||||
case "auto":
|
||||
return "auto", ¶llel
|
||||
case "any":
|
||||
return "required", ¶llel
|
||||
case "none":
|
||||
return "none", ¶llel
|
||||
default:
|
||||
return map[string]any{"type": "function", "name": choice.Name}, ¶llel
|
||||
}
|
||||
}
|
||||
|
||||
func convertResponsesResponseToAnthropic(body []byte, requestModel string) (anthropicMessageResponse, error) {
|
||||
var response openAIResponsesBridgeResponse
|
||||
if err := json.Unmarshal(body, &response); err != nil {
|
||||
return anthropicMessageResponse{}, fmt.Errorf("decode Responses response: %w", err)
|
||||
}
|
||||
if strings.TrimSpace(response.ID) == "" {
|
||||
return anthropicMessageResponse{}, fmt.Errorf("Responses response has no id")
|
||||
}
|
||||
content := make([]map[string]any, 0, len(response.Output))
|
||||
hasTools := false
|
||||
for _, item := range response.Output {
|
||||
switch item.Type {
|
||||
case "message":
|
||||
for _, part := range item.Content {
|
||||
if part.Type == "output_text" && part.Text != "" {
|
||||
content = append(content, map[string]any{"type": "text", "text": part.Text})
|
||||
}
|
||||
}
|
||||
case "reasoning":
|
||||
for _, part := range item.Summary {
|
||||
if part.Text != "" {
|
||||
content = append(content, map[string]any{"type": "thinking", "thinking": part.Text, "signature": ""})
|
||||
}
|
||||
}
|
||||
case "function_call":
|
||||
if item.CallID == "" || item.Name == "" || !json.Valid([]byte(item.Arguments)) {
|
||||
return anthropicMessageResponse{}, fmt.Errorf("Responses function call has invalid call_id, name, or arguments")
|
||||
}
|
||||
var input any
|
||||
if err := json.Unmarshal([]byte(item.Arguments), &input); err != nil {
|
||||
return anthropicMessageResponse{}, fmt.Errorf("decode Responses function arguments: %w", err)
|
||||
}
|
||||
content = append(content, map[string]any{
|
||||
"type": "tool_use", "id": encodeAnthropicBridgeToolID(item.CallID, openAIChatToolExtraContent{}),
|
||||
"name": item.Name, "input": input,
|
||||
})
|
||||
hasTools = true
|
||||
}
|
||||
}
|
||||
stopReason := "end_turn"
|
||||
if hasTools {
|
||||
stopReason = "tool_use"
|
||||
} else if response.Status == "incomplete" && response.IncompleteDetails.Reason == "max_output_tokens" {
|
||||
stopReason = "max_tokens"
|
||||
}
|
||||
model := requestModel
|
||||
if model == "" {
|
||||
model = response.Model
|
||||
}
|
||||
return anthropicMessageResponse{
|
||||
ID: response.ID, Type: "message", Role: "assistant", Model: model, Content: content,
|
||||
StopReason: &stopReason,
|
||||
Usage: anthropicUsage{
|
||||
InputTokens: response.Usage.InputTokens, OutputTokens: response.Usage.OutputTokens,
|
||||
CacheReadInputTokens: response.Usage.InputDetails.CachedTokens,
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
|
|
@ -247,6 +247,9 @@ func (s *providerFakeRunService) SubmitProviderPool(_ context.Context, req edges
|
|||
if selectedProvider := strings.TrimSpace(s.poolSelectedCandidate.ProviderID); selectedProvider != "" {
|
||||
tunnelReq.ProviderID = selectedProvider
|
||||
}
|
||||
if selectedTarget := strings.TrimSpace(s.poolSelectedCandidate.ActualModel); selectedTarget != "" {
|
||||
tunnelReq.Target = selectedTarget
|
||||
}
|
||||
if req.PrepareProtocolTunnel != nil {
|
||||
tunnelReqPrepared, prepErr := req.PrepareProtocolTunnel(tunnelReq, s.poolSelectedCandidate)
|
||||
if prepErr != nil {
|
||||
|
|
@ -291,6 +294,7 @@ func (s *providerFakeRunService) SubmitProviderPool(_ context.Context, req edges
|
|||
if selected := s.poolSelectedCandidate; selected.ProtocolProfile != nil {
|
||||
disp.ProfileID = selected.ProfileID
|
||||
disp.ProfileDriver = selected.ProfileDriver
|
||||
disp.ProfileOperation = req.Tunnel.Operation
|
||||
disp.ProfileCapabilities = append([]string(nil), selected.ProfileCapabilities...)
|
||||
if selected.ProviderID != "" {
|
||||
disp.ProviderID = selected.ProviderID
|
||||
|
|
|
|||
|
|
@ -320,6 +320,12 @@ func (s *Server) handleResponsesProviderPool(w http.ResponseWriter, requestCtx *
|
|||
}
|
||||
estimate := requestCtx.estimate
|
||||
contextClass := requestCtx.contextClass
|
||||
requirements, err := decodeResponsesProviderRequirements(rawBody)
|
||||
if err != nil {
|
||||
requestCtx.finishUsageRequest(usageStatusError, responseModePassthrough)
|
||||
writeError(w, http.StatusBadRequest, "invalid_request_error", err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
env := requestCtx.envelope
|
||||
runMeta := cloneMetadata(requestCtx.callerMetadata)
|
||||
|
|
@ -367,6 +373,7 @@ func (s *Server) handleResponsesProviderPool(w http.ResponseWriter, requestCtx *
|
|||
Run: baseRun,
|
||||
Tunnel: baseTunnel,
|
||||
}
|
||||
poolReq.AcceptCandidate = responsesCandidatePredicate(requirements)
|
||||
|
||||
if s.streamGateSemanticEnabled() {
|
||||
fctx, err := s.openAIResponsesOutputFilterContext(requestCtx)
|
||||
|
|
@ -381,7 +388,7 @@ func (s *Server) handleResponsesProviderPool(w http.ResponseWriter, requestCtx *
|
|||
writeError(w, http.StatusInternalServerError, "run_error", "stream gate runtime unavailable")
|
||||
return
|
||||
}
|
||||
poolReq.AcceptCandidate = predicate
|
||||
poolReq.AcceptCandidate = composeCandidatePredicates(poolReq.AcceptCandidate, predicate)
|
||||
}
|
||||
if requestCtx.route.Managed {
|
||||
poolReq.AcceptCandidate = composeCandidatePredicates(poolReq.AcceptCandidate, requestCtx.route.CandidatePredicate())
|
||||
|
|
@ -399,11 +406,29 @@ func (s *Server) handleResponsesProviderPool(w http.ResponseWriter, requestCtx *
|
|||
tunnelReq.Headers = headers
|
||||
return tunnelReq, nil
|
||||
}
|
||||
poolReq.PrepareProtocolTunnel = s.protocolTunnelPreparer(r, config.OperationResponses)
|
||||
basePreparer := s.protocolTunnelPreparer(r, config.OperationResponses)
|
||||
poolReq.PrepareProtocolTunnel = func(tunnelReq edgeservice.SubmitProviderTunnelRequest, selected edgeservice.ProviderPoolCandidate) (edgeservice.SubmitProviderTunnelRequest, error) {
|
||||
prepared, err := basePreparer(tunnelReq, selected)
|
||||
if err != nil {
|
||||
return tunnelReq, err
|
||||
}
|
||||
if selected.ProtocolProfile == nil {
|
||||
return prepared, nil
|
||||
}
|
||||
plan, err := selectProviderOperation(*selected.ProtocolProfile, config.OperationResponses, requirements)
|
||||
if err != nil {
|
||||
return tunnelReq, err
|
||||
}
|
||||
prepared.BuildBody = func(target string) ([]byte, error) {
|
||||
return rewriteResponsesProviderControls(rawBody, target, plan)
|
||||
}
|
||||
return prepared, nil
|
||||
}
|
||||
|
||||
// Tunnel branch: rewrite only the model field, preserve all other fields
|
||||
// (tools, max_output_tokens, custom fields). Stream/background gating is
|
||||
// skipped: the provider itself enforces those constraints (SDD S04).
|
||||
// Tunnel branch rewrites the model and, for concrete profiles, the mapped
|
||||
// reasoning effort. All other fields (tools, max_output_tokens, custom
|
||||
// fields) are preserved. Stream/background gating is skipped: the provider
|
||||
// itself enforces those constraints (SDD S04).
|
||||
bodyBuilder := newOpenAIProviderBodyBuilder(func(target string) (*openAIRebuiltLease, error) {
|
||||
return rewriteResponsesModelFromIngress(requestCtx.ingress, target)
|
||||
})
|
||||
|
|
|
|||
|
|
@ -126,6 +126,55 @@ func TestResponsesProtocolProfileOperationPassthroughNonStream(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
func TestResponsesProtocolProfileEffortFallsBackToNearestLowerGrade(t *testing.T) {
|
||||
profile, err := config.ResolveProtocolProfile("openai", "", config.BuiltInProtocolProfileCatalog())
|
||||
if err != nil {
|
||||
t.Fatalf("ResolveProtocolProfile: %v", err)
|
||||
}
|
||||
mapping := profile.Normalization.Effort[string(config.OperationResponses)]
|
||||
delete(mapping.Levels, "max")
|
||||
profile.Normalization.Effort[string(config.OperationResponses)] = mapping
|
||||
|
||||
fake := &providerFakeRunService{
|
||||
tunnelFrames: staticProviderTunnelFrames(`{"id":"resp-effort","object":"response","output":[]}`),
|
||||
poolSelectedCandidate: edgeservice.ProviderPoolCandidate{
|
||||
ActualModel: "gpt-served",
|
||||
ProviderID: "prov-openai-effort",
|
||||
ExecutionPath: string(edgeservice.ProviderPoolPathTunnel),
|
||||
ProfileID: profile.ID,
|
||||
ProfileDriver: string(profile.Driver),
|
||||
ProfileCapabilities: append([]string(nil), profile.Capabilities...),
|
||||
ProtocolProfile: &profile,
|
||||
},
|
||||
}
|
||||
srv := NewServer(config.EdgeOpenAIConf{}, fake, nil)
|
||||
srv.SetModelCatalog([]config.ModelCatalogEntry{{ID: "responses-effort", Providers: map[string]string{"prov-openai-effort": "gpt-served"}}})
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/v1/responses", strings.NewReader(`{
|
||||
"model":"responses-effort",
|
||||
"input":"use the tool",
|
||||
"reasoning":{"effort":"max","summary":"auto"},
|
||||
"tools":[{"type":"function","name":"lookup","parameters":{"type":"object"}}]
|
||||
}`))
|
||||
w := httptest.NewRecorder()
|
||||
srv.handleResponses(w, req)
|
||||
|
||||
if w.Code != http.StatusOK {
|
||||
t.Fatalf("status=%d body=%s", w.Code, w.Body.String())
|
||||
}
|
||||
var upstream map[string]any
|
||||
if err := json.Unmarshal(fake.tunnelBodiesSnapshot()[0], &upstream); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
reasoning, _ := upstream["reasoning"].(map[string]any)
|
||||
if reasoning["effort"] != "xhigh" || reasoning["summary"] != "auto" {
|
||||
t.Fatalf("reasoning=%v, want effort fallback with preserved summary", reasoning)
|
||||
}
|
||||
if upstream["model"] != "gpt-served" {
|
||||
t.Fatalf("model=%v, want served target", upstream["model"])
|
||||
}
|
||||
}
|
||||
|
||||
// TestResponsesProtocolProfileOperationPassthroughStream verifies that the
|
||||
// responses profile operation admits stream=true passthrough and that the
|
||||
// provider tunnel carries the stream flag to the upstream provider.
|
||||
|
|
|
|||
|
|
@ -392,6 +392,7 @@ func (s *Service) dispatchProviderPoolTunnel(
|
|||
disp.ExecutionPath = string(selected.executionPath)
|
||||
disp.QueueReason = queueReason
|
||||
disp.ProfileID, disp.ProfileDriver = profileFacts(selected.profile)
|
||||
disp.ProfileOperation = tunnelReq.Operation
|
||||
if selected.profile != nil {
|
||||
disp.ProfileCapabilities = append([]string(nil), selected.profile.Capabilities...)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -63,6 +63,7 @@ type RunDispatch struct {
|
|||
ExecutionPath string // non-empty for provider-pool dispatches
|
||||
ProfileID string
|
||||
ProfileDriver string
|
||||
ProfileOperation string
|
||||
ProfileCapabilities []string
|
||||
CredentialSlotRef string
|
||||
CredentialRevision uint64
|
||||
|
|
|
|||
|
|
@ -284,22 +284,25 @@ The deterministic Messages qualification succeeds alongside Chat: the Control Pl
|
|||
|
||||
```bash
|
||||
read -r IOP_AGY_SMOKE_TOKEN < token/.iop-principal
|
||||
export GEMINI_API_KEY="$IOP_AGY_SMOKE_TOKEN"
|
||||
export SSL_CERT_FILE="$PWD/token/iop-dev-ca.pem"
|
||||
export NODE_EXTRA_CA_CERTS="$PWD/token/iop-dev-ca.pem"
|
||||
|
||||
GEMINI_API_KEY="$IOP_AGY_SMOKE_TOKEN" \
|
||||
SSL_CERT_FILE="$PWD/token/iop-dev-ca.pem" \
|
||||
NODE_EXTRA_CA_CERTS="$PWD/token/iop-dev-ca.pem" \
|
||||
GOOGLE_GEMINI_BASE_URL="https://<edge-host>:<https-port>/gemini/<direct-route-id>" \
|
||||
agy --sandbox --output-format stream-json --model 'Gemini 3.6 Flash' \
|
||||
--print 'Reply only with OK. Do not use tools or modify files.'
|
||||
|
||||
GEMINI_API_KEY="$IOP_AGY_SMOKE_TOKEN" \
|
||||
SSL_CERT_FILE="$PWD/token/iop-dev-ca.pem" \
|
||||
NODE_EXTRA_CA_CERTS="$PWD/token/iop-dev-ca.pem" \
|
||||
GOOGLE_GEMINI_BASE_URL="https://<edge-host>:<https-port>/gemini/<hybrid-preset-id>" \
|
||||
agy --sandbox --output-format stream-json --model 'Gemini 3.6 Flash' \
|
||||
--print 'Inspect README.md and report only its first Markdown heading. Do not modify files.'
|
||||
|
||||
unset IOP_AGY_SMOKE_TOKEN GEMINI_API_KEY SSL_CERT_FILE NODE_EXTRA_CA_CERTS
|
||||
unset IOP_AGY_SMOKE_TOKEN
|
||||
```
|
||||
|
||||
`--effort`는 API-key provider 호출에 넣지 않는다. direct와 hybrid 모두 JSONL의 마지막 record가 `event=result`, 중첩 `result.status=SUCCESS` 한 건이어야 한다. hybrid는 plan/work/review가 포함되므로 direct보다 오래 걸릴 수 있다. 한 경로가 실패하면 다른 경로를 묶어 재실행하지 않고 해당 IOP ingress, route binding, preset stage 또는 caller terminal을 분리해 확인한다. Gemini ingress와 공식 event 구조의 상세 계약은 `agent-contract/outer/gemini-compatible-api.md`를 기준으로 한다.
|
||||
`SSL_CERT_FILE`과 `NODE_EXTRA_CA_CERTS`는 위 caller process에만 적용한다. Codex/IDE 시작 환경이나 ambient shell에 export하면 ChatGPT WebSocket 같은 공개 TLS 연결까지 사설 CA override를 사용하므로 금지한다. `--effort`는 API-key provider 호출에 넣지 않는다. direct와 hybrid 모두 JSONL의 마지막 record가 `event=result`, 중첩 `result.status=SUCCESS` 한 건이어야 한다. hybrid는 plan/work/review가 포함되므로 direct보다 오래 걸릴 수 있다. 한 경로가 실패하면 다른 경로를 묶어 재실행하지 않고 해당 IOP ingress, route binding, preset stage 또는 caller terminal을 분리해 확인한다. Gemini ingress와 공식 event 구조의 상세 계약은 `agent-contract/outer/gemini-compatible-api.md`를 기준으로 한다.
|
||||
|
||||
### Incident redaction check
|
||||
|
||||
|
|
|
|||
|
|
@ -101,6 +101,37 @@ type ProtocolAuthConf struct {
|
|||
Scheme string `mapstructure:"scheme" yaml:"scheme,omitempty"`
|
||||
}
|
||||
|
||||
// ProtocolEffortMappingConf declares how one provider operation represents
|
||||
// IOP's normalized reasoning-effort levels. Levels may map multiple IOP grades
|
||||
// to one provider grade when the provider exposes a smaller scale.
|
||||
type ProtocolEffortMappingConf struct {
|
||||
Levels map[string]string `mapstructure:"levels" yaml:"levels,omitempty"`
|
||||
WithTools bool `mapstructure:"with_tools" yaml:"with_tools,omitempty"`
|
||||
TokenBudget bool `mapstructure:"token_budget" yaml:"token_budget,omitempty"`
|
||||
Wire string `mapstructure:"wire" yaml:"wire,omitempty"`
|
||||
}
|
||||
|
||||
const (
|
||||
ProtocolEffortWireOpenAIChat = "openai_chat"
|
||||
ProtocolEffortWireOpenAIResponses = "openai_responses"
|
||||
ProtocolEffortWireAnthropicMessage = "anthropic_messages"
|
||||
ProtocolEffortWireGeminiChat = "gemini_openai_chat"
|
||||
)
|
||||
|
||||
var validProtocolEffortWires = map[string]struct{}{
|
||||
ProtocolEffortWireOpenAIChat: {},
|
||||
ProtocolEffortWireOpenAIResponses: {},
|
||||
ProtocolEffortWireAnthropicMessage: {},
|
||||
ProtocolEffortWireGeminiChat: {},
|
||||
}
|
||||
|
||||
// ProtocolNormalizationConf contains provider-wire normalization facts. It is
|
||||
// intentionally operation-scoped: a provider may support reasoning with tools
|
||||
// on Responses while rejecting the same semantic request on Chat Completions.
|
||||
type ProtocolNormalizationConf struct {
|
||||
Effort map[string]ProtocolEffortMappingConf `mapstructure:"effort" yaml:"effort,omitempty"`
|
||||
}
|
||||
|
||||
// ProtocolProfileConf is the overlayable configuration of a protocol profile.
|
||||
// It is the source of truth for endpoint, operation paths, auth, and
|
||||
// capabilities before concrete resolution.
|
||||
|
|
@ -128,6 +159,32 @@ type ProtocolProfileConf struct {
|
|||
// Extensions holds restricted profile-specific options that cannot be
|
||||
// expressed in the typed fields above.
|
||||
Extensions map[string]any `mapstructure:"extensions" yaml:"extensions,omitempty"`
|
||||
// Normalization maps IOP semantic request controls to provider operations.
|
||||
Normalization ProtocolNormalizationConf `mapstructure:"normalization" yaml:"normalization,omitempty"`
|
||||
}
|
||||
|
||||
var identityReasoningEffortLevels = map[string]string{
|
||||
"none": "none", "low": "low", "medium": "medium",
|
||||
"high": "high", "xhigh": "xhigh", "max": "max",
|
||||
}
|
||||
|
||||
var reasoningEffortOrder = []string{"none", "low", "medium", "high", "xhigh", "max"}
|
||||
|
||||
func reasoningEffortGradeIndex(level string) int {
|
||||
for index, candidate := range reasoningEffortOrder {
|
||||
if candidate == level {
|
||||
return index
|
||||
}
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
func identityEffortMapping(wire string, withTools, tokenBudget bool) ProtocolEffortMappingConf {
|
||||
levels := make(map[string]string, len(identityReasoningEffortLevels))
|
||||
for level, providerLevel := range identityReasoningEffortLevels {
|
||||
levels[level] = providerLevel
|
||||
}
|
||||
return ProtocolEffortMappingConf{Levels: levels, WithTools: withTools, TokenBudget: tokenBudget, Wire: wire}
|
||||
}
|
||||
|
||||
// ConcreteProtocolProfile is the immutable, resolved snapshot of a protocol
|
||||
|
|
@ -162,6 +219,10 @@ var builtInProtocolProfiles = map[string]ProtocolProfileConf{
|
|||
},
|
||||
Auth: ProtocolAuthConf{Header: "Authorization", Scheme: "Bearer"},
|
||||
Capabilities: []string{"models", "chat", "streaming", "tool_calling", "responses"},
|
||||
Normalization: ProtocolNormalizationConf{Effort: map[string]ProtocolEffortMappingConf{
|
||||
string(OperationChatCompletions): identityEffortMapping(ProtocolEffortWireOpenAIChat, false, false),
|
||||
string(OperationResponses): identityEffortMapping(ProtocolEffortWireOpenAIResponses, true, false),
|
||||
}},
|
||||
},
|
||||
"gemini": {
|
||||
Driver: ProtocolDriverOpenAIChat,
|
||||
|
|
@ -172,6 +233,9 @@ var builtInProtocolProfiles = map[string]ProtocolProfileConf{
|
|||
},
|
||||
Auth: ProtocolAuthConf{Header: "Authorization", Scheme: "Bearer"},
|
||||
Capabilities: []string{"models", "chat", "streaming", "tool_calling"},
|
||||
Normalization: ProtocolNormalizationConf{Effort: map[string]ProtocolEffortMappingConf{
|
||||
string(OperationChatCompletions): identityEffortMapping(ProtocolEffortWireGeminiChat, true, true),
|
||||
}},
|
||||
},
|
||||
"anthropic": {
|
||||
Driver: ProtocolDriverAnthropicMessages,
|
||||
|
|
@ -182,6 +246,9 @@ var builtInProtocolProfiles = map[string]ProtocolProfileConf{
|
|||
},
|
||||
Auth: ProtocolAuthConf{Header: "x-api-key"},
|
||||
Capabilities: []string{"messages", "streaming", "tool_calling", "count_tokens"},
|
||||
Normalization: ProtocolNormalizationConf{Effort: map[string]ProtocolEffortMappingConf{
|
||||
string(OperationMessages): identityEffortMapping(ProtocolEffortWireAnthropicMessage, true, true),
|
||||
}},
|
||||
},
|
||||
"glm": {
|
||||
Driver: ProtocolDriverOpenAIChat,
|
||||
|
|
@ -386,6 +453,27 @@ func deepCopyProfileConf(src ProtocolProfileConf) ProtocolProfileConf {
|
|||
if src.Extensions != nil {
|
||||
dst.Extensions = deepCopyExtensions(src.Extensions)
|
||||
}
|
||||
if src.Normalization.Effort != nil {
|
||||
dst.Normalization.Effort = make(map[string]ProtocolEffortMappingConf, len(src.Normalization.Effort))
|
||||
for operation, mapping := range src.Normalization.Effort {
|
||||
dst.Normalization.Effort[operation] = cloneEffortMapping(mapping)
|
||||
}
|
||||
}
|
||||
return dst
|
||||
}
|
||||
|
||||
func cloneEffortMapping(src ProtocolEffortMappingConf) ProtocolEffortMappingConf {
|
||||
dst := ProtocolEffortMappingConf{
|
||||
WithTools: src.WithTools,
|
||||
TokenBudget: src.TokenBudget,
|
||||
Wire: src.Wire,
|
||||
}
|
||||
if src.Levels != nil {
|
||||
dst.Levels = make(map[string]string, len(src.Levels))
|
||||
for level, providerLevel := range src.Levels {
|
||||
dst.Levels[level] = providerLevel
|
||||
}
|
||||
}
|
||||
return dst
|
||||
}
|
||||
|
||||
|
|
@ -533,6 +621,14 @@ func mergeProfileOverlay(base, overlay ProtocolProfileConf) (ProtocolProfileConf
|
|||
merged.Extensions[k] = v
|
||||
}
|
||||
}
|
||||
if len(overlay.Normalization.Effort) > 0 {
|
||||
if merged.Normalization.Effort == nil {
|
||||
merged.Normalization.Effort = make(map[string]ProtocolEffortMappingConf)
|
||||
}
|
||||
for operation, mapping := range overlay.Normalization.Effort {
|
||||
merged.Normalization.Effort[operation] = cloneEffortMapping(mapping)
|
||||
}
|
||||
}
|
||||
return merged, nil
|
||||
}
|
||||
|
||||
|
|
@ -583,6 +679,32 @@ func validateConcreteProfile(id string, p ProtocolProfileConf) error {
|
|||
}
|
||||
}
|
||||
}
|
||||
for operation, mapping := range p.Normalization.Effort {
|
||||
if _, ok := p.Operations[operation]; !ok {
|
||||
return fmt.Errorf("profile %q: effort normalization operation %q is not declared", id, operation)
|
||||
}
|
||||
if len(mapping.Levels) == 0 {
|
||||
return fmt.Errorf("profile %q: effort normalization operation %q has no levels", id, operation)
|
||||
}
|
||||
if _, ok := validProtocolEffortWires[mapping.Wire]; !ok {
|
||||
return fmt.Errorf("profile %q: effort normalization operation %q has invalid wire %q", id, operation, mapping.Wire)
|
||||
}
|
||||
for level, providerLevel := range mapping.Levels {
|
||||
levelIndex := reasoningEffortGradeIndex(level)
|
||||
if levelIndex < 0 {
|
||||
return fmt.Errorf("profile %q: effort normalization level %q is not recognized", id, level)
|
||||
}
|
||||
providerLevel = strings.TrimSpace(providerLevel)
|
||||
if providerLevel == "" {
|
||||
return fmt.Errorf("profile %q: effort normalization level %q has an empty provider value", id, level)
|
||||
}
|
||||
// Canonical provider grades may collapse downward, but an explicit
|
||||
// mapping must never silently upgrade the caller's requested grade.
|
||||
if providerIndex := reasoningEffortGradeIndex(providerLevel); providerIndex > levelIndex {
|
||||
return fmt.Errorf("profile %q: effort normalization level %q upgrades to %q", id, level, providerLevel)
|
||||
}
|
||||
}
|
||||
}
|
||||
if p.Auth.Header == "" {
|
||||
return fmt.Errorf("profile %q: auth.header must not be empty", id)
|
||||
}
|
||||
|
|
@ -704,6 +826,37 @@ func (p ConcreteProtocolProfile) MapModel(model string) string {
|
|||
return model
|
||||
}
|
||||
|
||||
// MapReasoningEffort maps one normalized IOP effort grade to the provider
|
||||
// value for an operation. If the exact grade is unsupported, it selects the
|
||||
// nearest explicitly supported lower grade. It never upgrades effort and
|
||||
// rejects unsupported operation/tool combinations.
|
||||
func (p ConcreteProtocolProfile) MapReasoningEffort(operation ProtocolOperation, level string, hasTools bool) (string, bool) {
|
||||
mapping, ok := p.Normalization.Effort[string(operation)]
|
||||
if !ok || (hasTools && !mapping.WithTools) {
|
||||
return "", false
|
||||
}
|
||||
requestedIndex := reasoningEffortGradeIndex(level)
|
||||
if requestedIndex < 0 {
|
||||
return "", false
|
||||
}
|
||||
for index := requestedIndex; index >= 0; index-- {
|
||||
mapped := strings.TrimSpace(mapping.Levels[reasoningEffortOrder[index]])
|
||||
if mapped != "" {
|
||||
return mapped, true
|
||||
}
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
// EffortMapping returns an operation-scoped copy of the normalization facts.
|
||||
func (p ConcreteProtocolProfile) EffortMapping(operation ProtocolOperation) (ProtocolEffortMappingConf, bool) {
|
||||
mapping, ok := p.Normalization.Effort[string(operation)]
|
||||
if !ok {
|
||||
return ProtocolEffortMappingConf{}, false
|
||||
}
|
||||
return cloneEffortMapping(mapping), true
|
||||
}
|
||||
|
||||
// Clone returns a deep copy of the concrete profile.
|
||||
func (p ConcreteProtocolProfile) Clone() ConcreteProtocolProfile {
|
||||
return ConcreteProtocolProfile{
|
||||
|
|
|
|||
|
|
@ -86,6 +86,72 @@ func TestProtocolProfileOverlayDeepCopyImmutability(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
func TestProtocolProfileReasoningEffortNormalization(t *testing.T) {
|
||||
profile, err := config.ResolveProtocolProfile("openai", "", config.BuiltInProtocolProfileCatalog())
|
||||
if err != nil {
|
||||
t.Fatalf("ResolveProtocolProfile: %v", err)
|
||||
}
|
||||
if got, ok := profile.MapReasoningEffort(config.OperationResponses, "xhigh", true); !ok || got != "xhigh" {
|
||||
t.Fatalf("Responses xhigh with tools = %q,%t; want xhigh,true", got, ok)
|
||||
}
|
||||
if got, ok := profile.MapReasoningEffort(config.OperationChatCompletions, "high", true); ok || got != "" {
|
||||
t.Fatalf("Chat high with tools = %q,%t; want unsupported", got, ok)
|
||||
}
|
||||
if got, ok := profile.MapReasoningEffort(config.OperationChatCompletions, "high", false); !ok || got != "high" {
|
||||
t.Fatalf("Chat high without tools = %q,%t; want high,true", got, ok)
|
||||
}
|
||||
}
|
||||
|
||||
func TestProtocolProfileEffortNormalizationOverlayAndValidation(t *testing.T) {
|
||||
catalog := config.BuiltInProtocolProfileCatalog()
|
||||
catalog["four-grade-openai"] = config.ProtocolProfileConf{
|
||||
Base: "openai",
|
||||
Normalization: config.ProtocolNormalizationConf{Effort: map[string]config.ProtocolEffortMappingConf{
|
||||
string(config.OperationResponses): {
|
||||
WithTools: true,
|
||||
Wire: config.ProtocolEffortWireOpenAIResponses,
|
||||
Levels: map[string]string{
|
||||
"none": "none", "low": "low", "medium": "medium",
|
||||
"high": "high", "xhigh": "xhigh",
|
||||
},
|
||||
},
|
||||
}},
|
||||
}
|
||||
profile, err := config.ResolveProtocolProfile("four-grade-openai", "", catalog)
|
||||
if err != nil {
|
||||
t.Fatalf("ResolveProtocolProfile: %v", err)
|
||||
}
|
||||
if got, ok := profile.MapReasoningEffort(config.OperationResponses, "max", true); !ok || got != "xhigh" {
|
||||
t.Fatalf("mapped max = %q,%t; want xhigh,true", got, ok)
|
||||
}
|
||||
|
||||
bad := config.BuiltInProtocolProfileCatalog()
|
||||
bad["bad-effort"] = config.ProtocolProfileConf{
|
||||
Base: "openai",
|
||||
Normalization: config.ProtocolNormalizationConf{Effort: map[string]config.ProtocolEffortMappingConf{
|
||||
"messages": {Wire: config.ProtocolEffortWireAnthropicMessage, Levels: map[string]string{"high": "high"}},
|
||||
}},
|
||||
}
|
||||
if _, err := config.ResolveProtocolProfile("bad-effort", "", bad); err == nil || !strings.Contains(err.Error(), "not declared") {
|
||||
t.Fatalf("expected undeclared effort operation error, got %v", err)
|
||||
}
|
||||
|
||||
badUpgrade := config.BuiltInProtocolProfileCatalog()
|
||||
badUpgrade["bad-upgrade"] = config.ProtocolProfileConf{
|
||||
Base: "openai",
|
||||
Normalization: config.ProtocolNormalizationConf{Effort: map[string]config.ProtocolEffortMappingConf{
|
||||
string(config.OperationResponses): {
|
||||
WithTools: true,
|
||||
Wire: config.ProtocolEffortWireOpenAIResponses,
|
||||
Levels: map[string]string{"high": "max"},
|
||||
},
|
||||
}},
|
||||
}
|
||||
if _, err := config.ResolveProtocolProfile("bad-upgrade", "", badUpgrade); err == nil || !strings.Contains(err.Error(), "upgrades") {
|
||||
t.Fatalf("expected effort upgrade error, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestProtocolProfileOverlayRejected(t *testing.T) {
|
||||
t.Run("cycle", func(t *testing.T) {
|
||||
catalog := map[string]config.ProtocolProfileConf{
|
||||
|
|
|
|||
Loading…
Reference in a new issue