From 46bd8563e2132dc571cc86d1f71ff96f22d31777 Mon Sep 17 00:00:00 2001 From: toki Date: Sat, 27 Jun 2026 05:16:25 +0900 Subject: [PATCH] =?UTF-8?q?feat:=20OpenAI=20compatible=20API=20=EB=B3=80?= =?UTF-8?q?=EA=B2=BD=20=EC=82=AC=ED=95=AD=20=EC=A0=81=EC=9A=A9?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - agent-contract: OpenAI compatible API 계약 문서 업데이트 - apps/edge: OpenAI API smoke/test 스크립트 및 핸들러 개선 - scripts: e2e 테스트 스크립트 업데이트 --- agent-contract/index.md | 2 +- .../provided/openai-compatible-api.md | 87 +++++- apps/edge/README.md | 8 +- apps/edge/internal/edgecmd/smoke_openai.go | 4 - apps/edge/internal/openai/chat_handler.go | 55 +++- .../edge/internal/openai/responses_handler.go | 101 +++++-- apps/edge/internal/openai/server_test.go | 259 +++++++++--------- apps/edge/internal/openai/types.go | 76 +---- apps/edge/internal/service/service_test.go | 14 +- scripts/e2e-openai-cli-workspace.sh | 2 +- scripts/e2e-openai-lemonade.sh | 4 +- scripts/e2e-openai-ollama.sh | 2 +- scripts/e2e-openai-vllm.sh | 4 +- 13 files changed, 347 insertions(+), 271 deletions(-) diff --git a/agent-contract/index.md b/agent-contract/index.md index 458f3d2..2f8d158 100644 --- a/agent-contract/index.md +++ b/agent-contract/index.md @@ -12,7 +12,7 @@ | id | 읽는 조건 | path | |----|-----------|------| -| `iop.openai-compatible-api` | OpenAI-compatible API, Responses API, Chat Completions, `model` route, Codex/CLI workspace, NomadCode authoring metadata, `metadata.workspace`, `metadata.task_id`, `metadata.source` | `agent-contract/provided/openai-compatible-api.md` | +| `iop.openai-compatible-api` | OpenAI-compatible API, Responses API, Chat Completions, legacy Completions, `model` route, Codex/CLI workspace, generic authoring metadata, `metadata.workspace`, `metadata.task_id` | `agent-contract/provided/openai-compatible-api.md` | ## 소비 계약 diff --git a/agent-contract/provided/openai-compatible-api.md b/agent-contract/provided/openai-compatible-api.md index fffa253..da3ba8f 100644 --- a/agent-contract/provided/openai-compatible-api.md +++ b/agent-contract/provided/openai-compatible-api.md @@ -44,52 +44,83 @@ CLI agent 실행으로 라우팅되는 요청의 최소 형태: } ``` +현재 `/v1/responses`에서 허용하는 표준형 요청 예시: + +```json +{ + "model": "codex", + "instructions": "응답은 짧게 작성해.", + "input": "현재 워크스페이스의 테스트 상태를 확인해줘.", + "stream": false, + "background": false, + "max_output_tokens": 4096, + "temperature": 0, + "top_p": 1, + "metadata": { + "workspace": "/config/workspace/iop", + "request_id": "req-001", + "task_id": "task-123" + } +} +``` + 필드 의미: -- `model`: Edge가 내부 `adapter + target`으로 해석할 외부 route 이름이다. -- `input`: agent에게 전달할 사용자 요청이다. +- `model`: Edge가 내부 `adapter + target`으로 해석할 외부 route 이름이다. IOP Edge에서는 라우팅을 위해 필수다. +- `instructions`: OpenAI Responses API의 top-level instruction field다. 있으면 `input` 앞에 배치해 agent 실행 prompt를 만든다. +- `input`: agent에게 전달할 사용자 요청이다. 현재 구현은 string input만 지원한다. +- `stream`: 현재 구현은 `false` 또는 생략만 지원한다. +- `background`: 현재 구현은 `false` 또는 생략만 지원한다. - `metadata.workspace`: CLI process를 실행할 작업 디렉터리다. CLI agent route에서는 필수 실행 문맥이다. +- `metadata`: OpenAI 표준 metadata container다. string key/value를 허용하고, IOP는 `workspace`만 실행 문맥으로 해석한다. 나머지 key는 caller-defined metadata로 보존하되 `source`는 지원하지 않는다. +- `metadata.request_id`, `metadata.task_id`: caller-defined metadata 예시다. 특별한 wrapper나 제품 전용 field가 아니다. +- `max_output_tokens`: 출력 길이 상한이다. 내부 provider option의 `max_tokens`로 전달된다. +- `temperature`: 생성 다양성 option이다. 대상 adapter가 지원하지 않으면 무시될 수 있다. +- `top_p`: nucleus sampling option이다. 대상 adapter가 지원하지 않으면 무시될 수 있다. 금지: - `metadata.cli` 같은 CLI 전용 wrapper를 추가하지 않는다. +- `metadata.inference`처럼 `model` route와 겹치는 target wrapper를 추가하지 않는다. +- `metadata.nomadcode`처럼 특정 소비자 제품명에 묶인 wrapper를 추가하지 않는다. +- `metadata.source`처럼 의미가 불명확한 호출 출처 field를 추가하지 않는다. - root-level `iop` 같은 별도 wrapper field를 추가하지 않는다. +- `/v1/responses`에 `options` wrapper를 추가하지 않는다. Responses API option은 OpenAI 표준 top-level field를 따른다. +- `session_id`, `timeout_sec` 같은 IOP 실행 제어 field를 request body 계약에 추가하지 않는다. logical session과 timeout은 route/config 기본값을 따른다. - workspace를 prompt 본문에 섞어 전달하지 않는다. 현재 구현 메모: - `/v1/responses`는 non-streaming 요청만 지원한다. -- 기존 metadata 계약인 `metadata.request_id`, `metadata.nomadcode.task_id`, `metadata.nomadcode.source`, `metadata.inference.target`은 유지한다. +- `metadata`는 최대 16개 string key/value를 허용한다. key는 64자 이하, value는 512자 이하를 기준으로 한다. - CLI route의 `metadata.workspace`는 이 문서의 계약 기준이다. 구현은 이 값을 Edge service의 run workspace와 Node CLI adapter의 process working directory로 전달해야 한다. +- `metadata.workspace`는 `RunRequest.Workspace`로 전달하고 generic run metadata에는 복사하지 않는다. +- 다른 Responses API 표준 field는 구현 필요가 생길 때 계약을 갱신한 뒤 추가한다. -## NomadCode Authoring Handoff +## Generic Authoring Handoff -NomadCode Core가 IOP Edge HTTP 표면으로 workspace authoring 작업을 넘길 때의 최소 요청 형태: +외부 caller가 IOP Edge HTTP 표면으로 workspace authoring 작업을 넘길 때의 최소 요청 형태: ```json { "model": "codex", "input": "Todo 항목에 필요한 산출물을 현재 checkout에 작성해줘.", "metadata": { - "workspace": "/config/workspace/nomadcode-slot-123", - "task_id": "todo-123", - "source": "nomadcode" + "workspace": "/config/workspace/work-slot-123", + "task_id": "todo-123" } } ``` -NomadCode task/source context는 flat `metadata.task_id`와 `metadata.source`로 전달할 수 있다. -동일한 의미의 structured 형태가 필요하면 `metadata.nomadcode.task_id`와 `metadata.nomadcode.source`를 사용하며, flat alias와 structured 값이 함께 있으면 structured 값을 우선한다. - -이 handoff는 `model`, `input`, `metadata.workspace`, task/source metadata만으로 충분해야 한다. -호출자는 `metadata.cli`, root-level `iop` wrapper, IOP CLI 직접 실행, prompt 본문 workspace 주입을 요구받지 않는다. +이 handoff는 `model`, `input`, `metadata.workspace`, 필요한 caller-defined metadata만으로 충분해야 한다. +호출자는 `metadata.cli`, 소비자 전용 metadata wrapper, root-level `iop` wrapper, IOP CLI 직접 실행, prompt 본문 workspace 주입을 요구받지 않는다. Workspace-bound route는 workspace가 없거나 상대 경로이면 OpenAI-compatible error로 거부한다. 존재하지 않는 경로, 권한 오류, agent process exit failure는 기본 cwd fallback으로 숨기지 않고 호출자가 실패로 구분할 수 있어야 한다. ## Chat Completions -`/v1/chat/completions`도 같은 metadata 원칙을 따른다. CLI route의 workspace는 `metadata.workspace`에 둔다. +`/v1/chat/completions`도 같은 metadata 원칙을 따른다. CLI route의 workspace는 `metadata.workspace`에 둔다. Chat Completions의 provider sampling option은 해당 endpoint의 OpenAI-compatible top-level request field를 따르며, `/v1/responses`와 마찬가지로 별도 `options` wrapper를 두지 않는다. ```json { @@ -106,6 +137,34 @@ Workspace-bound route는 workspace가 없거나 상대 경로이면 OpenAI-compa } ``` +현재 지원하는 Chat Completions request field: + +- `model` +- `messages` +- `stream` +- `metadata` +- `max_tokens` +- `max_completion_tokens` +- `temperature` +- `top_p` +- `presence_penalty` +- `frequency_penalty` +- `seed` +- `stop` +- `response_format` +- `tools` + +금지: + +- `metadata.source`, `metadata.cli`, `metadata.inference`, `metadata.nomadcode` +- `options`, `think`, `format`, `keep_alive` 같은 provider/Ollama 전용 request field +- `session_id`, `timeout_sec` 같은 IOP 실행 제어 field + +## Legacy Completions + +`POST /v1/completions`는 현재 IOP Edge OpenAI-compatible 표면에서 제공하지 않는다. +text completion 형태의 신규 호출은 `/v1/responses`를 사용하고, message 기반 호출은 `/v1/chat/completions`를 사용한다. + ## Routing Edge 설정이 `openai.model_routes[]`를 제공하면 `model`은 먼저 route catalog에서 해석된다. diff --git a/apps/edge/README.md b/apps/edge/README.md index eb677c8..93ad5bb 100644 --- a/apps/edge/README.md +++ b/apps/edge/README.md @@ -193,7 +193,7 @@ Edge 외부 입력은 OpenAI-compatible HTTP API와 A2A JSON-RPC HTTP API 두 이 표면은 외부 모델 클라이언트 호환을 위한 표준 경로다. Edge/Node 운영 제어, CLI logical session, background run, cancel/terminate-session, capabilities/status/session/transport command, node lifecycle event 같은 IOP 고유 기능은 OpenAI-compatible 요청에 억지로 싣지 않고 IOP native protocol(protobuf-socket) 계열에서 다룬다. -외부 프로젝트가 참조할 요청 계약 원문은 repo root의 `agent-contract/provided/openai-compatible-api.md`에 둔다. CLI agent route에서 사용할 workspace는 OpenAI request의 `metadata.workspace`에 둔다. NomadCode authoring handoff의 요청 shape도 같은 계약 원문을 기준으로 한다. +외부 프로젝트가 참조할 요청 계약 원문은 repo root의 `agent-contract/provided/openai-compatible-api.md`에 둔다. CLI agent route에서 사용할 workspace는 OpenAI request의 `metadata.workspace`에 둔다. Workspace authoring handoff의 요청 shape도 같은 계약 원문을 기준으로 한다. ```yaml openai: @@ -233,7 +233,7 @@ openai: target: "llama3:8b" ``` -`/v1/chat/completions`는 기본 non-streaming과 streaming SSE 응답을 모두 지원한다. `/v1/responses`는 현재 non-streaming 요청만 지원한다. 두 endpoint 모두 `metadata.workspace`를 run workspace로 전달하고, `metadata.request_id`, `metadata.inference.target`, `metadata.nomadcode.task_id`, `metadata.nomadcode.source`를 service boundary까지 전달한다. NomadCode compatibility alias로 root-level `metadata.task_id`와 `metadata.source`도 받아 각각 `nomadcode.task_id`, `nomadcode.source`로 매핑하며, structured `metadata.nomadcode.*` 값이 있으면 그 값을 우선한다. `metadata.cli`는 지원하지 않는다. +`/v1/chat/completions`는 기본 non-streaming과 streaming SSE 응답을 모두 지원한다. `/v1/responses`는 현재 non-streaming 요청만 지원한다. 두 endpoint 모두 `metadata.workspace`를 run workspace로 전달한다. `metadata`는 OpenAI 표준의 caller-defined string metadata container로 보고, IOP가 특별히 해석하는 key는 `workspace`뿐이다. `/v1/responses`의 `max_output_tokens`, `temperature`, `top_p`, `instructions`, `background`와 `/v1/chat/completions`의 `max_tokens`, `max_completion_tokens`, `temperature`, `top_p` 등은 OpenAI API처럼 top-level field에 둔다. `metadata.source`, `metadata.cli`, `metadata.inference`, 소비자 전용 metadata wrapper, `options`, `think`, `format`, `keep_alive` 같은 provider/Ollama 전용 request field는 지원하지 않는다. ```bash curl -s http://127.0.0.1:18081/v1/chat/completions \ @@ -244,10 +244,10 @@ curl -s http://127.0.0.1:18081/v1/chat/completions \ ```bash curl -s http://127.0.0.1:18081/v1/responses \ -H 'Content-Type: application/json' \ - -d '{"model":"qwen3.6:35b-a3b-bf16","input":"hello","stream":false,"metadata":{"request_id":"example","workspace":"/config/workspace/iop","nomadcode":{"task_id":"task-123","source":"manual"}}}' + -d '{"model":"qwen3.6:35b-a3b-bf16","instructions":"reply briefly","input":"hello","stream":false,"background":false,"max_output_tokens":256,"temperature":0,"top_p":1,"metadata":{"request_id":"example","workspace":"/config/workspace/iop","task_id":"task-123"}}' ``` -vLLM 같은 OpenAI-compatible inference server는 `openai_compat` adapter와 `provider: "vllm"` instance로 연결한다. 이 경로는 `/v1/models`, non-streaming/streaming `/v1/chat/completions`, provider header, request `options` 전달을 지원하며, Edge의 model route alias를 실제 served model로 매핑한다. +vLLM 같은 OpenAI-compatible inference server는 `openai_compat` adapter와 `provider: "vllm"` instance로 연결한다. 이 경로는 `/v1/models`, non-streaming/streaming `/v1/chat/completions`, provider header, 표준 top-level request field 전달을 지원하며, Edge의 model route alias를 실제 served model로 매핑한다. #### Model Provider Pool 호환 방향 diff --git a/apps/edge/internal/edgecmd/smoke_openai.go b/apps/edge/internal/edgecmd/smoke_openai.go index d00c0a0..74a5bca 100644 --- a/apps/edge/internal/edgecmd/smoke_openai.go +++ b/apps/edge/internal/edgecmd/smoke_openai.go @@ -243,10 +243,6 @@ If --base-url is not provided, it auto-discovers the OpenAI URL from the effecti metadata := map[string]interface{}{ "request_id": "iop-edge-smoke", "task_id": "smoke", - "source": "iop-edge-smoke", - "inference": map[string]string{ - "target": smokeModel, - }, } if smokeWorkspace != "" { metadata["workspace"] = smokeWorkspace diff --git a/apps/edge/internal/openai/chat_handler.go b/apps/edge/internal/openai/chat_handler.go index cb4f291..9113b00 100644 --- a/apps/edge/internal/openai/chat_handler.go +++ b/apps/edge/internal/openai/chat_handler.go @@ -24,17 +24,17 @@ func (s *Server) handleChatCompletions(w http.ResponseWriter, r *http.Request) { defer r.Body.Close() var req chatCompletionRequest - if err := json.NewDecoder(r.Body).Decode(&req); err != nil { - writeError(w, http.StatusBadRequest, "invalid_request_error", "invalid JSON request") + if err := decodeChatCompletionRequest(json.NewDecoder(r.Body), &req); err != nil { + writeError(w, http.StatusBadRequest, "invalid_request_error", err.Error()) return } - runMeta, inferenceTarget, workspace, err := parseOpenAIMetadata(req.Metadata) + runMeta, workspace, err := parseOpenAIMetadata(req.Metadata) if err != nil { writeError(w, http.StatusBadRequest, "invalid_request_error", err.Error()) return } - dispatch, ok := s.resolveRouteDispatch(req.Model, inferenceTarget) + dispatch, ok := s.resolveRouteDispatch(req.Model) if !ok { writeError(w, http.StatusBadRequest, "invalid_request_error", "model is required") return @@ -98,11 +98,44 @@ func (s *Server) handleChatCompletions(w http.ResponseWriter, r *http.Request) { s.completeChatCompletion(w, r, req, handle, outputPolicy) } +func decodeChatCompletionRequest(dec *json.Decoder, req *chatCompletionRequest) error { + var raw map[string]json.RawMessage + if err := dec.Decode(&raw); err != nil { + return fmt.Errorf("invalid JSON request") + } + for key := range raw { + switch key { + case "model", "messages", "stream", "metadata", "max_tokens", "max_completion_tokens", "temperature", "top_p", "presence_penalty", "frequency_penalty", "seed", "stop", "response_format", "tools": + default: + return fmt.Errorf("%s is not supported for /v1/chat/completions", key) + } + } + normalized, err := json.Marshal(raw) + if err != nil { + return fmt.Errorf("invalid JSON request") + } + if err := json.Unmarshal(normalized, req); err != nil { + return fmt.Errorf("invalid /v1/chat/completions request format") + } + if req.MaxTokens != nil && *req.MaxTokens <= 0 { + return fmt.Errorf("max_tokens must be greater than zero") + } + if req.MaxCompletionTokens != nil && *req.MaxCompletionTokens <= 0 { + return fmt.Errorf("max_completion_tokens must be greater than zero") + } + if req.Temperature != nil && (*req.Temperature < 0 || *req.Temperature > 2) { + return fmt.Errorf("temperature must be between 0 and 2") + } + if req.TopP != nil && (*req.TopP < 0 || *req.TopP > 1) { + return fmt.Errorf("top_p must be between 0 and 1") + } + return nil +} + func chatRunMetadata(runMeta map[string]string, req chatCompletionRequest, outputPolicy strictOutputPolicy) map[string]string { if runMeta == nil { runMeta = make(map[string]string) } - runMeta["source"] = "openai" runMeta["openai_model"] = req.Model runMeta["openai_stream"] = fmt.Sprintf("%t", req.Stream) runMeta["strict_output"] = fmt.Sprintf("%t", outputPolicy.Strict) @@ -149,16 +182,9 @@ func (s *Server) resolveAdapter() string { } func (s *Server) resolveTarget(model string) string { - return s.resolveTargetWithOverride(model, "") -} - -func (s *Server) resolveTargetWithOverride(model, override string) string { if s.cfg.Target != "" { return s.cfg.Target } - if override != "" { - return strings.TrimSpace(override) - } return strings.TrimSpace(model) } @@ -212,9 +238,8 @@ func (s *Server) findProviderPoolEntry(model string) *config.ModelCatalogEntry { // resolveRouteDispatch returns fully-resolved dispatch params for model. // Priority: provider-pool catalog → legacy model_routes → single-target fallback. -// metadataTarget is only used in the legacy fallback path. // Returns (dispatch, true) on success; (zero, false) when no target can be resolved. -func (s *Server) resolveRouteDispatch(model, metadataTarget string) (routeDispatch, bool) { +func (s *Server) resolveRouteDispatch(model string) (routeDispatch, bool) { // Provider-pool catalog takes highest priority. if s.findProviderPoolEntry(model) != nil { return routeDispatch{ @@ -252,7 +277,7 @@ func (s *Server) resolveRouteDispatch(model, metadataTarget string) (routeDispat WorkspaceRequired: route.WorkspaceRequired, }, true } - target := s.resolveTargetWithOverride(model, metadataTarget) + target := s.resolveTarget(model) if target == "" { return routeDispatch{}, false } diff --git a/apps/edge/internal/openai/responses_handler.go b/apps/edge/internal/openai/responses_handler.go index 2088040..6eb5bf3 100644 --- a/apps/edge/internal/openai/responses_handler.go +++ b/apps/edge/internal/openai/responses_handler.go @@ -20,8 +20,8 @@ func (s *Server) handleResponses(w http.ResponseWriter, r *http.Request) { defer r.Body.Close() var req responsesRequest - if err := json.NewDecoder(r.Body).Decode(&req); err != nil { - writeError(w, http.StatusBadRequest, "invalid_request_error", "invalid JSON request") + if err := decodeResponsesRequest(json.NewDecoder(r.Body), &req); err != nil { + writeError(w, http.StatusBadRequest, "invalid_request_error", err.Error()) return } @@ -29,6 +29,10 @@ func (s *Server) handleResponses(w http.ResponseWriter, r *http.Request) { writeError(w, http.StatusBadRequest, "invalid_request_error", "streaming is not supported for /v1/responses") return } + if req.Background { + writeError(w, http.StatusBadRequest, "invalid_request_error", "background is not supported for /v1/responses") + return + } inputStr, err := parseResponsesInput(req.Input) if err != nil { @@ -42,13 +46,13 @@ func (s *Server) handleResponses(w http.ResponseWriter, r *http.Request) { prompt = instruction + "\n" + prompt } - runMeta, inferenceTarget, workspace, err := parseOpenAIMetadata(req.Metadata) + runMeta, workspace, err := parseOpenAIMetadata(req.Metadata) if err != nil { writeError(w, http.StatusBadRequest, "invalid_request_error", err.Error()) return } - dispatch, ok := s.resolveRouteDispatch(req.Model, inferenceTarget) + dispatch, ok := s.resolveRouteDispatch(req.Model) if !ok { writeError(w, http.StatusBadRequest, "invalid_request_error", "model is required") return @@ -58,7 +62,6 @@ func (s *Server) handleResponses(w http.ResponseWriter, r *http.Request) { return } - runMeta["source"] = "openai-responses" runMeta["openai_model"] = req.Model runMeta["openai_stream"] = fmt.Sprintf("%t", req.Stream) runMeta["strict_output"] = fmt.Sprintf("%t", outputPolicy.Strict) @@ -105,6 +108,37 @@ func (s *Server) handleResponses(w http.ResponseWriter, r *http.Request) { s.completeResponse(w, r, req, handle, outputPolicy) } +func decodeResponsesRequest(dec *json.Decoder, req *responsesRequest) error { + var raw map[string]json.RawMessage + if err := dec.Decode(&raw); err != nil { + return fmt.Errorf("invalid JSON request") + } + for key := range raw { + switch key { + case "model", "input", "instructions", "stream", "background", "metadata", "max_output_tokens", "temperature", "top_p": + default: + return fmt.Errorf("%s is not supported for /v1/responses", key) + } + } + normalized, err := json.Marshal(raw) + if err != nil { + return fmt.Errorf("invalid JSON request") + } + if err := json.Unmarshal(normalized, req); err != nil { + return fmt.Errorf("invalid /v1/responses request format") + } + if req.MaxOutputTokens != nil && *req.MaxOutputTokens <= 0 { + return fmt.Errorf("max_output_tokens must be greater than zero") + } + if req.Temperature != nil && (*req.Temperature < 0 || *req.Temperature > 2) { + return fmt.Errorf("temperature must be between 0 and 2") + } + if req.TopP != nil && (*req.TopP < 0 || *req.TopP > 1) { + return fmt.Errorf("top_p must be between 0 and 1") + } + return nil +} + func (s *Server) completeResponse(w http.ResponseWriter, r *http.Request, req responsesRequest, handle edgeservice.RunResult, outputPolicy strictOutputPolicy) { text, reasoning, _, usage, err := collectRunResult(r.Context(), handle.Stream(), handle.WaitTimeout()) if err != nil { @@ -161,37 +195,60 @@ func parseResponsesInput(raw json.RawMessage) (string, error) { func buildResponsesPrompt(instructions, input string) string { instructions = strings.TrimSpace(instructions) - input = strings.TrimSpace(input) if instructions == "" { return input } - return instructions + "\n" + input + return instructions + "\n\n" + input } -func parseOpenAIMetadata(raw json.RawMessage) (map[string]string, string, string, error) { +func parseOpenAIMetadata(raw json.RawMessage) (map[string]string, string, error) { if len(raw) == 0 || string(raw) == "null" { - return make(map[string]string), "", "", nil + return make(map[string]string), "", nil } var rawMap map[string]json.RawMessage if err := json.Unmarshal(raw, &rawMap); err != nil { - return nil, "", "", fmt.Errorf("metadata must be an object") + return nil, "", fmt.Errorf("metadata must be an object") } - if _, hasCLI := rawMap["cli"]; hasCLI { - return nil, "", "", fmt.Errorf("metadata.cli is not supported") + if len(rawMap) > 16 { + return nil, "", fmt.Errorf("metadata must contain at most 16 keys") } - var meta responsesMetadata - if err := json.Unmarshal(raw, &meta); err != nil { - return nil, "", "", fmt.Errorf("invalid metadata format") + flat := make(map[string]string, len(rawMap)) + var workspace string + for key, rawValue := range rawMap { + if len(key) > 64 { + return nil, "", fmt.Errorf("metadata key %q exceeds 64 characters", key) + } + if key == "source" { + return nil, "", fmt.Errorf("metadata.source is not supported") + } + if key == "workspace" { + workspaceValue, err := metadataStringValue(key, rawValue) + if err != nil { + return nil, "", err + } + workspace = strings.TrimSpace(workspaceValue) + continue + } + value, err := metadataStringValue(key, rawValue) + if err != nil { + return nil, "", err + } + flat[key] = value } - flat := meta.metadataForRun() - inferenceTarget := "" - if meta.Inference != nil { - inferenceTarget = meta.Inference.Target - } - - return flat, inferenceTarget, strings.TrimSpace(meta.Workspace), nil + return flat, workspace, nil +} + +func metadataStringValue(key string, raw json.RawMessage) (string, error) { + var value string + if err := json.Unmarshal(raw, &value); err != nil { + return "", fmt.Errorf("metadata.%s must be a string", key) + } + if len(value) > 512 { + return "", fmt.Errorf("metadata.%s exceeds 512 characters", key) + } + return value, nil } diff --git a/apps/edge/internal/openai/server_test.go b/apps/edge/internal/openai/server_test.go index ad8fc7e..83f8eec 100644 --- a/apps/edge/internal/openai/server_test.go +++ b/apps/edge/internal/openai/server_test.go @@ -184,7 +184,7 @@ func TestChatCompletionsPreservesProviderFinishReason(t *testing.T) { } } -func TestChatCompletionsPassesOllamaOptions(t *testing.T) { +func TestChatCompletionsPassesStandardOptions(t *testing.T) { fake := &fakeRunService{events: make(chan *iop.RunEvent, 2)} fake.events <- &iop.RunEvent{Type: "delta", Delta: "ok"} fake.events <- &iop.RunEvent{Type: "complete"} @@ -193,12 +193,11 @@ func TestChatCompletionsPassesOllamaOptions(t *testing.T) { req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{ "model":"from-request", "messages":[{"role":"user","content":"hi"}], - "options":{"temperature":0.2,"top_p":0.9,"num_predict":32,"stop":["END"]}, "max_tokens":12, "temperature":0.1, - "keep_alive":"10m", - "think":false, - "format":"json", + "top_p":0.9, + "stop":["END"], + "response_format":{"type":"json_object"}, "tools":[{"type":"function","function":{"name":"lookup"}}] }`)) w := httptest.NewRecorder() @@ -212,14 +211,17 @@ func TestChatCompletionsPassesOllamaOptions(t *testing.T) { if !ok { t.Fatalf("options not passed: %+v", fake.req.Input) } - if options["temperature"].(float64) != 0.1 || options["top_p"].(float64) != 0.9 || options["num_predict"].(float64) != 32 { + if options["temperature"].(float64) != 0.1 || options["top_p"].(float64) != 0.9 { t.Fatalf("unexpected options: %+v", options) } if options["max_tokens"].(int) != 12 { t.Fatalf("max_tokens not passed: %+v", options) } - if fake.req.Input["keep_alive"] != "10m" || fake.req.Input["think"] != false || fake.req.Input["format"] != "json" { - t.Fatalf("top-level ollama fields not passed: %+v", fake.req.Input) + if _, ok := options["stop"]; !ok { + t.Fatalf("stop not passed: %+v", options) + } + if _, ok := options["response_format"]; !ok { + t.Fatalf("response_format not passed: %+v", options) } if tools, ok := fake.req.Input["tools"].([]any); !ok || len(tools) != 1 { t.Fatalf("tools not passed: %+v", fake.req.Input["tools"]) @@ -545,7 +547,7 @@ func TestOpenAIProviderPoolRouteUsesRefreshedCatalog(t *testing.T) { ID: "model-new", Providers: map[string]string{"prov-a": "served-a"}, }}) - dispatch, ok := srv.resolveRouteDispatch("model-new", "") + dispatch, ok := srv.resolveRouteDispatch("model-new") if !ok || !dispatch.ProviderPool { t.Fatalf("expected provider-pool dispatch for refreshed model, got ok=%v dispatch=%+v", ok, dispatch) } @@ -608,14 +610,19 @@ func TestResponsesDispatchesNonStreamingRequest(t *testing.T) { fake.events <- &iop.RunEvent{Type: "complete", Usage: &iop.Usage{InputTokens: 1, OutputTokens: 1}} srv := NewServer(config.EdgeOpenAIConf{ - Adapter: "ollama", - Target: "llama-fixed", + Adapter: "ollama", + Target: "llama-fixed", + SessionID: "configured-session", + TimeoutSec: 42, }, fake, nil) req := httptest.NewRequest(http.MethodPost, "/v1/responses", strings.NewReader(`{ "model":"client-model", "input":"say hello", + "instructions":"Use short answers.", "max_output_tokens":16, - "temperature":0.2 + "temperature":0.2, + "top_p":0.9, + "background":false }`)) w := httptest.NewRecorder() @@ -627,16 +634,25 @@ func TestResponsesDispatchesNonStreamingRequest(t *testing.T) { if fake.req.Target != "llama-fixed" { t.Fatalf("target: got %q, want llama-fixed", fake.req.Target) } - if fake.req.Prompt != "say hello" { + if fake.req.Prompt != "Use short answers.\n\nsay hello" { t.Fatalf("prompt: got %q", fake.req.Prompt) } options, ok := fake.req.Input["options"].(map[string]any) if !ok { t.Fatalf("options not passed: %+v", fake.req.Input) } - if options["max_tokens"].(int) != 16 || options["temperature"].(float64) != 0.2 { + if options["max_tokens"].(int) != 16 || options["temperature"].(float64) != 0.2 || options["top_p"].(float64) != 0.9 { t.Fatalf("unexpected response options: %+v", options) } + if fake.req.TimeoutSec != 42 { + t.Fatalf("timeout: got %d, want 42", fake.req.TimeoutSec) + } + if fake.req.SessionID != "configured-session" { + t.Fatalf("session_id: got %q, want configured-session", fake.req.SessionID) + } + if fake.req.Background { + t.Fatal("background should remain false for responses") + } var resp responsesResponse if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil { @@ -663,6 +679,10 @@ func TestResponsesRejectsUnsupportedRequests(t *testing.T) { {"stream=true", http.MethodPost, `{"model":"m","input":"hi","stream":true}`, http.StatusBadRequest}, {"empty input", http.MethodPost, `{"model":"m","input":""}`, http.StatusBadRequest}, {"non-string input", http.MethodPost, `{"model":"m","input":["a","b"]}`, http.StatusBadRequest}, + {"options wrapper unsupported", http.MethodPost, `{"model":"m","input":"hi","options":{"max_output_tokens":16}}`, http.StatusBadRequest}, + {"background unsupported", http.MethodPost, `{"model":"m","input":"hi","background":true}`, http.StatusBadRequest}, + {"bad max_output_tokens", http.MethodPost, `{"model":"m","input":"hi","max_output_tokens":0}`, http.StatusBadRequest}, + {"bad top_p", http.MethodPost, `{"model":"m","input":"hi","top_p":1.5}`, http.StatusBadRequest}, } for _, tc := range cases { @@ -677,7 +697,7 @@ func TestResponsesRejectsUnsupportedRequests(t *testing.T) { } } -func TestResponsesMetadataContractAndTargetOverride(t *testing.T) { +func TestResponsesGenericMetadataContract(t *testing.T) { fake := &fakeRunService{events: make(chan *iop.RunEvent, 2)} fake.events <- &iop.RunEvent{Type: "delta", Delta: "ok"} fake.events <- &iop.RunEvent{Type: "complete"} @@ -689,8 +709,8 @@ func TestResponsesMetadataContractAndTargetOverride(t *testing.T) { "metadata":{ "request_id":"req-001", "workspace":"/config/workspace/iop", - "inference":{"target":"metadata-target"}, - "nomadcode":{"task_id":"task-123","source":"nomadcode"} + "task_id":"task-123", + "custom":"value" } }`)) w := httptest.NewRecorder() @@ -699,8 +719,8 @@ func TestResponsesMetadataContractAndTargetOverride(t *testing.T) { if w.Code != http.StatusOK { t.Fatalf("status: got %d body=%s", w.Code, w.Body.String()) } - if fake.req.Target != "metadata-target" { - t.Fatalf("target: got %q, want metadata-target", fake.req.Target) + if fake.req.Target != "client-model" { + t.Fatalf("target: got %q, want client-model", fake.req.Target) } if fake.req.Workspace != "/config/workspace/iop" { t.Fatalf("workspace: got %q", fake.req.Workspace) @@ -711,17 +731,11 @@ func TestResponsesMetadataContractAndTargetOverride(t *testing.T) { if _, ok := fake.req.Metadata["workspace"]; ok { t.Fatal("workspace should not be copied into run metadata") } - if fake.req.Metadata["inference.target"] != "metadata-target" { - t.Fatalf("inference.target: got %q", fake.req.Metadata["inference.target"]) + if fake.req.Metadata["task_id"] != "task-123" { + t.Fatalf("task_id: got %q", fake.req.Metadata["task_id"]) } - if fake.req.Metadata["nomadcode.task_id"] != "task-123" { - t.Fatalf("nomadcode.task_id: got %q", fake.req.Metadata["nomadcode.task_id"]) - } - if fake.req.Metadata["nomadcode.source"] != "nomadcode" { - t.Fatalf("nomadcode.source: got %q", fake.req.Metadata["nomadcode.source"]) - } - if fake.req.Metadata["source"] != "openai-responses" { - t.Fatalf("source: got %q", fake.req.Metadata["source"]) + if fake.req.Metadata["custom"] != "value" { + t.Fatalf("custom: got %q", fake.req.Metadata["custom"]) } if fake.req.Metadata["openai_model"] != "client-model" { t.Fatalf("openai_model: got %q", fake.req.Metadata["openai_model"]) @@ -749,8 +763,7 @@ func TestChatCompletionsMetadataContractAndWorkspace(t *testing.T) { "metadata":{ "request_id":"req-chat-001", "workspace":"/config/workspace/iop", - "inference":{"target":"metadata-target"}, - "nomadcode":{"task_id":"task-123","source":"nomadcode"} + "task_id":"task-123" } }`)) w := httptest.NewRecorder() @@ -759,8 +772,8 @@ func TestChatCompletionsMetadataContractAndWorkspace(t *testing.T) { if w.Code != http.StatusOK { t.Fatalf("status: got %d body=%s", w.Code, w.Body.String()) } - if fake.req.Target != "metadata-target" { - t.Fatalf("target: got %q, want metadata-target", fake.req.Target) + if fake.req.Target != "client-model" { + t.Fatalf("target: got %q, want client-model", fake.req.Target) } if fake.req.Workspace != "/config/workspace/iop" { t.Fatalf("workspace: got %q", fake.req.Workspace) @@ -768,17 +781,8 @@ func TestChatCompletionsMetadataContractAndWorkspace(t *testing.T) { if fake.req.Metadata["request_id"] != "req-chat-001" { t.Fatalf("request_id: got %q", fake.req.Metadata["request_id"]) } - if fake.req.Metadata["inference.target"] != "metadata-target" { - t.Fatalf("inference.target: got %q", fake.req.Metadata["inference.target"]) - } - if fake.req.Metadata["nomadcode.task_id"] != "task-123" { - t.Fatalf("nomadcode.task_id: got %q", fake.req.Metadata["nomadcode.task_id"]) - } - if fake.req.Metadata["nomadcode.source"] != "nomadcode" { - t.Fatalf("nomadcode.source: got %q", fake.req.Metadata["nomadcode.source"]) - } - if fake.req.Metadata["source"] != "openai" { - t.Fatalf("source: got %q", fake.req.Metadata["source"]) + if fake.req.Metadata["task_id"] != "task-123" { + t.Fatalf("task_id: got %q", fake.req.Metadata["task_id"]) } if fake.req.ModelGroupKey != "client-model" { t.Fatalf("model group key: got %q, want client-model", fake.req.ModelGroupKey) @@ -788,7 +792,7 @@ func TestChatCompletionsMetadataContractAndWorkspace(t *testing.T) { } } -func TestChatCompletionsRejectsCLIMetadata(t *testing.T) { +func TestChatCompletionsRejectsObjectMetadata(t *testing.T) { srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "llama"}, &fakeRunService{}, nil) req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{ "model":"m", @@ -802,6 +806,65 @@ func TestChatCompletionsRejectsCLIMetadata(t *testing.T) { } } +func TestChatCompletionsRejectsUnsupportedFields(t *testing.T) { + srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "llama"}, &fakeRunService{}, nil) + cases := []struct { + name string + body string + }{ + {"options wrapper", `{"model":"m","messages":[{"role":"user","content":"hi"}],"options":{"num_ctx":8192}}`}, + {"think", `{"model":"m","messages":[{"role":"user","content":"hi"}],"think":false}`}, + {"format", `{"model":"m","messages":[{"role":"user","content":"hi"}],"format":"json"}`}, + {"keep_alive", `{"model":"m","messages":[{"role":"user","content":"hi"}],"keep_alive":"10m"}`}, + {"bad max_tokens", `{"model":"m","messages":[{"role":"user","content":"hi"}],"max_tokens":0}`}, + {"bad top_p", `{"model":"m","messages":[{"role":"user","content":"hi"}],"top_p":2}`}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(tc.body)) + w := httptest.NewRecorder() + srv.routes().ServeHTTP(w, req) + if w.Code != http.StatusBadRequest { + t.Fatalf("got %d want 400, body=%s", w.Code, w.Body.String()) + } + }) + } +} + +func TestChatCompletionsRejectsSourceMetadata(t *testing.T) { + srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "llama"}, &fakeRunService{}, nil) + req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{ + "model":"m", + "messages":[{"role":"user","content":"hi"}], + "metadata":{"source":"manual"} + }`)) + w := httptest.NewRecorder() + srv.routes().ServeHTTP(w, req) + if w.Code != http.StatusBadRequest { + t.Fatalf("got %d want 400, body=%s", w.Code, w.Body.String()) + } + if !strings.Contains(w.Body.String(), "metadata.source is not supported") { + t.Fatalf("expected source unsupported error, got %s", w.Body.String()) + } +} + +func TestChatCompletionsRejectsNonStringMetadataValue(t *testing.T) { + srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "llama"}, &fakeRunService{}, nil) + req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{ + "model":"m", + "messages":[{"role":"user","content":"hi"}], + "metadata":{"attempt":2} + }`)) + w := httptest.NewRecorder() + srv.routes().ServeHTTP(w, req) + if w.Code != http.StatusBadRequest { + t.Fatalf("got %d want 400, body=%s", w.Code, w.Body.String()) + } + if !strings.Contains(w.Body.String(), "metadata.attempt must be a string") { + t.Fatalf("expected string metadata error, got %s", w.Body.String()) + } +} + func TestChatCompletionsRejectsObjectWorkspaceMetadata(t *testing.T) { srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "llama"}, &fakeRunService{}, nil) req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{ @@ -819,12 +882,12 @@ func TestChatCompletionsRejectsObjectWorkspaceMetadata(t *testing.T) { if w.Code != http.StatusBadRequest { t.Fatalf("got %d want 400, body=%s", w.Code, w.Body.String()) } - if !strings.Contains(w.Body.String(), "invalid metadata format") { - t.Fatalf("expected 'invalid metadata format' error, got %s", w.Body.String()) + if !strings.Contains(w.Body.String(), "metadata.workspace must be a string") { + t.Fatalf("expected workspace string error, got %s", w.Body.String()) } } -func TestResponsesAcceptsFlatNomadCodeMetadataAliases(t *testing.T) { +func TestResponsesPreservesGenericTaskMetadata(t *testing.T) { fake := &fakeRunService{events: make(chan *iop.RunEvent, 2)} fake.events <- &iop.RunEvent{Type: "delta", Delta: "ok"} fake.events <- &iop.RunEvent{Type: "complete"} @@ -835,9 +898,7 @@ func TestResponsesAcceptsFlatNomadCodeMetadataAliases(t *testing.T) { "input":"test", "metadata":{ "request_id":"req-flat-001", - "task_id":"task-flat", - "source":"plane", - "inference":{"target":"metadata-target"} + "task_id":"task-flat" } }`)) w := httptest.NewRecorder() @@ -846,50 +907,12 @@ func TestResponsesAcceptsFlatNomadCodeMetadataAliases(t *testing.T) { if w.Code != http.StatusOK { t.Fatalf("status: got %d body=%s", w.Code, w.Body.String()) } - if fake.req.Metadata["nomadcode.task_id"] != "task-flat" { - t.Fatalf("nomadcode.task_id: got %q", fake.req.Metadata["nomadcode.task_id"]) - } - if fake.req.Metadata["nomadcode.source"] != "plane" { - t.Fatalf("nomadcode.source: got %q", fake.req.Metadata["nomadcode.source"]) - } - if fake.req.Metadata["source"] != "openai-responses" { - t.Fatalf("source should remain IOP-owned: got %q", fake.req.Metadata["source"]) + if fake.req.Metadata["task_id"] != "task-flat" { + t.Fatalf("task_id: got %q", fake.req.Metadata["task_id"]) } } -func TestResponsesStructuredNomadCodeMetadataWinsOverFlatAliases(t *testing.T) { - fake := &fakeRunService{events: make(chan *iop.RunEvent, 2)} - fake.events <- &iop.RunEvent{Type: "delta", Delta: "ok"} - fake.events <- &iop.RunEvent{Type: "complete"} - - srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama"}, fake, nil) - req := httptest.NewRequest(http.MethodPost, "/v1/responses", strings.NewReader(`{ - "model":"client-model", - "input":"test", - "metadata":{ - "task_id":"task-flat", - "source":"plane", - "nomadcode":{"task_id":"task-structured","source":"manual"} - } - }`)) - w := httptest.NewRecorder() - srv.routes().ServeHTTP(w, req) - - if w.Code != http.StatusOK { - t.Fatalf("status: got %d body=%s", w.Code, w.Body.String()) - } - if fake.req.Metadata["nomadcode.task_id"] != "task-structured" { - t.Fatalf("nomadcode.task_id: got %q", fake.req.Metadata["nomadcode.task_id"]) - } - if fake.req.Metadata["nomadcode.source"] != "manual" { - t.Fatalf("nomadcode.source: got %q", fake.req.Metadata["nomadcode.source"]) - } - if fake.req.Metadata["source"] != "openai-responses" { - t.Fatalf("source should remain IOP-owned: got %q", fake.req.Metadata["source"]) - } -} - -func TestResponsesConfiguredTargetWinsOverMetadataTarget(t *testing.T) { +func TestResponsesConfiguredTargetWinsOverRequestModel(t *testing.T) { fake := &fakeRunService{events: make(chan *iop.RunEvent, 2)} fake.events <- &iop.RunEvent{Type: "delta", Delta: "ok"} fake.events <- &iop.RunEvent{Type: "complete"} @@ -897,8 +920,7 @@ func TestResponsesConfiguredTargetWinsOverMetadataTarget(t *testing.T) { srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "config-target"}, fake, nil) req := httptest.NewRequest(http.MethodPost, "/v1/responses", strings.NewReader(`{ "model":"client-model", - "input":"test", - "metadata":{"inference":{"target":"metadata-target"}} + "input":"test" }`)) w := httptest.NewRecorder() srv.routes().ServeHTTP(w, req) @@ -919,7 +941,7 @@ func TestResponsesConfiguredTargetWinsOverMetadataTarget(t *testing.T) { } } -func TestResponsesRejectsCLIMetadata(t *testing.T) { +func TestResponsesRejectsNonStringMetadata(t *testing.T) { srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "llama"}, &fakeRunService{}, nil) cases := []struct { @@ -927,7 +949,11 @@ func TestResponsesRejectsCLIMetadata(t *testing.T) { body string }{ {"cli only", `{"model":"m","input":"hi","metadata":{"cli":{"flag":"x"}}}`}, - {"inference and cli", `{"model":"m","input":"hi","metadata":{"inference":{"target":"t"},"cli":{"flag":"x"}}}`}, + {"inference target", `{"model":"m","input":"hi","metadata":{"inference":{"target":"t"}}}`}, + {"nomadcode metadata", `{"model":"m","input":"hi","metadata":{"nomadcode":{"task_id":"t"}}}`}, + {"source metadata", `{"model":"m","input":"hi","metadata":{"source":"manual"}}`}, + {"number metadata", `{"model":"m","input":"hi","metadata":{"attempt":2}}`}, + {"boolean metadata", `{"model":"m","input":"hi","metadata":{"urgent":true}}`}, } for _, tc := range cases { @@ -959,12 +985,12 @@ func TestResponsesRejectsObjectWorkspaceMetadata(t *testing.T) { if w.Code != http.StatusBadRequest { t.Fatalf("got %d want 400, body=%s", w.Code, w.Body.String()) } - if !strings.Contains(w.Body.String(), "invalid metadata format") { - t.Fatalf("expected 'invalid metadata format' error, got %s", w.Body.String()) + if !strings.Contains(w.Body.String(), "metadata.workspace must be a string") { + t.Fatalf("expected workspace string error, got %s", w.Body.String()) } } -func TestResponsesReturnsNomadCodeCompatibleShape(t *testing.T) { +func TestResponsesReturnsOpenAICompatibleShape(t *testing.T) { fake := &fakeRunService{events: make(chan *iop.RunEvent, 3)} fake.events <- &iop.RunEvent{Type: "delta", Delta: "answer"} fake.events <- &iop.RunEvent{Type: "complete", Usage: &iop.Usage{InputTokens: 1, OutputTokens: 1}} @@ -1041,8 +1067,7 @@ func TestResponsesStrictOutputNormalizesAgentResponse(t *testing.T) { srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "llama", StrictOutput: true}, fake, nil) req := httptest.NewRequest(http.MethodPost, "/v1/responses", strings.NewReader(`{ "model":"client-model", - "instructions":"Once you've completed the user's task, you must use the attempt_completion tool to present the result.\n\n\ndone\n", - "input":"finish" + "input":"Once you've completed the user's task, you must use the attempt_completion tool to present the result.\n\n\ndone\n\n\nfinish" }`)) w := httptest.NewRecorder() srv.routes().ServeHTTP(w, req) @@ -1272,7 +1297,7 @@ func TestResponsesRouteCatalogDispatchesRoute(t *testing.T) { } } -func TestResponsesRouteCatalogTakesPriorityOverMetadataTarget(t *testing.T) { +func TestResponsesRouteCatalogDispatchesQueuePolicy(t *testing.T) { fake := &fakeRunService{events: make(chan *iop.RunEvent, 2)} fake.events <- &iop.RunEvent{Type: "delta", Delta: "ok"} fake.events <- &iop.RunEvent{Type: "complete"} @@ -1284,8 +1309,7 @@ func TestResponsesRouteCatalogTakesPriorityOverMetadataTarget(t *testing.T) { }, fake, nil) req := httptest.NewRequest(http.MethodPost, "/v1/responses", strings.NewReader(`{ "model":"model-a", - "input":"test", - "metadata":{"inference":{"target":"metadata-target"}} + "input":"test" }`)) w := httptest.NewRecorder() srv.routes().ServeHTTP(w, req) @@ -1304,14 +1328,8 @@ func TestResponsesRouteCatalogTakesPriorityOverMetadataTarget(t *testing.T) { } } -func TestChatCompletionsEdgePassesNumCtxForAdapterEnforcement(t *testing.T) { - // Edge passes request options.num_ctx to the service unchanged. - // The Ollama adapter enforces Edge-owned context_size over this value. - fake := &fakeRunService{events: make(chan *iop.RunEvent, 2)} - fake.events <- &iop.RunEvent{Type: "delta", Delta: "ok"} - fake.events <- &iop.RunEvent{Type: "complete"} - - srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "llama"}, fake, nil) +func TestChatCompletionsRejectsNumCtxOptionsWrapper(t *testing.T) { + srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "llama"}, &fakeRunService{}, nil) req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{ "model":"from-request", "messages":[{"role":"user","content":"hi"}], @@ -1320,15 +1338,8 @@ func TestChatCompletionsEdgePassesNumCtxForAdapterEnforcement(t *testing.T) { w := httptest.NewRecorder() srv.handleChatCompletions(w, req) - if w.Code != http.StatusOK { - t.Fatalf("status: got %d body=%s", w.Code, w.Body.String()) - } - options, ok := fake.req.Input["options"].(map[string]any) - if !ok { - t.Fatalf("options not passed to service: %+v", fake.req.Input) - } - if options["num_ctx"].(float64) != 8192 { - t.Fatalf("edge should pass num_ctx unchanged for adapter enforcement: got %v", options["num_ctx"]) + if w.Code != http.StatusBadRequest { + t.Fatalf("got %d want 400, body=%s", w.Code, w.Body.String()) } } @@ -1340,7 +1351,7 @@ func TestResolveRouteDispatchPreservesWorkspaceRequired(t *testing.T) { }, }, &fakeRunService{}, nil) - dispatch, ok := srv.resolveRouteDispatch("codex", "") + dispatch, ok := srv.resolveRouteDispatch("codex") if !ok { t.Fatal("expected dispatch to succeed for codex route") } @@ -1348,7 +1359,7 @@ func TestResolveRouteDispatchPreservesWorkspaceRequired(t *testing.T) { t.Fatalf("expected workspace_required=true for codex route, got false") } - dispatch, ok = srv.resolveRouteDispatch("llama3", "") + dispatch, ok = srv.resolveRouteDispatch("llama3") if !ok { t.Fatal("expected dispatch to succeed for llama3 route") } @@ -1366,7 +1377,7 @@ func TestResolveRouteDispatchFallbackWorkspaceRequiredFalse(t *testing.T) { }, }, &fakeRunService{}, nil) - dispatch, ok := srv.resolveRouteDispatch("unknown-model", "") + dispatch, ok := srv.resolveRouteDispatch("unknown-model") if !ok { t.Fatal("expected fallback dispatch to succeed") } diff --git a/apps/edge/internal/openai/types.go b/apps/edge/internal/openai/types.go index 2b40e4c..508141e 100644 --- a/apps/edge/internal/openai/types.go +++ b/apps/edge/internal/openai/types.go @@ -10,7 +10,6 @@ type chatCompletionRequest struct { Messages []chatMessage `json:"messages"` Stream bool `json:"stream"` Metadata json.RawMessage `json:"metadata,omitempty"` - Options map[string]any `json:"options,omitempty"` MaxTokens *int `json:"max_tokens,omitempty"` MaxCompletionTokens *int `json:"max_completion_tokens,omitempty"` Temperature *float64 `json:"temperature,omitempty"` @@ -20,9 +19,6 @@ type chatCompletionRequest struct { Seed *int `json:"seed,omitempty"` Stop any `json:"stop,omitempty"` ResponseFormat any `json:"response_format,omitempty"` - Format any `json:"format,omitempty"` - KeepAlive any `json:"keep_alive,omitempty"` - Think any `json:"think,omitempty"` Tools []any `json:"tools,omitempty"` } @@ -61,7 +57,7 @@ func (req chatCompletionRequest) runInput(prompt string, messages []chatMessage, "prompt": prompt, "messages": chatMessagesInput(messages), } - options := cloneOptions(req.Options) + options := map[string]any{} if req.MaxTokens != nil { options["max_tokens"] = *req.MaxTokens } else { @@ -81,15 +77,7 @@ func (req chatCompletionRequest) runInput(prompt string, messages []chatMessage, if len(options) > 0 { input["options"] = options } - if req.Format != nil { - input["format"] = req.Format - } - if req.KeepAlive != nil { - input["keep_alive"] = req.KeepAlive - } - if req.Think != nil { - input["think"] = req.Think - } else if strictOutput { + if strictOutput { input["think"] = false } if len(req.Tools) > 0 { @@ -98,17 +86,6 @@ func (req chatCompletionRequest) runInput(prompt string, messages []chatMessage, return input } -func cloneOptions(options map[string]any) map[string]any { - if len(options) == 0 { - return map[string]any{} - } - out := make(map[string]any, len(options)) - for k, v := range options { - out[k] = v - } - return out -} - func setOptionInt(options map[string]any, key string, val *int) { if val != nil { options[key] = *val @@ -257,9 +234,9 @@ type responsesRequest struct { Input json.RawMessage `json:"input"` Instructions string `json:"instructions,omitempty"` Stream bool `json:"stream"` + Background bool `json:"background,omitempty"` Metadata json.RawMessage `json:"metadata,omitempty"` MaxOutputTokens *int `json:"max_output_tokens,omitempty"` - MaxTokens *int `json:"max_tokens,omitempty"` Temperature *float64 `json:"temperature,omitempty"` TopP *float64 `json:"top_p,omitempty"` } @@ -268,59 +245,12 @@ func (req responsesRequest) providerOptions() map[string]any { options := map[string]any{} if req.MaxOutputTokens != nil { options["max_tokens"] = *req.MaxOutputTokens - } else if req.MaxTokens != nil { - options["max_tokens"] = *req.MaxTokens } setOptionFloat(options, "temperature", req.Temperature) setOptionFloat(options, "top_p", req.TopP) return options } -type responsesMetadata struct { - RequestID string `json:"request_id,omitempty"` - Workspace string `json:"workspace,omitempty"` - TaskID string `json:"task_id,omitempty"` - Source string `json:"source,omitempty"` - Inference *responsesInferenceMetadata `json:"inference,omitempty"` - NomadCode *responsesNomadCodeMetadata `json:"nomadcode,omitempty"` -} - -type responsesInferenceMetadata struct { - Target string `json:"target,omitempty"` -} - -type responsesNomadCodeMetadata struct { - TaskID string `json:"task_id,omitempty"` - Source string `json:"source,omitempty"` -} - -func (m *responsesMetadata) metadataForRun() map[string]string { - out := make(map[string]string) - if m.RequestID != "" { - out["request_id"] = m.RequestID - } - if m.Inference != nil && m.Inference.Target != "" { - out["inference.target"] = m.Inference.Target - } - taskID := m.TaskID - source := m.Source - if m.NomadCode != nil { - if m.NomadCode.TaskID != "" { - taskID = m.NomadCode.TaskID - } - if m.NomadCode.Source != "" { - source = m.NomadCode.Source - } - } - if taskID != "" { - out["nomadcode.task_id"] = taskID - } - if source != "" { - out["nomadcode.source"] = source - } - return out -} - type responsesResponse struct { ID string `json:"id"` Object string `json:"object"` diff --git a/apps/edge/internal/service/service_test.go b/apps/edge/internal/service/service_test.go index af5faa6..bb68155 100644 --- a/apps/edge/internal/service/service_test.go +++ b/apps/edge/internal/service/service_test.go @@ -151,13 +151,13 @@ func TestBuildRunRequestCopiesMetadata(t *testing.T) { Adapter: "cli", Target: "codex", Prompt: "hello", - Metadata: map[string]string{"source": "edge-ops-console", "x-custom": "value"}, + Metadata: map[string]string{"request_id": "req-001", "x-custom": "value"}, }) if err != nil { t.Fatalf("BuildRunRequest: %v", err) } - if got := req.GetMetadata()["source"]; got != "edge-ops-console" { - t.Errorf("source: got %q, want %q", got, "edge-ops-console") + if got := req.GetMetadata()["request_id"]; got != "req-001" { + t.Errorf("request_id: got %q, want %q", got, "req-001") } if got := req.GetMetadata()["x-custom"]; got != "value" { t.Errorf("x-custom: got %q, want %q", got, "value") @@ -170,16 +170,14 @@ func TestBuildRunRequestPreservesResponsesMetadataKeys(t *testing.T) { Target: "llama", Prompt: "test", Metadata: map[string]string{ - "request_id": "req-001", - "inference.target": "metadata-target", - "nomadcode.task_id": "task-123", - "nomadcode.source": "nomadcode", + "request_id": "req-001", + "task_id": "task-123", }, }) if err != nil { t.Fatalf("BuildRunRequest: %v", err) } - for _, key := range []string{"request_id", "inference.target", "nomadcode.task_id", "nomadcode.source"} { + for _, key := range []string{"request_id", "task_id"} { if req.GetMetadata()[key] == "" { t.Errorf("metadata key %q not preserved", key) } diff --git a/scripts/e2e-openai-cli-workspace.sh b/scripts/e2e-openai-cli-workspace.sh index 785972b..c74d51e 100755 --- a/scripts/e2e-openai-cli-workspace.sh +++ b/scripts/e2e-openai-cli-workspace.sh @@ -133,7 +133,7 @@ done RESPONSES_OUT="$TMP_DIR/responses.json" curl -fsS \ -H "Content-Type: application/json" \ - -d '{"model":"'"$MODEL"'","input":"echo '\''marker_content'\'' > marker.txt && echo '\''IOP_CLI_SMOKE_OK'\''","stream":false,"metadata":{"request_id":"e2e-openai-cli-workspace","task_id":"task-smoke","source":"manual","workspace":"'"$WORKSPACE"'","inference":{"target":"smoke-sh"}}}' \ + -d '{"model":"'"$MODEL"'","input":"echo '\''marker_content'\'' > marker.txt && echo '\''IOP_CLI_SMOKE_OK'\''","stream":false,"metadata":{"request_id":"e2e-openai-cli-workspace","task_id":"task-smoke","workspace":"'"$WORKSPACE"'"}}' \ "http://127.0.0.1:$OPENAI_PORT/v1/responses" > "$RESPONSES_OUT" grep -q "IOP_CLI_SMOKE_OK" "$RESPONSES_OUT" diff --git a/scripts/e2e-openai-lemonade.sh b/scripts/e2e-openai-lemonade.sh index 595a38f..a13f0aa 100755 --- a/scripts/e2e-openai-lemonade.sh +++ b/scripts/e2e-openai-lemonade.sh @@ -254,7 +254,7 @@ grep -q "$MODEL" "$MODELS_OUT" CHAT_OUT="$TMP_DIR/chat.json" curl -fsS \ -H "Content-Type: application/json" \ - -d '{"model":"client-request-model","think":false,"options":{"max_tokens":32},"messages":[{"role":"user","content":"Reply with exactly: smoke token"}]}' \ + -d '{"model":"client-request-model","max_tokens":32,"messages":[{"role":"user","content":"Reply with exactly: smoke token"}]}' \ "http://127.0.0.1:$OPENAI_PORT/v1/chat/completions" > "$CHAT_OUT" if [ "$MODE" = "fake" ]; then grep -q "IOP_OPENAI_LEMONADE_OK" "$CHAT_OUT" @@ -269,7 +269,7 @@ fi STREAM_OUT="$TMP_DIR/stream.txt" curl -fsS -N \ -H "Content-Type: application/json" \ - -d '{"model":"client-request-model","stream":true,"think":false,"options":{"max_tokens":32},"messages":[{"role":"user","content":"Reply with exactly: stream token"}]}' \ + -d '{"model":"client-request-model","stream":true,"max_tokens":32,"messages":[{"role":"user","content":"Reply with exactly: stream token"}]}' \ "http://127.0.0.1:$OPENAI_PORT/v1/chat/completions" > "$STREAM_OUT" if [ "$MODE" = "fake" ]; then grep -q '"reasoning_content":"thinking..."' "$STREAM_OUT" diff --git a/scripts/e2e-openai-ollama.sh b/scripts/e2e-openai-ollama.sh index 4435359..c26e9bb 100755 --- a/scripts/e2e-openai-ollama.sh +++ b/scripts/e2e-openai-ollama.sh @@ -203,7 +203,7 @@ grep -q "reasoning_content" "$CHAT_OUT" RESPONSES_OUT="$TMP_DIR/responses.json" curl -fsS \ -H "Content-Type: application/json" \ - -d '{"model":"client-request-model","input":"say the responses test token","stream":false,"metadata":{"request_id":"e2e-openai-ollama","task_id":"task-smoke","source":"manual","inference":{"target":"fake-ollama-model"}}}' \ + -d '{"model":"client-request-model","input":"say the responses test token","stream":false,"metadata":{"request_id":"e2e-openai-ollama","task_id":"task-smoke"}}' \ "http://127.0.0.1:$OPENAI_PORT/v1/responses" > "$RESPONSES_OUT" grep -q "IOP_OPENAI_OLLAMA_OK" "$RESPONSES_OUT" grep -q '"output_text"' "$RESPONSES_OUT" diff --git a/scripts/e2e-openai-vllm.sh b/scripts/e2e-openai-vllm.sh index 54ad8b8..d84c3ec 100755 --- a/scripts/e2e-openai-vllm.sh +++ b/scripts/e2e-openai-vllm.sh @@ -251,7 +251,7 @@ grep -q "$ROUTE_ALIAS" "$MODELS_OUT" CHAT_OUT="$TMP_DIR/chat.json" curl -fsS \ -H "Content-Type: application/json" \ - -d "{\"model\":\"$ROUTE_ALIAS\",\"think\":false,\"options\":{\"max_tokens\":32},\"messages\":[{\"role\":\"user\",\"content\":\"Reply with exactly: smoke token\"}]}" \ + -d "{\"model\":\"$ROUTE_ALIAS\",\"max_tokens\":32,\"messages\":[{\"role\":\"user\",\"content\":\"Reply with exactly: smoke token\"}]}" \ "http://127.0.0.1:$OPENAI_PORT/v1/chat/completions" > "$CHAT_OUT" if [ "$MODE" = "fake" ]; then grep -q "IOP_OPENAI_VLLM_OK" "$CHAT_OUT" @@ -264,7 +264,7 @@ grep -q '"finish_reason":"' "$CHAT_OUT" STREAM_OUT="$TMP_DIR/stream.txt" curl -fsS -N \ -H "Content-Type: application/json" \ - -d "{\"model\":\"$ROUTE_ALIAS\",\"stream\":true,\"think\":false,\"options\":{\"max_tokens\":32},\"messages\":[{\"role\":\"user\",\"content\":\"Reply with exactly: stream token\"}]}" \ + -d "{\"model\":\"$ROUTE_ALIAS\",\"stream\":true,\"max_tokens\":32,\"messages\":[{\"role\":\"user\",\"content\":\"Reply with exactly: stream token\"}]}" \ "http://127.0.0.1:$OPENAI_PORT/v1/chat/completions" > "$STREAM_OUT" if [ "$MODE" = "fake" ]; then grep -q '"content":"IOP_OPENAI_"' "$STREAM_OUT"