feat: OpenAI compatible API 변경 사항 적용

- agent-contract: OpenAI compatible API 계약 문서 업데이트
- apps/edge: OpenAI API smoke/test 스크립트 및 핸들러 개선
- scripts: e2e 테스트 스크립트 업데이트
This commit is contained in:
toki 2026-06-27 05:16:25 +09:00
parent 6dfbd03fd3
commit 46bd8563e2
13 changed files with 347 additions and 271 deletions

View file

@ -12,7 +12,7 @@
| id | 읽는 조건 | path |
|----|-----------|------|
| `iop.openai-compatible-api` | OpenAI-compatible API, Responses API, Chat Completions, `model` route, Codex/CLI workspace, NomadCode authoring metadata, `metadata.workspace`, `metadata.task_id`, `metadata.source` | `agent-contract/provided/openai-compatible-api.md` |
| `iop.openai-compatible-api` | OpenAI-compatible API, Responses API, Chat Completions, legacy Completions, `model` route, Codex/CLI workspace, generic authoring metadata, `metadata.workspace`, `metadata.task_id` | `agent-contract/provided/openai-compatible-api.md` |
## 소비 계약

View file

@ -44,52 +44,83 @@ CLI agent 실행으로 라우팅되는 요청의 최소 형태:
}
```
현재 `/v1/responses`에서 허용하는 표준형 요청 예시:
```json
{
"model": "codex",
"instructions": "응답은 짧게 작성해.",
"input": "현재 워크스페이스의 테스트 상태를 확인해줘.",
"stream": false,
"background": false,
"max_output_tokens": 4096,
"temperature": 0,
"top_p": 1,
"metadata": {
"workspace": "/config/workspace/iop",
"request_id": "req-001",
"task_id": "task-123"
}
}
```
필드 의미:
- `model`: Edge가 내부 `adapter + target`으로 해석할 외부 route 이름이다.
- `input`: agent에게 전달할 사용자 요청이다.
- `model`: Edge가 내부 `adapter + target`으로 해석할 외부 route 이름이다. IOP Edge에서는 라우팅을 위해 필수다.
- `instructions`: OpenAI Responses API의 top-level instruction field다. 있으면 `input` 앞에 배치해 agent 실행 prompt를 만든다.
- `input`: agent에게 전달할 사용자 요청이다. 현재 구현은 string input만 지원한다.
- `stream`: 현재 구현은 `false` 또는 생략만 지원한다.
- `background`: 현재 구현은 `false` 또는 생략만 지원한다.
- `metadata.workspace`: CLI process를 실행할 작업 디렉터리다. CLI agent route에서는 필수 실행 문맥이다.
- `metadata`: OpenAI 표준 metadata container다. string key/value를 허용하고, IOP는 `workspace`만 실행 문맥으로 해석한다. 나머지 key는 caller-defined metadata로 보존하되 `source`는 지원하지 않는다.
- `metadata.request_id`, `metadata.task_id`: caller-defined metadata 예시다. 특별한 wrapper나 제품 전용 field가 아니다.
- `max_output_tokens`: 출력 길이 상한이다. 내부 provider option의 `max_tokens`로 전달된다.
- `temperature`: 생성 다양성 option이다. 대상 adapter가 지원하지 않으면 무시될 수 있다.
- `top_p`: nucleus sampling option이다. 대상 adapter가 지원하지 않으면 무시될 수 있다.
금지:
- `metadata.cli` 같은 CLI 전용 wrapper를 추가하지 않는다.
- `metadata.inference`처럼 `model` route와 겹치는 target wrapper를 추가하지 않는다.
- `metadata.nomadcode`처럼 특정 소비자 제품명에 묶인 wrapper를 추가하지 않는다.
- `metadata.source`처럼 의미가 불명확한 호출 출처 field를 추가하지 않는다.
- root-level `iop` 같은 별도 wrapper field를 추가하지 않는다.
- `/v1/responses``options` wrapper를 추가하지 않는다. Responses API option은 OpenAI 표준 top-level field를 따른다.
- `session_id`, `timeout_sec` 같은 IOP 실행 제어 field를 request body 계약에 추가하지 않는다. logical session과 timeout은 route/config 기본값을 따른다.
- workspace를 prompt 본문에 섞어 전달하지 않는다.
현재 구현 메모:
- `/v1/responses`는 non-streaming 요청만 지원한다.
- 기존 metadata 계약인 `metadata.request_id`, `metadata.nomadcode.task_id`, `metadata.nomadcode.source`, `metadata.inference.target`은 유지한다.
- `metadata`는 최대 16개 string key/value를 허용한다. key는 64자 이하, value는 512자 이하를 기준으로 한다.
- CLI route의 `metadata.workspace`는 이 문서의 계약 기준이다. 구현은 이 값을 Edge service의 run workspace와 Node CLI adapter의 process working directory로 전달해야 한다.
- `metadata.workspace``RunRequest.Workspace`로 전달하고 generic run metadata에는 복사하지 않는다.
- 다른 Responses API 표준 field는 구현 필요가 생길 때 계약을 갱신한 뒤 추가한다.
## NomadCode Authoring Handoff
## Generic Authoring Handoff
NomadCode Core가 IOP Edge HTTP 표면으로 workspace authoring 작업을 넘길 때의 최소 요청 형태:
외부 caller가 IOP Edge HTTP 표면으로 workspace authoring 작업을 넘길 때의 최소 요청 형태:
```json
{
"model": "codex",
"input": "Todo 항목에 필요한 산출물을 현재 checkout에 작성해줘.",
"metadata": {
"workspace": "/config/workspace/nomadcode-slot-123",
"task_id": "todo-123",
"source": "nomadcode"
"workspace": "/config/workspace/work-slot-123",
"task_id": "todo-123"
}
}
```
NomadCode task/source context는 flat `metadata.task_id``metadata.source`로 전달할 수 있다.
동일한 의미의 structured 형태가 필요하면 `metadata.nomadcode.task_id``metadata.nomadcode.source`를 사용하며, flat alias와 structured 값이 함께 있으면 structured 값을 우선한다.
이 handoff는 `model`, `input`, `metadata.workspace`, task/source metadata만으로 충분해야 한다.
호출자는 `metadata.cli`, root-level `iop` wrapper, IOP CLI 직접 실행, prompt 본문 workspace 주입을 요구받지 않는다.
이 handoff는 `model`, `input`, `metadata.workspace`, 필요한 caller-defined metadata만으로 충분해야 한다.
호출자는 `metadata.cli`, 소비자 전용 metadata wrapper, root-level `iop` wrapper, IOP CLI 직접 실행, prompt 본문 workspace 주입을 요구받지 않는다.
Workspace-bound route는 workspace가 없거나 상대 경로이면 OpenAI-compatible error로 거부한다.
존재하지 않는 경로, 권한 오류, agent process exit failure는 기본 cwd fallback으로 숨기지 않고 호출자가 실패로 구분할 수 있어야 한다.
## Chat Completions
`/v1/chat/completions`도 같은 metadata 원칙을 따른다. CLI route의 workspace는 `metadata.workspace`에 둔다.
`/v1/chat/completions`도 같은 metadata 원칙을 따른다. CLI route의 workspace는 `metadata.workspace`에 둔다. Chat Completions의 provider sampling option은 해당 endpoint의 OpenAI-compatible top-level request field를 따르며, `/v1/responses`와 마찬가지로 별도 `options` wrapper를 두지 않는다.
```json
{
@ -106,6 +137,34 @@ Workspace-bound route는 workspace가 없거나 상대 경로이면 OpenAI-compa
}
```
현재 지원하는 Chat Completions request field:
- `model`
- `messages`
- `stream`
- `metadata`
- `max_tokens`
- `max_completion_tokens`
- `temperature`
- `top_p`
- `presence_penalty`
- `frequency_penalty`
- `seed`
- `stop`
- `response_format`
- `tools`
금지:
- `metadata.source`, `metadata.cli`, `metadata.inference`, `metadata.nomadcode`
- `options`, `think`, `format`, `keep_alive` 같은 provider/Ollama 전용 request field
- `session_id`, `timeout_sec` 같은 IOP 실행 제어 field
## Legacy Completions
`POST /v1/completions`는 현재 IOP Edge OpenAI-compatible 표면에서 제공하지 않는다.
text completion 형태의 신규 호출은 `/v1/responses`를 사용하고, message 기반 호출은 `/v1/chat/completions`를 사용한다.
## Routing
Edge 설정이 `openai.model_routes[]`를 제공하면 `model`은 먼저 route catalog에서 해석된다.

View file

@ -193,7 +193,7 @@ Edge 외부 입력은 OpenAI-compatible HTTP API와 A2A JSON-RPC HTTP API 두
이 표면은 외부 모델 클라이언트 호환을 위한 표준 경로다. Edge/Node 운영 제어, CLI logical session, background run, cancel/terminate-session, capabilities/status/session/transport command, node lifecycle event 같은 IOP 고유 기능은 OpenAI-compatible 요청에 억지로 싣지 않고 IOP native protocol(protobuf-socket) 계열에서 다룬다.
외부 프로젝트가 참조할 요청 계약 원문은 repo root의 `agent-contract/provided/openai-compatible-api.md`에 둔다. CLI agent route에서 사용할 workspace는 OpenAI request의 `metadata.workspace`에 둔다. NomadCode authoring handoff의 요청 shape도 같은 계약 원문을 기준으로 한다.
외부 프로젝트가 참조할 요청 계약 원문은 repo root의 `agent-contract/provided/openai-compatible-api.md`에 둔다. CLI agent route에서 사용할 workspace는 OpenAI request의 `metadata.workspace`에 둔다. Workspace authoring handoff의 요청 shape도 같은 계약 원문을 기준으로 한다.
```yaml
openai:
@ -233,7 +233,7 @@ openai:
target: "llama3:8b"
```
`/v1/chat/completions`는 기본 non-streaming과 streaming SSE 응답을 모두 지원한다. `/v1/responses`는 현재 non-streaming 요청만 지원한다. 두 endpoint 모두 `metadata.workspace`를 run workspace로 전달하고, `metadata.request_id`, `metadata.inference.target`, `metadata.nomadcode.task_id`, `metadata.nomadcode.source`를 service boundary까지 전달한다. NomadCode compatibility alias로 root-level `metadata.task_id``metadata.source`도 받아 각각 `nomadcode.task_id`, `nomadcode.source`로 매핑하며, structured `metadata.nomadcode.*` 값이 있으면 그 값을 우선한다. `metadata.cli`는 지원하지 않는다.
`/v1/chat/completions`는 기본 non-streaming과 streaming SSE 응답을 모두 지원한다. `/v1/responses`는 현재 non-streaming 요청만 지원한다. 두 endpoint 모두 `metadata.workspace`를 run workspace로 전달한다. `metadata`는 OpenAI 표준의 caller-defined string metadata container로 보고, IOP가 특별히 해석하는 key는 `workspace`뿐이다. `/v1/responses``max_output_tokens`, `temperature`, `top_p`, `instructions`, `background``/v1/chat/completions``max_tokens`, `max_completion_tokens`, `temperature`, `top_p` 등은 OpenAI API처럼 top-level field에 둔다. `metadata.source`, `metadata.cli`, `metadata.inference`, 소비자 전용 metadata wrapper, `options`, `think`, `format`, `keep_alive` 같은 provider/Ollama 전용 request field는 지원하지 않는다.
```bash
curl -s http://127.0.0.1:18081/v1/chat/completions \
@ -244,10 +244,10 @@ curl -s http://127.0.0.1:18081/v1/chat/completions \
```bash
curl -s http://127.0.0.1:18081/v1/responses \
-H 'Content-Type: application/json' \
-d '{"model":"qwen3.6:35b-a3b-bf16","input":"hello","stream":false,"metadata":{"request_id":"example","workspace":"/config/workspace/iop","nomadcode":{"task_id":"task-123","source":"manual"}}}'
-d '{"model":"qwen3.6:35b-a3b-bf16","instructions":"reply briefly","input":"hello","stream":false,"background":false,"max_output_tokens":256,"temperature":0,"top_p":1,"metadata":{"request_id":"example","workspace":"/config/workspace/iop","task_id":"task-123"}}'
```
vLLM 같은 OpenAI-compatible inference server는 `openai_compat` adapter와 `provider: "vllm"` instance로 연결한다. 이 경로는 `/v1/models`, non-streaming/streaming `/v1/chat/completions`, provider header, request `options` 전달을 지원하며, Edge의 model route alias를 실제 served model로 매핑한다.
vLLM 같은 OpenAI-compatible inference server는 `openai_compat` adapter와 `provider: "vllm"` instance로 연결한다. 이 경로는 `/v1/models`, non-streaming/streaming `/v1/chat/completions`, provider header, 표준 top-level request field 전달을 지원하며, Edge의 model route alias를 실제 served model로 매핑한다.
#### Model Provider Pool 호환 방향

View file

@ -243,10 +243,6 @@ If --base-url is not provided, it auto-discovers the OpenAI URL from the effecti
metadata := map[string]interface{}{
"request_id": "iop-edge-smoke",
"task_id": "smoke",
"source": "iop-edge-smoke",
"inference": map[string]string{
"target": smokeModel,
},
}
if smokeWorkspace != "" {
metadata["workspace"] = smokeWorkspace

View file

@ -24,17 +24,17 @@ func (s *Server) handleChatCompletions(w http.ResponseWriter, r *http.Request) {
defer r.Body.Close()
var req chatCompletionRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_request_error", "invalid JSON request")
if err := decodeChatCompletionRequest(json.NewDecoder(r.Body), &req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_request_error", err.Error())
return
}
runMeta, inferenceTarget, workspace, err := parseOpenAIMetadata(req.Metadata)
runMeta, workspace, err := parseOpenAIMetadata(req.Metadata)
if err != nil {
writeError(w, http.StatusBadRequest, "invalid_request_error", err.Error())
return
}
dispatch, ok := s.resolveRouteDispatch(req.Model, inferenceTarget)
dispatch, ok := s.resolveRouteDispatch(req.Model)
if !ok {
writeError(w, http.StatusBadRequest, "invalid_request_error", "model is required")
return
@ -98,11 +98,44 @@ func (s *Server) handleChatCompletions(w http.ResponseWriter, r *http.Request) {
s.completeChatCompletion(w, r, req, handle, outputPolicy)
}
func decodeChatCompletionRequest(dec *json.Decoder, req *chatCompletionRequest) error {
var raw map[string]json.RawMessage
if err := dec.Decode(&raw); err != nil {
return fmt.Errorf("invalid JSON request")
}
for key := range raw {
switch key {
case "model", "messages", "stream", "metadata", "max_tokens", "max_completion_tokens", "temperature", "top_p", "presence_penalty", "frequency_penalty", "seed", "stop", "response_format", "tools":
default:
return fmt.Errorf("%s is not supported for /v1/chat/completions", key)
}
}
normalized, err := json.Marshal(raw)
if err != nil {
return fmt.Errorf("invalid JSON request")
}
if err := json.Unmarshal(normalized, req); err != nil {
return fmt.Errorf("invalid /v1/chat/completions request format")
}
if req.MaxTokens != nil && *req.MaxTokens <= 0 {
return fmt.Errorf("max_tokens must be greater than zero")
}
if req.MaxCompletionTokens != nil && *req.MaxCompletionTokens <= 0 {
return fmt.Errorf("max_completion_tokens must be greater than zero")
}
if req.Temperature != nil && (*req.Temperature < 0 || *req.Temperature > 2) {
return fmt.Errorf("temperature must be between 0 and 2")
}
if req.TopP != nil && (*req.TopP < 0 || *req.TopP > 1) {
return fmt.Errorf("top_p must be between 0 and 1")
}
return nil
}
func chatRunMetadata(runMeta map[string]string, req chatCompletionRequest, outputPolicy strictOutputPolicy) map[string]string {
if runMeta == nil {
runMeta = make(map[string]string)
}
runMeta["source"] = "openai"
runMeta["openai_model"] = req.Model
runMeta["openai_stream"] = fmt.Sprintf("%t", req.Stream)
runMeta["strict_output"] = fmt.Sprintf("%t", outputPolicy.Strict)
@ -149,16 +182,9 @@ func (s *Server) resolveAdapter() string {
}
func (s *Server) resolveTarget(model string) string {
return s.resolveTargetWithOverride(model, "")
}
func (s *Server) resolveTargetWithOverride(model, override string) string {
if s.cfg.Target != "" {
return s.cfg.Target
}
if override != "" {
return strings.TrimSpace(override)
}
return strings.TrimSpace(model)
}
@ -212,9 +238,8 @@ func (s *Server) findProviderPoolEntry(model string) *config.ModelCatalogEntry {
// resolveRouteDispatch returns fully-resolved dispatch params for model.
// Priority: provider-pool catalog → legacy model_routes → single-target fallback.
// metadataTarget is only used in the legacy fallback path.
// Returns (dispatch, true) on success; (zero, false) when no target can be resolved.
func (s *Server) resolveRouteDispatch(model, metadataTarget string) (routeDispatch, bool) {
func (s *Server) resolveRouteDispatch(model string) (routeDispatch, bool) {
// Provider-pool catalog takes highest priority.
if s.findProviderPoolEntry(model) != nil {
return routeDispatch{
@ -252,7 +277,7 @@ func (s *Server) resolveRouteDispatch(model, metadataTarget string) (routeDispat
WorkspaceRequired: route.WorkspaceRequired,
}, true
}
target := s.resolveTargetWithOverride(model, metadataTarget)
target := s.resolveTarget(model)
if target == "" {
return routeDispatch{}, false
}

View file

@ -20,8 +20,8 @@ func (s *Server) handleResponses(w http.ResponseWriter, r *http.Request) {
defer r.Body.Close()
var req responsesRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_request_error", "invalid JSON request")
if err := decodeResponsesRequest(json.NewDecoder(r.Body), &req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_request_error", err.Error())
return
}
@ -29,6 +29,10 @@ func (s *Server) handleResponses(w http.ResponseWriter, r *http.Request) {
writeError(w, http.StatusBadRequest, "invalid_request_error", "streaming is not supported for /v1/responses")
return
}
if req.Background {
writeError(w, http.StatusBadRequest, "invalid_request_error", "background is not supported for /v1/responses")
return
}
inputStr, err := parseResponsesInput(req.Input)
if err != nil {
@ -42,13 +46,13 @@ func (s *Server) handleResponses(w http.ResponseWriter, r *http.Request) {
prompt = instruction + "\n" + prompt
}
runMeta, inferenceTarget, workspace, err := parseOpenAIMetadata(req.Metadata)
runMeta, workspace, err := parseOpenAIMetadata(req.Metadata)
if err != nil {
writeError(w, http.StatusBadRequest, "invalid_request_error", err.Error())
return
}
dispatch, ok := s.resolveRouteDispatch(req.Model, inferenceTarget)
dispatch, ok := s.resolveRouteDispatch(req.Model)
if !ok {
writeError(w, http.StatusBadRequest, "invalid_request_error", "model is required")
return
@ -58,7 +62,6 @@ func (s *Server) handleResponses(w http.ResponseWriter, r *http.Request) {
return
}
runMeta["source"] = "openai-responses"
runMeta["openai_model"] = req.Model
runMeta["openai_stream"] = fmt.Sprintf("%t", req.Stream)
runMeta["strict_output"] = fmt.Sprintf("%t", outputPolicy.Strict)
@ -105,6 +108,37 @@ func (s *Server) handleResponses(w http.ResponseWriter, r *http.Request) {
s.completeResponse(w, r, req, handle, outputPolicy)
}
func decodeResponsesRequest(dec *json.Decoder, req *responsesRequest) error {
var raw map[string]json.RawMessage
if err := dec.Decode(&raw); err != nil {
return fmt.Errorf("invalid JSON request")
}
for key := range raw {
switch key {
case "model", "input", "instructions", "stream", "background", "metadata", "max_output_tokens", "temperature", "top_p":
default:
return fmt.Errorf("%s is not supported for /v1/responses", key)
}
}
normalized, err := json.Marshal(raw)
if err != nil {
return fmt.Errorf("invalid JSON request")
}
if err := json.Unmarshal(normalized, req); err != nil {
return fmt.Errorf("invalid /v1/responses request format")
}
if req.MaxOutputTokens != nil && *req.MaxOutputTokens <= 0 {
return fmt.Errorf("max_output_tokens must be greater than zero")
}
if req.Temperature != nil && (*req.Temperature < 0 || *req.Temperature > 2) {
return fmt.Errorf("temperature must be between 0 and 2")
}
if req.TopP != nil && (*req.TopP < 0 || *req.TopP > 1) {
return fmt.Errorf("top_p must be between 0 and 1")
}
return nil
}
func (s *Server) completeResponse(w http.ResponseWriter, r *http.Request, req responsesRequest, handle edgeservice.RunResult, outputPolicy strictOutputPolicy) {
text, reasoning, _, usage, err := collectRunResult(r.Context(), handle.Stream(), handle.WaitTimeout())
if err != nil {
@ -161,37 +195,60 @@ func parseResponsesInput(raw json.RawMessage) (string, error) {
func buildResponsesPrompt(instructions, input string) string {
instructions = strings.TrimSpace(instructions)
input = strings.TrimSpace(input)
if instructions == "" {
return input
}
return instructions + "\n" + input
return instructions + "\n\n" + input
}
func parseOpenAIMetadata(raw json.RawMessage) (map[string]string, string, string, error) {
func parseOpenAIMetadata(raw json.RawMessage) (map[string]string, string, error) {
if len(raw) == 0 || string(raw) == "null" {
return make(map[string]string), "", "", nil
return make(map[string]string), "", nil
}
var rawMap map[string]json.RawMessage
if err := json.Unmarshal(raw, &rawMap); err != nil {
return nil, "", "", fmt.Errorf("metadata must be an object")
return nil, "", fmt.Errorf("metadata must be an object")
}
if _, hasCLI := rawMap["cli"]; hasCLI {
return nil, "", "", fmt.Errorf("metadata.cli is not supported")
if len(rawMap) > 16 {
return nil, "", fmt.Errorf("metadata must contain at most 16 keys")
}
var meta responsesMetadata
if err := json.Unmarshal(raw, &meta); err != nil {
return nil, "", "", fmt.Errorf("invalid metadata format")
flat := make(map[string]string, len(rawMap))
var workspace string
for key, rawValue := range rawMap {
if len(key) > 64 {
return nil, "", fmt.Errorf("metadata key %q exceeds 64 characters", key)
}
if key == "source" {
return nil, "", fmt.Errorf("metadata.source is not supported")
}
if key == "workspace" {
workspaceValue, err := metadataStringValue(key, rawValue)
if err != nil {
return nil, "", err
}
workspace = strings.TrimSpace(workspaceValue)
continue
}
value, err := metadataStringValue(key, rawValue)
if err != nil {
return nil, "", err
}
flat[key] = value
}
flat := meta.metadataForRun()
inferenceTarget := ""
if meta.Inference != nil {
inferenceTarget = meta.Inference.Target
}
return flat, inferenceTarget, strings.TrimSpace(meta.Workspace), nil
return flat, workspace, nil
}
func metadataStringValue(key string, raw json.RawMessage) (string, error) {
var value string
if err := json.Unmarshal(raw, &value); err != nil {
return "", fmt.Errorf("metadata.%s must be a string", key)
}
if len(value) > 512 {
return "", fmt.Errorf("metadata.%s exceeds 512 characters", key)
}
return value, nil
}

View file

@ -184,7 +184,7 @@ func TestChatCompletionsPreservesProviderFinishReason(t *testing.T) {
}
}
func TestChatCompletionsPassesOllamaOptions(t *testing.T) {
func TestChatCompletionsPassesStandardOptions(t *testing.T) {
fake := &fakeRunService{events: make(chan *iop.RunEvent, 2)}
fake.events <- &iop.RunEvent{Type: "delta", Delta: "ok"}
fake.events <- &iop.RunEvent{Type: "complete"}
@ -193,12 +193,11 @@ func TestChatCompletionsPassesOllamaOptions(t *testing.T) {
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{
"model":"from-request",
"messages":[{"role":"user","content":"hi"}],
"options":{"temperature":0.2,"top_p":0.9,"num_predict":32,"stop":["END"]},
"max_tokens":12,
"temperature":0.1,
"keep_alive":"10m",
"think":false,
"format":"json",
"top_p":0.9,
"stop":["END"],
"response_format":{"type":"json_object"},
"tools":[{"type":"function","function":{"name":"lookup"}}]
}`))
w := httptest.NewRecorder()
@ -212,14 +211,17 @@ func TestChatCompletionsPassesOllamaOptions(t *testing.T) {
if !ok {
t.Fatalf("options not passed: %+v", fake.req.Input)
}
if options["temperature"].(float64) != 0.1 || options["top_p"].(float64) != 0.9 || options["num_predict"].(float64) != 32 {
if options["temperature"].(float64) != 0.1 || options["top_p"].(float64) != 0.9 {
t.Fatalf("unexpected options: %+v", options)
}
if options["max_tokens"].(int) != 12 {
t.Fatalf("max_tokens not passed: %+v", options)
}
if fake.req.Input["keep_alive"] != "10m" || fake.req.Input["think"] != false || fake.req.Input["format"] != "json" {
t.Fatalf("top-level ollama fields not passed: %+v", fake.req.Input)
if _, ok := options["stop"]; !ok {
t.Fatalf("stop not passed: %+v", options)
}
if _, ok := options["response_format"]; !ok {
t.Fatalf("response_format not passed: %+v", options)
}
if tools, ok := fake.req.Input["tools"].([]any); !ok || len(tools) != 1 {
t.Fatalf("tools not passed: %+v", fake.req.Input["tools"])
@ -545,7 +547,7 @@ func TestOpenAIProviderPoolRouteUsesRefreshedCatalog(t *testing.T) {
ID: "model-new",
Providers: map[string]string{"prov-a": "served-a"},
}})
dispatch, ok := srv.resolveRouteDispatch("model-new", "")
dispatch, ok := srv.resolveRouteDispatch("model-new")
if !ok || !dispatch.ProviderPool {
t.Fatalf("expected provider-pool dispatch for refreshed model, got ok=%v dispatch=%+v", ok, dispatch)
}
@ -608,14 +610,19 @@ func TestResponsesDispatchesNonStreamingRequest(t *testing.T) {
fake.events <- &iop.RunEvent{Type: "complete", Usage: &iop.Usage{InputTokens: 1, OutputTokens: 1}}
srv := NewServer(config.EdgeOpenAIConf{
Adapter: "ollama",
Target: "llama-fixed",
Adapter: "ollama",
Target: "llama-fixed",
SessionID: "configured-session",
TimeoutSec: 42,
}, fake, nil)
req := httptest.NewRequest(http.MethodPost, "/v1/responses", strings.NewReader(`{
"model":"client-model",
"input":"say hello",
"instructions":"Use short answers.",
"max_output_tokens":16,
"temperature":0.2
"temperature":0.2,
"top_p":0.9,
"background":false
}`))
w := httptest.NewRecorder()
@ -627,16 +634,25 @@ func TestResponsesDispatchesNonStreamingRequest(t *testing.T) {
if fake.req.Target != "llama-fixed" {
t.Fatalf("target: got %q, want llama-fixed", fake.req.Target)
}
if fake.req.Prompt != "say hello" {
if fake.req.Prompt != "Use short answers.\n\nsay hello" {
t.Fatalf("prompt: got %q", fake.req.Prompt)
}
options, ok := fake.req.Input["options"].(map[string]any)
if !ok {
t.Fatalf("options not passed: %+v", fake.req.Input)
}
if options["max_tokens"].(int) != 16 || options["temperature"].(float64) != 0.2 {
if options["max_tokens"].(int) != 16 || options["temperature"].(float64) != 0.2 || options["top_p"].(float64) != 0.9 {
t.Fatalf("unexpected response options: %+v", options)
}
if fake.req.TimeoutSec != 42 {
t.Fatalf("timeout: got %d, want 42", fake.req.TimeoutSec)
}
if fake.req.SessionID != "configured-session" {
t.Fatalf("session_id: got %q, want configured-session", fake.req.SessionID)
}
if fake.req.Background {
t.Fatal("background should remain false for responses")
}
var resp responsesResponse
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
@ -663,6 +679,10 @@ func TestResponsesRejectsUnsupportedRequests(t *testing.T) {
{"stream=true", http.MethodPost, `{"model":"m","input":"hi","stream":true}`, http.StatusBadRequest},
{"empty input", http.MethodPost, `{"model":"m","input":""}`, http.StatusBadRequest},
{"non-string input", http.MethodPost, `{"model":"m","input":["a","b"]}`, http.StatusBadRequest},
{"options wrapper unsupported", http.MethodPost, `{"model":"m","input":"hi","options":{"max_output_tokens":16}}`, http.StatusBadRequest},
{"background unsupported", http.MethodPost, `{"model":"m","input":"hi","background":true}`, http.StatusBadRequest},
{"bad max_output_tokens", http.MethodPost, `{"model":"m","input":"hi","max_output_tokens":0}`, http.StatusBadRequest},
{"bad top_p", http.MethodPost, `{"model":"m","input":"hi","top_p":1.5}`, http.StatusBadRequest},
}
for _, tc := range cases {
@ -677,7 +697,7 @@ func TestResponsesRejectsUnsupportedRequests(t *testing.T) {
}
}
func TestResponsesMetadataContractAndTargetOverride(t *testing.T) {
func TestResponsesGenericMetadataContract(t *testing.T) {
fake := &fakeRunService{events: make(chan *iop.RunEvent, 2)}
fake.events <- &iop.RunEvent{Type: "delta", Delta: "ok"}
fake.events <- &iop.RunEvent{Type: "complete"}
@ -689,8 +709,8 @@ func TestResponsesMetadataContractAndTargetOverride(t *testing.T) {
"metadata":{
"request_id":"req-001",
"workspace":"/config/workspace/iop",
"inference":{"target":"metadata-target"},
"nomadcode":{"task_id":"task-123","source":"nomadcode"}
"task_id":"task-123",
"custom":"value"
}
}`))
w := httptest.NewRecorder()
@ -699,8 +719,8 @@ func TestResponsesMetadataContractAndTargetOverride(t *testing.T) {
if w.Code != http.StatusOK {
t.Fatalf("status: got %d body=%s", w.Code, w.Body.String())
}
if fake.req.Target != "metadata-target" {
t.Fatalf("target: got %q, want metadata-target", fake.req.Target)
if fake.req.Target != "client-model" {
t.Fatalf("target: got %q, want client-model", fake.req.Target)
}
if fake.req.Workspace != "/config/workspace/iop" {
t.Fatalf("workspace: got %q", fake.req.Workspace)
@ -711,17 +731,11 @@ func TestResponsesMetadataContractAndTargetOverride(t *testing.T) {
if _, ok := fake.req.Metadata["workspace"]; ok {
t.Fatal("workspace should not be copied into run metadata")
}
if fake.req.Metadata["inference.target"] != "metadata-target" {
t.Fatalf("inference.target: got %q", fake.req.Metadata["inference.target"])
if fake.req.Metadata["task_id"] != "task-123" {
t.Fatalf("task_id: got %q", fake.req.Metadata["task_id"])
}
if fake.req.Metadata["nomadcode.task_id"] != "task-123" {
t.Fatalf("nomadcode.task_id: got %q", fake.req.Metadata["nomadcode.task_id"])
}
if fake.req.Metadata["nomadcode.source"] != "nomadcode" {
t.Fatalf("nomadcode.source: got %q", fake.req.Metadata["nomadcode.source"])
}
if fake.req.Metadata["source"] != "openai-responses" {
t.Fatalf("source: got %q", fake.req.Metadata["source"])
if fake.req.Metadata["custom"] != "value" {
t.Fatalf("custom: got %q", fake.req.Metadata["custom"])
}
if fake.req.Metadata["openai_model"] != "client-model" {
t.Fatalf("openai_model: got %q", fake.req.Metadata["openai_model"])
@ -749,8 +763,7 @@ func TestChatCompletionsMetadataContractAndWorkspace(t *testing.T) {
"metadata":{
"request_id":"req-chat-001",
"workspace":"/config/workspace/iop",
"inference":{"target":"metadata-target"},
"nomadcode":{"task_id":"task-123","source":"nomadcode"}
"task_id":"task-123"
}
}`))
w := httptest.NewRecorder()
@ -759,8 +772,8 @@ func TestChatCompletionsMetadataContractAndWorkspace(t *testing.T) {
if w.Code != http.StatusOK {
t.Fatalf("status: got %d body=%s", w.Code, w.Body.String())
}
if fake.req.Target != "metadata-target" {
t.Fatalf("target: got %q, want metadata-target", fake.req.Target)
if fake.req.Target != "client-model" {
t.Fatalf("target: got %q, want client-model", fake.req.Target)
}
if fake.req.Workspace != "/config/workspace/iop" {
t.Fatalf("workspace: got %q", fake.req.Workspace)
@ -768,17 +781,8 @@ func TestChatCompletionsMetadataContractAndWorkspace(t *testing.T) {
if fake.req.Metadata["request_id"] != "req-chat-001" {
t.Fatalf("request_id: got %q", fake.req.Metadata["request_id"])
}
if fake.req.Metadata["inference.target"] != "metadata-target" {
t.Fatalf("inference.target: got %q", fake.req.Metadata["inference.target"])
}
if fake.req.Metadata["nomadcode.task_id"] != "task-123" {
t.Fatalf("nomadcode.task_id: got %q", fake.req.Metadata["nomadcode.task_id"])
}
if fake.req.Metadata["nomadcode.source"] != "nomadcode" {
t.Fatalf("nomadcode.source: got %q", fake.req.Metadata["nomadcode.source"])
}
if fake.req.Metadata["source"] != "openai" {
t.Fatalf("source: got %q", fake.req.Metadata["source"])
if fake.req.Metadata["task_id"] != "task-123" {
t.Fatalf("task_id: got %q", fake.req.Metadata["task_id"])
}
if fake.req.ModelGroupKey != "client-model" {
t.Fatalf("model group key: got %q, want client-model", fake.req.ModelGroupKey)
@ -788,7 +792,7 @@ func TestChatCompletionsMetadataContractAndWorkspace(t *testing.T) {
}
}
func TestChatCompletionsRejectsCLIMetadata(t *testing.T) {
func TestChatCompletionsRejectsObjectMetadata(t *testing.T) {
srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "llama"}, &fakeRunService{}, nil)
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{
"model":"m",
@ -802,6 +806,65 @@ func TestChatCompletionsRejectsCLIMetadata(t *testing.T) {
}
}
func TestChatCompletionsRejectsUnsupportedFields(t *testing.T) {
srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "llama"}, &fakeRunService{}, nil)
cases := []struct {
name string
body string
}{
{"options wrapper", `{"model":"m","messages":[{"role":"user","content":"hi"}],"options":{"num_ctx":8192}}`},
{"think", `{"model":"m","messages":[{"role":"user","content":"hi"}],"think":false}`},
{"format", `{"model":"m","messages":[{"role":"user","content":"hi"}],"format":"json"}`},
{"keep_alive", `{"model":"m","messages":[{"role":"user","content":"hi"}],"keep_alive":"10m"}`},
{"bad max_tokens", `{"model":"m","messages":[{"role":"user","content":"hi"}],"max_tokens":0}`},
{"bad top_p", `{"model":"m","messages":[{"role":"user","content":"hi"}],"top_p":2}`},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(tc.body))
w := httptest.NewRecorder()
srv.routes().ServeHTTP(w, req)
if w.Code != http.StatusBadRequest {
t.Fatalf("got %d want 400, body=%s", w.Code, w.Body.String())
}
})
}
}
func TestChatCompletionsRejectsSourceMetadata(t *testing.T) {
srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "llama"}, &fakeRunService{}, nil)
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{
"model":"m",
"messages":[{"role":"user","content":"hi"}],
"metadata":{"source":"manual"}
}`))
w := httptest.NewRecorder()
srv.routes().ServeHTTP(w, req)
if w.Code != http.StatusBadRequest {
t.Fatalf("got %d want 400, body=%s", w.Code, w.Body.String())
}
if !strings.Contains(w.Body.String(), "metadata.source is not supported") {
t.Fatalf("expected source unsupported error, got %s", w.Body.String())
}
}
func TestChatCompletionsRejectsNonStringMetadataValue(t *testing.T) {
srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "llama"}, &fakeRunService{}, nil)
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{
"model":"m",
"messages":[{"role":"user","content":"hi"}],
"metadata":{"attempt":2}
}`))
w := httptest.NewRecorder()
srv.routes().ServeHTTP(w, req)
if w.Code != http.StatusBadRequest {
t.Fatalf("got %d want 400, body=%s", w.Code, w.Body.String())
}
if !strings.Contains(w.Body.String(), "metadata.attempt must be a string") {
t.Fatalf("expected string metadata error, got %s", w.Body.String())
}
}
func TestChatCompletionsRejectsObjectWorkspaceMetadata(t *testing.T) {
srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "llama"}, &fakeRunService{}, nil)
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{
@ -819,12 +882,12 @@ func TestChatCompletionsRejectsObjectWorkspaceMetadata(t *testing.T) {
if w.Code != http.StatusBadRequest {
t.Fatalf("got %d want 400, body=%s", w.Code, w.Body.String())
}
if !strings.Contains(w.Body.String(), "invalid metadata format") {
t.Fatalf("expected 'invalid metadata format' error, got %s", w.Body.String())
if !strings.Contains(w.Body.String(), "metadata.workspace must be a string") {
t.Fatalf("expected workspace string error, got %s", w.Body.String())
}
}
func TestResponsesAcceptsFlatNomadCodeMetadataAliases(t *testing.T) {
func TestResponsesPreservesGenericTaskMetadata(t *testing.T) {
fake := &fakeRunService{events: make(chan *iop.RunEvent, 2)}
fake.events <- &iop.RunEvent{Type: "delta", Delta: "ok"}
fake.events <- &iop.RunEvent{Type: "complete"}
@ -835,9 +898,7 @@ func TestResponsesAcceptsFlatNomadCodeMetadataAliases(t *testing.T) {
"input":"test",
"metadata":{
"request_id":"req-flat-001",
"task_id":"task-flat",
"source":"plane",
"inference":{"target":"metadata-target"}
"task_id":"task-flat"
}
}`))
w := httptest.NewRecorder()
@ -846,50 +907,12 @@ func TestResponsesAcceptsFlatNomadCodeMetadataAliases(t *testing.T) {
if w.Code != http.StatusOK {
t.Fatalf("status: got %d body=%s", w.Code, w.Body.String())
}
if fake.req.Metadata["nomadcode.task_id"] != "task-flat" {
t.Fatalf("nomadcode.task_id: got %q", fake.req.Metadata["nomadcode.task_id"])
}
if fake.req.Metadata["nomadcode.source"] != "plane" {
t.Fatalf("nomadcode.source: got %q", fake.req.Metadata["nomadcode.source"])
}
if fake.req.Metadata["source"] != "openai-responses" {
t.Fatalf("source should remain IOP-owned: got %q", fake.req.Metadata["source"])
if fake.req.Metadata["task_id"] != "task-flat" {
t.Fatalf("task_id: got %q", fake.req.Metadata["task_id"])
}
}
func TestResponsesStructuredNomadCodeMetadataWinsOverFlatAliases(t *testing.T) {
fake := &fakeRunService{events: make(chan *iop.RunEvent, 2)}
fake.events <- &iop.RunEvent{Type: "delta", Delta: "ok"}
fake.events <- &iop.RunEvent{Type: "complete"}
srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama"}, fake, nil)
req := httptest.NewRequest(http.MethodPost, "/v1/responses", strings.NewReader(`{
"model":"client-model",
"input":"test",
"metadata":{
"task_id":"task-flat",
"source":"plane",
"nomadcode":{"task_id":"task-structured","source":"manual"}
}
}`))
w := httptest.NewRecorder()
srv.routes().ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d body=%s", w.Code, w.Body.String())
}
if fake.req.Metadata["nomadcode.task_id"] != "task-structured" {
t.Fatalf("nomadcode.task_id: got %q", fake.req.Metadata["nomadcode.task_id"])
}
if fake.req.Metadata["nomadcode.source"] != "manual" {
t.Fatalf("nomadcode.source: got %q", fake.req.Metadata["nomadcode.source"])
}
if fake.req.Metadata["source"] != "openai-responses" {
t.Fatalf("source should remain IOP-owned: got %q", fake.req.Metadata["source"])
}
}
func TestResponsesConfiguredTargetWinsOverMetadataTarget(t *testing.T) {
func TestResponsesConfiguredTargetWinsOverRequestModel(t *testing.T) {
fake := &fakeRunService{events: make(chan *iop.RunEvent, 2)}
fake.events <- &iop.RunEvent{Type: "delta", Delta: "ok"}
fake.events <- &iop.RunEvent{Type: "complete"}
@ -897,8 +920,7 @@ func TestResponsesConfiguredTargetWinsOverMetadataTarget(t *testing.T) {
srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "config-target"}, fake, nil)
req := httptest.NewRequest(http.MethodPost, "/v1/responses", strings.NewReader(`{
"model":"client-model",
"input":"test",
"metadata":{"inference":{"target":"metadata-target"}}
"input":"test"
}`))
w := httptest.NewRecorder()
srv.routes().ServeHTTP(w, req)
@ -919,7 +941,7 @@ func TestResponsesConfiguredTargetWinsOverMetadataTarget(t *testing.T) {
}
}
func TestResponsesRejectsCLIMetadata(t *testing.T) {
func TestResponsesRejectsNonStringMetadata(t *testing.T) {
srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "llama"}, &fakeRunService{}, nil)
cases := []struct {
@ -927,7 +949,11 @@ func TestResponsesRejectsCLIMetadata(t *testing.T) {
body string
}{
{"cli only", `{"model":"m","input":"hi","metadata":{"cli":{"flag":"x"}}}`},
{"inference and cli", `{"model":"m","input":"hi","metadata":{"inference":{"target":"t"},"cli":{"flag":"x"}}}`},
{"inference target", `{"model":"m","input":"hi","metadata":{"inference":{"target":"t"}}}`},
{"nomadcode metadata", `{"model":"m","input":"hi","metadata":{"nomadcode":{"task_id":"t"}}}`},
{"source metadata", `{"model":"m","input":"hi","metadata":{"source":"manual"}}`},
{"number metadata", `{"model":"m","input":"hi","metadata":{"attempt":2}}`},
{"boolean metadata", `{"model":"m","input":"hi","metadata":{"urgent":true}}`},
}
for _, tc := range cases {
@ -959,12 +985,12 @@ func TestResponsesRejectsObjectWorkspaceMetadata(t *testing.T) {
if w.Code != http.StatusBadRequest {
t.Fatalf("got %d want 400, body=%s", w.Code, w.Body.String())
}
if !strings.Contains(w.Body.String(), "invalid metadata format") {
t.Fatalf("expected 'invalid metadata format' error, got %s", w.Body.String())
if !strings.Contains(w.Body.String(), "metadata.workspace must be a string") {
t.Fatalf("expected workspace string error, got %s", w.Body.String())
}
}
func TestResponsesReturnsNomadCodeCompatibleShape(t *testing.T) {
func TestResponsesReturnsOpenAICompatibleShape(t *testing.T) {
fake := &fakeRunService{events: make(chan *iop.RunEvent, 3)}
fake.events <- &iop.RunEvent{Type: "delta", Delta: "answer"}
fake.events <- &iop.RunEvent{Type: "complete", Usage: &iop.Usage{InputTokens: 1, OutputTokens: 1}}
@ -1041,8 +1067,7 @@ func TestResponsesStrictOutputNormalizesAgentResponse(t *testing.T) {
srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "llama", StrictOutput: true}, fake, nil)
req := httptest.NewRequest(http.MethodPost, "/v1/responses", strings.NewReader(`{
"model":"client-model",
"instructions":"Once you've completed the user's task, you must use the attempt_completion tool to present the result.\n\n<attempt_completion>\n<result>done</result>\n</attempt_completion>",
"input":"finish"
"input":"Once you've completed the user's task, you must use the attempt_completion tool to present the result.\n\n<attempt_completion>\n<result>done</result>\n</attempt_completion>\n\nfinish"
}`))
w := httptest.NewRecorder()
srv.routes().ServeHTTP(w, req)
@ -1272,7 +1297,7 @@ func TestResponsesRouteCatalogDispatchesRoute(t *testing.T) {
}
}
func TestResponsesRouteCatalogTakesPriorityOverMetadataTarget(t *testing.T) {
func TestResponsesRouteCatalogDispatchesQueuePolicy(t *testing.T) {
fake := &fakeRunService{events: make(chan *iop.RunEvent, 2)}
fake.events <- &iop.RunEvent{Type: "delta", Delta: "ok"}
fake.events <- &iop.RunEvent{Type: "complete"}
@ -1284,8 +1309,7 @@ func TestResponsesRouteCatalogTakesPriorityOverMetadataTarget(t *testing.T) {
}, fake, nil)
req := httptest.NewRequest(http.MethodPost, "/v1/responses", strings.NewReader(`{
"model":"model-a",
"input":"test",
"metadata":{"inference":{"target":"metadata-target"}}
"input":"test"
}`))
w := httptest.NewRecorder()
srv.routes().ServeHTTP(w, req)
@ -1304,14 +1328,8 @@ func TestResponsesRouteCatalogTakesPriorityOverMetadataTarget(t *testing.T) {
}
}
func TestChatCompletionsEdgePassesNumCtxForAdapterEnforcement(t *testing.T) {
// Edge passes request options.num_ctx to the service unchanged.
// The Ollama adapter enforces Edge-owned context_size over this value.
fake := &fakeRunService{events: make(chan *iop.RunEvent, 2)}
fake.events <- &iop.RunEvent{Type: "delta", Delta: "ok"}
fake.events <- &iop.RunEvent{Type: "complete"}
srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "llama"}, fake, nil)
func TestChatCompletionsRejectsNumCtxOptionsWrapper(t *testing.T) {
srv := NewServer(config.EdgeOpenAIConf{Adapter: "ollama", Target: "llama"}, &fakeRunService{}, nil)
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{
"model":"from-request",
"messages":[{"role":"user","content":"hi"}],
@ -1320,15 +1338,8 @@ func TestChatCompletionsEdgePassesNumCtxForAdapterEnforcement(t *testing.T) {
w := httptest.NewRecorder()
srv.handleChatCompletions(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d body=%s", w.Code, w.Body.String())
}
options, ok := fake.req.Input["options"].(map[string]any)
if !ok {
t.Fatalf("options not passed to service: %+v", fake.req.Input)
}
if options["num_ctx"].(float64) != 8192 {
t.Fatalf("edge should pass num_ctx unchanged for adapter enforcement: got %v", options["num_ctx"])
if w.Code != http.StatusBadRequest {
t.Fatalf("got %d want 400, body=%s", w.Code, w.Body.String())
}
}
@ -1340,7 +1351,7 @@ func TestResolveRouteDispatchPreservesWorkspaceRequired(t *testing.T) {
},
}, &fakeRunService{}, nil)
dispatch, ok := srv.resolveRouteDispatch("codex", "")
dispatch, ok := srv.resolveRouteDispatch("codex")
if !ok {
t.Fatal("expected dispatch to succeed for codex route")
}
@ -1348,7 +1359,7 @@ func TestResolveRouteDispatchPreservesWorkspaceRequired(t *testing.T) {
t.Fatalf("expected workspace_required=true for codex route, got false")
}
dispatch, ok = srv.resolveRouteDispatch("llama3", "")
dispatch, ok = srv.resolveRouteDispatch("llama3")
if !ok {
t.Fatal("expected dispatch to succeed for llama3 route")
}
@ -1366,7 +1377,7 @@ func TestResolveRouteDispatchFallbackWorkspaceRequiredFalse(t *testing.T) {
},
}, &fakeRunService{}, nil)
dispatch, ok := srv.resolveRouteDispatch("unknown-model", "")
dispatch, ok := srv.resolveRouteDispatch("unknown-model")
if !ok {
t.Fatal("expected fallback dispatch to succeed")
}

View file

@ -10,7 +10,6 @@ type chatCompletionRequest struct {
Messages []chatMessage `json:"messages"`
Stream bool `json:"stream"`
Metadata json.RawMessage `json:"metadata,omitempty"`
Options map[string]any `json:"options,omitempty"`
MaxTokens *int `json:"max_tokens,omitempty"`
MaxCompletionTokens *int `json:"max_completion_tokens,omitempty"`
Temperature *float64 `json:"temperature,omitempty"`
@ -20,9 +19,6 @@ type chatCompletionRequest struct {
Seed *int `json:"seed,omitempty"`
Stop any `json:"stop,omitempty"`
ResponseFormat any `json:"response_format,omitempty"`
Format any `json:"format,omitempty"`
KeepAlive any `json:"keep_alive,omitempty"`
Think any `json:"think,omitempty"`
Tools []any `json:"tools,omitempty"`
}
@ -61,7 +57,7 @@ func (req chatCompletionRequest) runInput(prompt string, messages []chatMessage,
"prompt": prompt,
"messages": chatMessagesInput(messages),
}
options := cloneOptions(req.Options)
options := map[string]any{}
if req.MaxTokens != nil {
options["max_tokens"] = *req.MaxTokens
} else {
@ -81,15 +77,7 @@ func (req chatCompletionRequest) runInput(prompt string, messages []chatMessage,
if len(options) > 0 {
input["options"] = options
}
if req.Format != nil {
input["format"] = req.Format
}
if req.KeepAlive != nil {
input["keep_alive"] = req.KeepAlive
}
if req.Think != nil {
input["think"] = req.Think
} else if strictOutput {
if strictOutput {
input["think"] = false
}
if len(req.Tools) > 0 {
@ -98,17 +86,6 @@ func (req chatCompletionRequest) runInput(prompt string, messages []chatMessage,
return input
}
func cloneOptions(options map[string]any) map[string]any {
if len(options) == 0 {
return map[string]any{}
}
out := make(map[string]any, len(options))
for k, v := range options {
out[k] = v
}
return out
}
func setOptionInt(options map[string]any, key string, val *int) {
if val != nil {
options[key] = *val
@ -257,9 +234,9 @@ type responsesRequest struct {
Input json.RawMessage `json:"input"`
Instructions string `json:"instructions,omitempty"`
Stream bool `json:"stream"`
Background bool `json:"background,omitempty"`
Metadata json.RawMessage `json:"metadata,omitempty"`
MaxOutputTokens *int `json:"max_output_tokens,omitempty"`
MaxTokens *int `json:"max_tokens,omitempty"`
Temperature *float64 `json:"temperature,omitempty"`
TopP *float64 `json:"top_p,omitempty"`
}
@ -268,59 +245,12 @@ func (req responsesRequest) providerOptions() map[string]any {
options := map[string]any{}
if req.MaxOutputTokens != nil {
options["max_tokens"] = *req.MaxOutputTokens
} else if req.MaxTokens != nil {
options["max_tokens"] = *req.MaxTokens
}
setOptionFloat(options, "temperature", req.Temperature)
setOptionFloat(options, "top_p", req.TopP)
return options
}
type responsesMetadata struct {
RequestID string `json:"request_id,omitempty"`
Workspace string `json:"workspace,omitempty"`
TaskID string `json:"task_id,omitempty"`
Source string `json:"source,omitempty"`
Inference *responsesInferenceMetadata `json:"inference,omitempty"`
NomadCode *responsesNomadCodeMetadata `json:"nomadcode,omitempty"`
}
type responsesInferenceMetadata struct {
Target string `json:"target,omitempty"`
}
type responsesNomadCodeMetadata struct {
TaskID string `json:"task_id,omitempty"`
Source string `json:"source,omitempty"`
}
func (m *responsesMetadata) metadataForRun() map[string]string {
out := make(map[string]string)
if m.RequestID != "" {
out["request_id"] = m.RequestID
}
if m.Inference != nil && m.Inference.Target != "" {
out["inference.target"] = m.Inference.Target
}
taskID := m.TaskID
source := m.Source
if m.NomadCode != nil {
if m.NomadCode.TaskID != "" {
taskID = m.NomadCode.TaskID
}
if m.NomadCode.Source != "" {
source = m.NomadCode.Source
}
}
if taskID != "" {
out["nomadcode.task_id"] = taskID
}
if source != "" {
out["nomadcode.source"] = source
}
return out
}
type responsesResponse struct {
ID string `json:"id"`
Object string `json:"object"`

View file

@ -151,13 +151,13 @@ func TestBuildRunRequestCopiesMetadata(t *testing.T) {
Adapter: "cli",
Target: "codex",
Prompt: "hello",
Metadata: map[string]string{"source": "edge-ops-console", "x-custom": "value"},
Metadata: map[string]string{"request_id": "req-001", "x-custom": "value"},
})
if err != nil {
t.Fatalf("BuildRunRequest: %v", err)
}
if got := req.GetMetadata()["source"]; got != "edge-ops-console" {
t.Errorf("source: got %q, want %q", got, "edge-ops-console")
if got := req.GetMetadata()["request_id"]; got != "req-001" {
t.Errorf("request_id: got %q, want %q", got, "req-001")
}
if got := req.GetMetadata()["x-custom"]; got != "value" {
t.Errorf("x-custom: got %q, want %q", got, "value")
@ -170,16 +170,14 @@ func TestBuildRunRequestPreservesResponsesMetadataKeys(t *testing.T) {
Target: "llama",
Prompt: "test",
Metadata: map[string]string{
"request_id": "req-001",
"inference.target": "metadata-target",
"nomadcode.task_id": "task-123",
"nomadcode.source": "nomadcode",
"request_id": "req-001",
"task_id": "task-123",
},
})
if err != nil {
t.Fatalf("BuildRunRequest: %v", err)
}
for _, key := range []string{"request_id", "inference.target", "nomadcode.task_id", "nomadcode.source"} {
for _, key := range []string{"request_id", "task_id"} {
if req.GetMetadata()[key] == "" {
t.Errorf("metadata key %q not preserved", key)
}

View file

@ -133,7 +133,7 @@ done
RESPONSES_OUT="$TMP_DIR/responses.json"
curl -fsS \
-H "Content-Type: application/json" \
-d '{"model":"'"$MODEL"'","input":"echo '\''marker_content'\'' > marker.txt && echo '\''IOP_CLI_SMOKE_OK'\''","stream":false,"metadata":{"request_id":"e2e-openai-cli-workspace","task_id":"task-smoke","source":"manual","workspace":"'"$WORKSPACE"'","inference":{"target":"smoke-sh"}}}' \
-d '{"model":"'"$MODEL"'","input":"echo '\''marker_content'\'' > marker.txt && echo '\''IOP_CLI_SMOKE_OK'\''","stream":false,"metadata":{"request_id":"e2e-openai-cli-workspace","task_id":"task-smoke","workspace":"'"$WORKSPACE"'"}}' \
"http://127.0.0.1:$OPENAI_PORT/v1/responses" > "$RESPONSES_OUT"
grep -q "IOP_CLI_SMOKE_OK" "$RESPONSES_OUT"

View file

@ -254,7 +254,7 @@ grep -q "$MODEL" "$MODELS_OUT"
CHAT_OUT="$TMP_DIR/chat.json"
curl -fsS \
-H "Content-Type: application/json" \
-d '{"model":"client-request-model","think":false,"options":{"max_tokens":32},"messages":[{"role":"user","content":"Reply with exactly: smoke token"}]}' \
-d '{"model":"client-request-model","max_tokens":32,"messages":[{"role":"user","content":"Reply with exactly: smoke token"}]}' \
"http://127.0.0.1:$OPENAI_PORT/v1/chat/completions" > "$CHAT_OUT"
if [ "$MODE" = "fake" ]; then
grep -q "IOP_OPENAI_LEMONADE_OK" "$CHAT_OUT"
@ -269,7 +269,7 @@ fi
STREAM_OUT="$TMP_DIR/stream.txt"
curl -fsS -N \
-H "Content-Type: application/json" \
-d '{"model":"client-request-model","stream":true,"think":false,"options":{"max_tokens":32},"messages":[{"role":"user","content":"Reply with exactly: stream token"}]}' \
-d '{"model":"client-request-model","stream":true,"max_tokens":32,"messages":[{"role":"user","content":"Reply with exactly: stream token"}]}' \
"http://127.0.0.1:$OPENAI_PORT/v1/chat/completions" > "$STREAM_OUT"
if [ "$MODE" = "fake" ]; then
grep -q '"reasoning_content":"thinking..."' "$STREAM_OUT"

View file

@ -203,7 +203,7 @@ grep -q "reasoning_content" "$CHAT_OUT"
RESPONSES_OUT="$TMP_DIR/responses.json"
curl -fsS \
-H "Content-Type: application/json" \
-d '{"model":"client-request-model","input":"say the responses test token","stream":false,"metadata":{"request_id":"e2e-openai-ollama","task_id":"task-smoke","source":"manual","inference":{"target":"fake-ollama-model"}}}' \
-d '{"model":"client-request-model","input":"say the responses test token","stream":false,"metadata":{"request_id":"e2e-openai-ollama","task_id":"task-smoke"}}' \
"http://127.0.0.1:$OPENAI_PORT/v1/responses" > "$RESPONSES_OUT"
grep -q "IOP_OPENAI_OLLAMA_OK" "$RESPONSES_OUT"
grep -q '"output_text"' "$RESPONSES_OUT"

View file

@ -251,7 +251,7 @@ grep -q "$ROUTE_ALIAS" "$MODELS_OUT"
CHAT_OUT="$TMP_DIR/chat.json"
curl -fsS \
-H "Content-Type: application/json" \
-d "{\"model\":\"$ROUTE_ALIAS\",\"think\":false,\"options\":{\"max_tokens\":32},\"messages\":[{\"role\":\"user\",\"content\":\"Reply with exactly: smoke token\"}]}" \
-d "{\"model\":\"$ROUTE_ALIAS\",\"max_tokens\":32,\"messages\":[{\"role\":\"user\",\"content\":\"Reply with exactly: smoke token\"}]}" \
"http://127.0.0.1:$OPENAI_PORT/v1/chat/completions" > "$CHAT_OUT"
if [ "$MODE" = "fake" ]; then
grep -q "IOP_OPENAI_VLLM_OK" "$CHAT_OUT"
@ -264,7 +264,7 @@ grep -q '"finish_reason":"' "$CHAT_OUT"
STREAM_OUT="$TMP_DIR/stream.txt"
curl -fsS -N \
-H "Content-Type: application/json" \
-d "{\"model\":\"$ROUTE_ALIAS\",\"stream\":true,\"think\":false,\"options\":{\"max_tokens\":32},\"messages\":[{\"role\":\"user\",\"content\":\"Reply with exactly: stream token\"}]}" \
-d "{\"model\":\"$ROUTE_ALIAS\",\"stream\":true,\"max_tokens\":32,\"messages\":[{\"role\":\"user\",\"content\":\"Reply with exactly: stream token\"}]}" \
"http://127.0.0.1:$OPENAI_PORT/v1/chat/completions" > "$STREAM_OUT"
if [ "$MODE" = "fake" ]; then
grep -q '"content":"IOP_OPENAI_"' "$STREAM_OUT"