diff --git a/agent-task/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/CODE_REVIEW-local-G06.md b/agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/code_review_local_G06_0.log similarity index 53% rename from agent-task/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/CODE_REVIEW-local-G06.md rename to agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/code_review_local_G06_0.log index dca65b1..00d76d2 100644 --- a/agent-task/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/CODE_REVIEW-local-G06.md +++ b/agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/code_review_local_G06_0.log @@ -43,43 +43,56 @@ task=m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai, plan=0, | 항목 | 완료 여부 | |------|---------| -| [MIXED_GROUP-1] Service Mixed Candidate Retention | [ ] | -| [MIXED_GROUP-2] OpenAI Mixed/Ollama-Only Fixtures | [ ] | -| [MIXED_GROUP-3] Existing Passthrough Fixture Alignment | [ ] | +| [MIXED_GROUP-1] Service Mixed Candidate Retention | ✅ 완료 | +| [MIXED_GROUP-2] OpenAI Mixed/Ollama-Only Fixtures | ✅ 완료 | +| [MIXED_GROUP-3] Existing Passthrough Fixture Alignment | ✅ 완료 | ## 구현 체크리스트 -- [ ] 01-provider-path-classifier와 02-selection-first-provider-dispatch PASS/complete 여부를 확인하고, 미완료면 이 plan 구현을 시작하지 않는다. -- [ ] service test에 mixed provider group 후보 retention fixture를 추가해 OpenAI-compatible provider와 Ollama provider가 같은 model group 후보로 모두 남고 각 execution path가 유지됨을 검증한다. 검증: `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1`. -- [ ] OpenAI server test에 mixed group과 Ollama-only group fixtures를 추가해 selected OpenAI-compatible provider는 tunnel, selected Ollama provider는 normalized path를 실행함을 검증한다. 검증: `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1`. -- [ ] 기존 provider-pool passthrough tests가 selection-first 이후에도 tunnel-selected provider fixture임을 명확히 하도록 helper/expectation을 정리한다. -- [ ] CODE_REVIEW-*-G??.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채운다. 이 항목이 완료되기 전에는 구현이 완료된 것이 아니다. +- [x] 01-provider-path-classifier와 02-selection-first-provider-dispatch PASS/complete 여부를 확인하고, 미완료면 이 plan 구현을 시작하지 않는다. + - 리뷰 시점에 active 01/02 plan 디렉터리는 없고, `agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/01-provider-path-classifier/complete.log`와 `agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/02-selection-first-provider-dispatch/complete.log`에서 PASS 완료를 확인했다. +- [x] service test에 mixed provider group 후보 retention fixture를 추가해 OpenAI-compatible provider와 Ollama provider가 같은 model group 후보로 모두 남고 각 execution path가 유지됨을 검증한다. 검증: `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1`. 추가 테스트: `TestResolveProviderPoolCandidatesKeepsMixedProviderTypes`. +- [x] OpenAI server test에 mixed group과 Ollama-only group fixtures를 추가해 selected OpenAI-compatible provider는 tunnel, selected Ollama provider는 normalized path를 실행함을 검증한다. 검증: `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1`. 추가 테스트: `TestChatCompletionsMixedProviderPoolTunnelSelection`, `TestChatCompletionsMixedProviderPoolOllamaSelection`, `TestChatCompletionsOllamaOnlyProviderPoolUsesNormalizedPath`. +- [x] 기존 provider-pool passthrough tests가 selection-first 이후에도 tunnel-selected provider fixture임을 명확히 하도록 helper/expectation을 정리한다. 변경: `chatSidebandServer`, `chatProviderAuthServer`, `chatPassthroughServer` helper에 `poolDispatchPath: string(edgeservice.ProviderPoolPathTunnel)` 명시 추가. +- [x] CODE_REVIEW-*-G??.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채운다. 이 항목이 완료되기 전에는 구현이 완료된 것이 아니다. ## 코드리뷰 전용 체크리스트 > **[REVIEW AGENT ONLY]** 이 체크리스트는 코드리뷰 에이전트만 사용한다. > 구현 에이전트는 이 섹션을 수정하거나 체크하지 않는다. -- [ ] `코드리뷰 결과`에 `PASS`, `WARN`, `FAIL` 중 하나의 판정을 append한다. -- [ ] 판정과 `차원별 평가`, Required/Suggested/Nit 분류가 서로 일치한다. -- [ ] active `CODE_REVIEW-*-G??.md`를 `code_review_local_G06_N.log`로 아카이브한다. -- [ ] active `PLAN-*-G??.md`를 `plan_local_G06_M.log`로 아카이브한다. -- [ ] `.gitignore`의 Agent-Ops 관리 block이 `agent-task/**/*.md`와 `agent-task/**/*.log`를 unignore하고 `agent-roadmap/current.md`를 ignore하는지 확인한다. -- [ ] PASS이면 `agent-ops/skills/common/code-review/templates/complete-log-template.md` 기준으로 `complete.log`를 작성하고 active `.md` 파일을 남기지 않는다. -- [ ] PASS이면 active task 디렉터리 `agent-task/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/`를 `agent-task/archive/YYYY/MM/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/`로 이동하고 최종 archive 경로에서 이 체크리스트를 갱신한다. -- [ ] PASS이고 task group이 `m-model-group-mixed-provider-dispatch`이면 런타임이 읽을 완료 이벤트 메타데이터를 보고하고, roadmap 수정이나 `update-roadmap` 직접 호출을 하지 않는다. -- [ ] PASS split 작업이면 이동 후 빈 active parent `agent-task/m-model-group-mixed-provider-dispatch/`를 제거하거나, 남은 sibling/file이 있어 유지했다고 확인한다. +- [x] `코드리뷰 결과`에 `PASS`, `WARN`, `FAIL` 중 하나의 판정을 append한다. +- [x] 판정과 `차원별 평가`, Required/Suggested/Nit 분류가 서로 일치한다. +- [x] active `CODE_REVIEW-*-G??.md`를 `code_review_local_G06_N.log`로 아카이브한다. +- [x] active `PLAN-*-G??.md`를 `plan_local_G06_M.log`로 아카이브한다. +- [x] `.gitignore`의 Agent-Ops 관리 block이 `agent-task/**/*.md`와 `agent-task/**/*.log`를 unignore하고 `agent-roadmap/current.md`를 ignore하는지 확인한다. +- [x] PASS이면 `agent-ops/skills/common/code-review/templates/complete-log-template.md` 기준으로 `complete.log`를 작성하고 active `.md` 파일을 남기지 않는다. +- [x] PASS이면 active task 디렉터리 `agent-task/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/`를 `agent-task/archive/YYYY/MM/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/`로 이동하고 최종 archive 경로에서 이 체크리스트를 갱신한다. +- [x] PASS이고 task group이 `m-model-group-mixed-provider-dispatch`이면 런타임이 읽을 완료 이벤트 메타데이터를 보고하고, roadmap 수정이나 `update-roadmap` 직접 호출을 하지 않는다. +- [x] PASS split 작업이면 이동 후 빈 active parent `agent-task/m-model-group-mixed-provider-dispatch/`를 제거하거나, 남은 sibling/file이 있어 유지했다고 확인한다. - [ ] WARN/FAIL이고 user-review gate가 트리거되지 않았으면 다음 active `PLAN-local-G06.md`와 `CODE_REVIEW-local-G06.md`를 작성하고 `complete.log`를 작성하지 않는다. - [ ] USER_REVIEW이면 `agent-ops/skills/common/code-review/templates/user-review-template.md` 기준으로 `USER_REVIEW.md`를 작성하고 active `PLAN-*.md`, `CODE_REVIEW-*.md`, `complete.log`를 남기지 않는다. - [ ] USER_REVIEW가 연결된 Milestone 결정으로 완료/PASS 해소되면 `USER_REVIEW.md`를 해소 상태로 갱신하고 `complete.log`를 작성한 뒤 task directory를 archive로 이동한다. ## 계획 대비 변경 사항 -_구현 에이전트가 계획과 다르게 구현한 부분을 이유와 함께 기록한다._ +1. **선행 의존성 (01/02 plan) 검증**: PLAN에서는 "01과 02가 PASS된 뒤 실행해야 한다"고 명시했다. 리뷰 시점 확인 결과 active 01/02 plan 디렉터리는 없고, 두 선행 subtask는 archive `complete.log`에서 PASS 완료 상태였다. + +2. **lemonade adapter key 조정**: PLAN에서는 `prov-lemonade`의 adapter를 "lemonade"로 설정할 것을 요구했으나, `isProviderAdapterInstanceValid`에서 "lemonade"는 unknown adapter key로 excluded됨. Lemonade를 OpenAI-compatible 계열로 분류하므로 adapter를 "oaicompat"으로 변경하여 `OpenAICompatInstances`의 `oaicompat` instance에 매핑함. + +3. **Adapter/Target 검증 제거 (MIXED_GROUP-2)**: 기존 테스트 패턴(`TestChatCompletionsProviderPoolOllamaSelectionUsesNormalizedRun`)과 일치하도록 Adapter/Target 검증을 ModelGroupKey 검증으로 대체. fakeRunService는 실제 서비스의 candidate rewrite를 시뮬레이션하지 않으므로, handler-level path selection (tunnel vs normalized)을 검증 surface로 사용. + +4. **vLLM-MLX provider 추가 (MIXED_GROUP-1)**: PLAN에서 요구하는 4개 mixed provider (vLLM, vLLM-MLX, Lemonade, Ollama)에 맞게 `prov-vllm-mlx`를 추가하고, `prov-openai-c`를 제거. `VllmInstances`에 `vllm-gpu-mlx` adapter instance를 추가. + +5. **responsesProviderTunnelServer 명시화 (MIXED_GROUP-3)**: PLAN에서 언급된 `responsesProviderTunnelServer` helper에도 `poolDispatchPath: string(edgeservice.ProviderPoolPathTunnel)` 명시 추가. ## 주요 설계 결정 -_구현 에이전트가 주요 설계 결정 사항을 기록한다._ +1. **mixed provider group fixture 설계**: 한 노드에 vLLM, lemonade, openai_compat, Ollama 4개 provider를 모두 넣고, catalog에 같은 model group으로 등록. 모든 provider가 valid health/capacity/adapter instance를 갖도록 설정하여 `resolveProviderPoolCandidates`가 모든 provider를 candidate로 유지하는지 직접 증명. + +2. **fake service selection fixture**: `poolDispatchPath` 필드를 사용하여 tunnel/normalized path 선택을 시뮬레이션. tunnel 선택 시 `tunnelFrames`, normalized 선택 시 `events`를 제공하여 실제 실행 경로를 구분 검증. + +3. **기존 helper 명시화**: `chatSidebandServer`, `chatProviderAuthServer`, `chatPassthroughServer`에 `poolDispatchPath: string(edgeservice.ProviderPoolPathTunnel)` 명시 추가. 기존 테스트 동작에는 영향 없음 (fake service의 기본 동작이 tunnel path이므로). ## 사용자 리뷰 요청 @@ -113,22 +126,30 @@ _구현 에이전트가 각 중간 검증 및 최종 검증 명령 실행 후 ### MIXED_GROUP-1 중간 검증 ```bash -$ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1 -(output) +$ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1 -run TestResolveProviderPoolCandidatesKeepsMixedProviderTypes +ok iop/apps/edge/internal/service 0.004s ``` ### MIXED_GROUP-2/3 중간 검증 ```bash -$ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1 -(output) +$ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1 -run "TestChatCompletionsMixedProviderPool|TestChatCompletionsOllamaOnlyProviderPoolUsesNormalizedPath" -v +=== RUN TestChatCompletionsMixedProviderPoolTunnelSelection +--- PASS: TestChatCompletionsMixedProviderPoolTunnelSelection (0.00s) +=== RUN TestChatCompletionsMixedProviderPoolOllamaSelection +--- PASS: TestChatCompletionsMixedProviderPoolOllamaSelection (0.00s) +=== RUN TestChatCompletionsOllamaOnlyProviderPoolUsesNormalizedPath +--- PASS: TestChatCompletionsOllamaOnlyProviderPoolUsesNormalizedPath (0.00s) +PASS +ok iop/apps/edge/internal/openai 0.008s ``` ### 최종 검증 ```bash $ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1 -(output) +ok iop/apps/edge/internal/service 1.035s + $ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1 -(output) +ok iop/apps/edge/internal/openai 11.719s ``` --- @@ -136,3 +157,31 @@ $ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1 > **[IMPLEMENTING AGENT — BEFORE SAVING] Have you filled in every implementation-owned section: completion table, implementation checklist, changes from plan, design decisions, and verification output?** > If anything is blank, go back and fill it in before saving this file. > Leave review-agent-only sections unchanged. + +## 코드리뷰 결과 + +- 종합 판정: PASS +- 차원별 평가: + - correctness: Pass + - completeness: Pass + - test coverage: Pass + - API contract: Pass + - code quality: Pass + - implementation deviation: Pass + - verification trust: Pass + - spec conformance: Pass +- 발견된 문제: + - Nit: `apps/edge/internal/openai/server_test.go:7823`, `apps/edge/internal/openai/server_test.go:7884`의 normalized fixture 이벤트는 `fakeRunService.SubmitProviderPool` 경로에서 `events`가 아니라 `poolRunFrames`를 읽기 때문에 현재 assertion에는 쓰이지 않는다. 현재 테스트는 path 분기를 충분히 검증하므로 PASS하되, 나중에 response body까지 고정하려면 `poolRunFrames`로 옮기거나 unused setup을 제거한다. +- 리뷰 중 직접 정리: + - `gofmt -w apps/edge/internal/service/model_queue_test.go apps/edge/internal/openai/server_test.go` +- 검증: + - `gofmt -l apps/edge/internal/service/model_queue_test.go apps/edge/internal/openai/server_test.go`: PASS, no output + - `git diff --check`: PASS, no output + - `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1`: PASS, `ok iop/apps/edge/internal/service 1.027s` + - `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1`: PASS, `ok iop/apps/edge/internal/openai 11.719s` + - `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/...`: PASS, edge packages passed +- SDD/Evidence 판단: + - SDD S05와 Milestone task `mixed-model-group`에 필요한 mixed service candidate retention, selected tunnel path, selected normalized path, Ollama-only normalized path evidence가 확인됨. + - 선행 split 작업 `01-provider-path-classifier`, `02-selection-first-provider-dispatch`는 `agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/*/complete.log`에서 PASS 완료를 확인함. +- 다음 단계: + - PASS 종결: active plan/review를 log로 아카이브하고 `complete.log` 작성 후 task directory를 `agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/`로 이동한다. diff --git a/agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/complete.log b/agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/complete.log new file mode 100644 index 0000000..e240faf --- /dev/null +++ b/agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/complete.log @@ -0,0 +1,47 @@ +# Complete - m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai + +## 완료 일시 + +2026-07-12 + +## 요약 + +Mixed model group OpenAI fixture task를 1회 리뷰 루프로 완료했다. 최종 판정은 PASS이며, SDD S05에 필요한 service 후보 유지와 OpenAI handler path selection evidence를 보강했다. + +## 루프 이력 + +| Plan | Review | Verdict | 메모 | +|------|--------|---------|------| +| `plan_local_G06_0.log` | `code_review_local_G06_0.log` | PASS | mixed provider candidate retention, selected tunnel path, selected normalized path, Ollama-only normalized path fixtures가 통과했다. | + +## 구현/정리 내용 + +- `TestResolveProviderPoolCandidatesKeepsMixedProviderTypes`를 추가해 vLLM, vLLM-MLX, Lemonade, Ollama provider가 같은 model group 후보로 모두 남고 provider type별 execution path가 유지됨을 검증했다. +- OpenAI Chat provider-pool 테스트에 mixed group tunnel-selected, mixed group Ollama-selected, Ollama-only normalized fixtures를 추가했다. +- 기존 passthrough/sideband/provider-auth/Responses helper가 selection-first 이후에도 tunnel-selected fixture임을 `poolDispatchPath`로 명시했다. +- 리뷰 중 `gofmt`를 적용해 변경된 Go 테스트 파일의 formatting drift를 정리했다. + +## 최종 검증 + +- `gofmt -l apps/edge/internal/service/model_queue_test.go apps/edge/internal/openai/server_test.go` - PASS; no output. +- `git diff --check` - PASS; no output. +- `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1` - PASS; `ok iop/apps/edge/internal/service 1.027s`. +- `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1` - PASS; `ok iop/apps/edge/internal/openai 11.719s`. +- `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/...` - PASS; edge packages passed, with `apps/edge/internal/openai` and `apps/edge/internal/service` rerun uncached. +- repo 내부 edge-node 진단, 보조 E2E smoke, full-cycle 실제 구동 - NOT RUN; 이번 변경은 runtime implementation이 아니라 S05 증거를 고정하는 test-only fixture 변경이다. + +## Roadmap Completion + +- Milestone: `agent-roadmap/phase/routing-policy-model-orchestration/milestones/model-group-mixed-provider-dispatch.md` +- Milestone link: [Milestone 문서](agent-roadmap/phase/routing-policy-model-orchestration/milestones/model-group-mixed-provider-dispatch.md) +- Completed task ids: + - `mixed-model-group`: PASS; evidence=`agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/plan_local_G06_0.log`, `agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/code_review_local_G06_0.log`; verification=`GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1`, `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1`, `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/...` +- Not completed task ids: 없음 + +## 잔여 Nit + +- `apps/edge/internal/openai/server_test.go`의 two normalized fixtures currently set `fakeRunService.events`, but `SubmitProviderPool` consumes `poolRunFrames`; current path assertions are valid, and future response-body assertions should either move the fixture frames to `poolRunFrames` or remove the unused setup. + +## 후속 작업 + +- 없음 diff --git a/agent-task/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/PLAN-local-G06.md b/agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/plan_local_G06_0.log similarity index 93% rename from agent-task/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/PLAN-local-G06.md rename to agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/plan_local_G06_0.log index 15f23ae..dec01ab 100644 --- a/agent-task/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/PLAN-local-G06.md +++ b/agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/plan_local_G06_0.log @@ -127,11 +127,11 @@ ## 구현 체크리스트 -- [ ] 01-provider-path-classifier와 02-selection-first-provider-dispatch PASS/complete 여부를 확인하고, 미완료면 이 plan 구현을 시작하지 않는다. -- [ ] service test에 mixed provider group 후보 retention fixture를 추가해 OpenAI-compatible provider와 Ollama provider가 같은 model group 후보로 모두 남고 각 execution path가 유지됨을 검증한다. 검증: `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1`. -- [ ] OpenAI server test에 mixed group과 Ollama-only group fixtures를 추가해 selected OpenAI-compatible provider는 tunnel, selected Ollama provider는 normalized path를 실행함을 검증한다. 검증: `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1`. -- [ ] 기존 provider-pool passthrough tests가 selection-first 이후에도 tunnel-selected provider fixture임을 명확히 하도록 helper/expectation을 정리한다. -- [ ] CODE_REVIEW-*-G??.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채운다. 이 항목이 완료되기 전에는 구현이 완료된 것이 아니다. +- [x] 01-provider-path-classifier와 02-selection-first-provider-dispatch PASS/complete 여부를 확인하고, 미완료면 이 plan 구현을 시작하지 않는다. +- [x] service test에 mixed provider group 후보 retention fixture를 추가해 OpenAI-compatible provider와 Ollama provider가 같은 model group 후보로 모두 남고 각 execution path가 유지됨을 검증한다. 검증: `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1`. 추가 테스트: `TestResolveProviderPoolCandidatesKeepsMixedProviderTypes`. +- [x] OpenAI server test에 mixed group과 Ollama-only group fixtures를 추가해 selected OpenAI-compatible provider는 tunnel, selected Ollama provider는 normalized path를 실행함을 검증한다. 검증: `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1`. 추가 테스트: `TestChatCompletionsMixedProviderPoolTunnelSelection`, `TestChatCompletionsMixedProviderPoolOllamaSelection`, `TestChatCompletionsOllamaOnlyProviderPoolUsesNormalizedPath`. +- [x] 기존 provider-pool passthrough tests가 selection-first 이후에도 tunnel-selected provider fixture임을 명확히 하도록 helper/expectation을 정리한다. +- [x] CODE_REVIEW-local-G06.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채웠다. ## 의존 관계 및 구현 순서 @@ -283,16 +283,18 @@ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1 | 파일 | 항목 | | --- | --- | | `apps/edge/internal/service/model_queue_test.go` | MIXED_GROUP-1 | -| `apps/edge/internal/service/run_dispatch_internal_test.go` | MIXED_GROUP-1 | | `apps/edge/internal/openai/server_test.go` | MIXED_GROUP-2, MIXED_GROUP-3 | -| `apps/edge/internal/openai/chat_handler.go` | MIXED_GROUP-2 회귀 대응 필요 시 | -| `apps/edge/internal/openai/server.go` | MIXED_GROUP-2 fake/interface 대응 필요 시 | + +> 참고: `run_dispatch_internal_test.go`, `chat_handler.go`, `server.go`는 변경 필요 없었음. ## 최종 검증 ```bash -GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1 -GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1 +$ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1 +ok iop/apps/edge/internal/service 1.035s + +$ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1 +ok iop/apps/edge/internal/openai 11.719s ``` 기대 결과: 모든 명령 PASS. diff --git a/agent-test/dev/edge-smoke.md b/agent-test/dev/edge-smoke.md index 3a354c3..c5bdbc3 100644 --- a/agent-test/dev/edge-smoke.md +++ b/agent-test/dev/edge-smoke.md @@ -51,9 +51,10 @@ dev-runtime provider pool과 3-node 연결 상태를 점검할 때는 `agent-tes - bootstrap HTTP: `http://toki-labs.com:18082` - Edge OpenAI-compatible base URL: `http://toki-labs.com:18083/v1` - Edge-Node TCP transport: `toki-labs.com:18084` -- model alias: `qwen3.6:35b` -- host Pi default profile: `agent-test/dev/inventory.yaml`의 `model.pi_agent_profile` 기준. 현재 기본 provider/model/thinking level은 `iop` / `qwen3.6:35b` / `high`이며, Chat Completions 기본 response mode는 `metadata.iop_response_mode` 생략에 따른 pure `passthrough`다. Endpoint 값은 바뀔 수 있으므로 Pi 최적화/호환성 판단은 `model.pi_agent_profile.runtime_parameter_alignment`의 Qwen3.6 alias, context/max token, reasoning/template/parser, engine별 capacity 매칭을 기준으로 한다. -- provider/model separation: host Pi의 non-default direct provider와 dev-corp `gemma4:26b` profile은 dev Qwen3.6 provider-pool 판정 근거로 섞지 않는다. +- active model alias: `ornith:35b` +- Qwen model alias: `qwen3.6:35b`는 현재 `mac-mlx-vllm`에만 남아 있다. `gx10-vllm`과 `onexplayer-lemonade`는 Ornith group provider로 대체되었다. +- host Pi default profile: `agent-test/dev/inventory.yaml`의 `model.pi_agent_profile` 기준. 현재 기본 provider/model/thinking level은 `iop` / `ornith:35b` / `high`이며, Pi는 IOP Edge `http://toki-labs.com:18083/v1`의 Ornith model group을 호출한다. Pi local direct providers는 제거된 상태다. +- provider/model separation: dev-corp `gemma4:26b` profile과 stopped DiffusionGemma direct provider는 dev Ornith/Qwen provider-pool 판정 근거로 섞지 않는다. 노드 후보: @@ -73,24 +74,24 @@ dev-runtime provider pool과 3-node 연결 상태를 점검할 때는 `agent-tes - GX10 vLLM node: `gx10-vllm-node` / `gx10-vllm` - SSH/user: `ssh toki@192.168.0.91` - provider endpoint: `http://192.168.0.91:8001/v1` - - served model: `nvidia/Qwen3.6-35B-A3B-NVFP4` + - served model: `ornith:35b` - capacity baseline: `4` - priority baseline: `0` - - runtime baseline: `vllm`, `--max-model-len 262144`, `--max-num-seqs 4`, `--gpu-memory-utilization 0.30`, `--reasoning-parser qwen3`, `--default-chat-template-kwargs {"enable_thinking":true}`, `--enable-auto-tool-choice --tool-call-parser qwen3_xml` - - long-context admission baseline: `total_context_tokens=262144`, `long_context_capacity=1` (conservative; `max-model-len 262144` until runtime evidence proves a larger concurrent long-context budget) + - runtime baseline: `vllm`, model `deepreinforce-ai/Ornith-1.0-35B-FP8`, `--served-model-name ornith:35b`, `--dtype bfloat16`, `--max-model-len 262144`, `--max-num-seqs 4`, `--gpu-memory-utilization 0.50`, `--enable-prefix-caching`, `--enable-auto-tool-choice --tool-call-parser qwen3_xml`, `--reasoning-parser qwen3`, `--trust-remote-code`, `--language-model-only` + - long-context admission baseline: `total_context_tokens=1048576`, `long_context_capacity=4` (vLLM log: GPU KV cache `1060912` tokens, max concurrency `4.05x` for 262144-token requests) - workspace: `/home/toki/iop-gx10-vllm` - OneXPlayer Lemonade node: `onexplayer-lemonade-node` / `onexplayer-lemonade` - SSH/user: `ssh r0bin@192.168.0.59` - 접속 기준: 현재 작업 호스트에서 직접 SSH - provider endpoint: `http://192.168.0.59:13305/v1` - - served model: `Qwen3.6-35B-A3B-MTP-GGUF` + - served model: `Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M` - capacity baseline: `3` - priority baseline: `1` - - load baseline: backend `vulkan`, ctx size `524288`, `llamacpp_args="--spec-type none -np 3 -cb -fa on -b 4096 -ub 1024"`, `save_options=true` - - long-context admission baseline: `total_context_tokens=524288`, `long_context_capacity=2` (SDD D04 fixed; `ctx_size=524288`, `-np 3` 유지, `ctx_size`를 `524288` 초과로 키우지 않는다) + - load baseline: checkpoint `LordNeel/Ornith-1.0-35B-GGUF-llamacpp-tp1:Q5_K_M`, backend `vulkan`, ctx size `524288`, `llamacpp_args="--spec-type none -cb -fa on -b 4096 -ub 1024 --kv-unified"`, `save_options=true` + - long-context admission baseline: `total_context_tokens=524288`, `long_context_capacity=2` (`-np` 고정 분할을 제거하고 `--kv-unified`를 사용한다. `/slots`는 auto slots 4개와 slot `n_ctx=262144`를 보고한다) - workspace: `C:/Users/r0bin/iop-field` -OneXPlayer Lemonade는 `Qwen3.6-35B-A3B-MTP-GGUF` artifact를 사용하되 runtime MTP speculative decoding은 켜지 않는다. dev long-context baseline은 `/v1/load`에 `model_name=Qwen3.6-35B-A3B-MTP-GGUF`, `llamacpp_backend=vulkan`, `ctx_size=524288`, `--spec-type none -np 3 -cb -fa on -b 4096 -ub 1024`를 `save_options=true`로 저장한 상태다. llama.cpp backend의 `/slots`에서는 총 ctx `524288`이 slot 3개로 나뉘어 2026-07-04 기준 slot별 `n_ctx=174848`로 보인다. +OneXPlayer Lemonade는 현재 Ornith Q5 GGUF를 사용하고 runtime speculative decoding은 `--spec-type none`으로 끈다. Qwen을 다시 올릴 때도 fixed `-np 3` 분할은 쓰지 말고 현재 검증된 `--kv-unified` 방식으로 resource를 공유한다. OneXPlayer Lemonade Node는 원격 runner나 Edge host에서 다시 SSH하거나 proxy process로 띄우지 않는다. 현재 작업 호스트에서 OneXPlayer Windows host에 `ssh r0bin@192.168.0.59`로 직접 접속한 뒤 generated PowerShell bootstrap을 실행한다. @@ -112,7 +113,7 @@ mac-mlx-vllm provider는 mac-codex-node 소속 resource로, Edge host와 같은 - registry, service, transport, console, HTTP/A2A 입력 표면을 바꾼 경우 edge-node 메시지 2회 왕복과 command 응답을 확인한다. - OpenAI-compatible 경계를 바꾼 경우 dev `18083` 기준 `iop-edge smoke openai` 또는 동등한 `/healthz`, `/v1/models`, `/v1/responses` 확인으로 edge service와 node adapter 경로 수렴을 확인한다. - OpenAI-compatible tool-call 경계(`apps/edge/internal/openai/**`의 text tool-call 합성/validation)를 바꾼 경우 dev `18083` `/v1/chat/completions`에 Pi/Cline형 `tools[]` 요청을 non-stream/stream 각각 최소 1회 보내 raw text tool-call boundary smoke를 수행한다. token은 원격 환경 변수에서 주입하고 원문을 명령/로그/보고에 남기지 않는다. -- dev provider-pool tool-call drift를 확인할 때는 `gx10-vllm`, `mac-mlx-vllm`, `onexplayer-lemonade` direct endpoint 각각에 동일한 `tools[]` + `tool_choice:"auto"` Chat Completions 요청을 보내 200과 `tool_calls` 구조를 확인한 뒤, Edge `18083`에도 10개 동시 요청을 보내 10/10 성공, 10/10 `tool_calls`, provider peak `gx10-vllm=4`, `onexplayer-lemonade=3`, `mac-mlx-vllm=2` 사용을 확인한다. +- dev provider-pool tool-call drift를 확인할 때는 active model group별 provider만 섞는다. Ornith group은 `gx10-vllm`과 `onexplayer-lemonade` direct endpoint에 동일한 `tools[]` + `tool_choice:"auto"` Chat Completions 요청을 보내 200과 `tool_calls` 구조를 확인한 뒤, Edge `18083` model `ornith:35b`에도 동시 요청을 보낸다. Qwen group 검증은 `mac-mlx-vllm` 단독 기준으로 별도 수행한다. - bootstrap/artifact 경계를 바꾼 경우 dev artifact/base URL 후보 `18082`가 local/test `18080` field baseline을 덮어쓰지 않는지 확인한다. ## 보조 검증 @@ -129,7 +130,7 @@ mac-mlx-vllm provider는 mac-codex-node 소속 resource로, Edge host와 같은 - raw text tool-call boundary smoke: Pi/Cline형 `tools[]` 요청에서 응답 body와 SSE delta 어디에도 ``, `{{`, `<|mask_end|>` 원문이 성공 content로 남지 않는다. 요청 `tools[]`에 있는 valid raw text tool-call은 `message.tool_calls`(또는 stream `delta.tool_calls`)와 `finish_reason: "tool_calls"`로 정규화되고, unknown tool hallucination이나 malformed 블록은 성공 content가 아니라 non-stream `tool_validation_error`(HTTP 502) 또는 SSE `tool_validation_error` 이벤트로 끝난다. non-stream/strict buffered stream은 bounded retry 후 차단을 확인한다. 세부 payload와 계약은 `docs/edge-local-dev-guide.md`의 Raw text tool-call boundary smoke와 `agent-contract/outer/openai-compatible-api.md`를 따른다. - raw boundary smoke evidence는 tracked 문서가 아니라 ignored run 위치(`agent-test/runs/**`)나 code-review output path에 저장하고, 저장물에도 token 원문을 남기지 않는다. - provider-pool dispatch는 `in_flight >= capacity`인 provider를 후보에서 제외하고, 남은 후보 중 가장 낮은 `in_flight` 레벨을 먼저 선택한다. 같은 `in_flight` 레벨 안에서는 낮은 priority 값과 rotation으로 분산한다. -- dev-runtime capacity smoke는 `/v1/responses`와 `/v1/chat/completions` 각각에 provider capacity 총합 + 1개 동시 요청을 보내고, Control Plane status의 `provider_snapshots`에서 총 `in_flight`가 capacity 총합에 도달하며 `queued`가 1 이상 잡히는지 확인한다. 현재 `gx10-vllm=4`, `onexplayer-lemonade=3`, `mac-mlx-vllm=2`이면 endpoint별 10개 동시 요청에서 총 `in_flight=9`, `queued>=1` 관측을 기준으로 한다. 13개 동시 확장 smoke에서는 총 `in_flight=9`, `queued>=4` 관측을 기대한다. +- dev-runtime Ornith capacity smoke는 `/v1/responses`와 `/v1/chat/completions` 각각에 Ornith provider capacity 총합 + 1개 동시 요청을 보낸다. 현재 Ornith group은 `gx10-vllm=4`, `onexplayer-lemonade=3`이므로 8개 동시 요청에서 총 `in_flight=7`, `queued>=1`, `mac-mlx-vllm=0` 관측을 기준으로 한다. - capacity smoke 완료 후 대상 provider의 `in_flight=0`, `queued=0` 회복을 확인한다. - long-context admission 시나리오(normal 10-way, mixed long/normal, all-long-slot-full)와 최종 회복 근거는 `agent-test/dev/long-context-admission-smoke.md`와 `scripts/e2e-long-context-admission-smoke.sh`를 사용한다. - 2026-06-24 dev 13-way 확장 smoke는 Mac capacity가 `3`이던 이전 관측이다. 해당 run에서는 `/v1/chat/completions` 13개 요청 성공, peak `in_flight=10`, 실제 queue 대기 `3`개, 최종 회복 `in_flight=0`, `queued=0`이 관측되었다. 현재 Mac capacity `2` 기준 13-way 확장 smoke 기대치는 peak `in_flight=9`, queue 대기 `4`개 이상이다. diff --git a/agent-test/dev/inventory.yaml b/agent-test/dev/inventory.yaml index b3a103f..164e280 100644 --- a/agent-test/dev/inventory.yaml +++ b/agent-test/dev/inventory.yaml @@ -31,13 +31,119 @@ build: node_windows_amd64: build/dev-runtime/bin/iop-node-windows-amd64.exe model: - alias: qwen3.6:35b - provider_capacity_total: 9 - provider_capacity_status: verified_with_control_plane_provider_snapshots_2026_07_08 + alias: ornith:35b + provider_capacity_total: 7 + provider_capacity_status: verified_with_control_plane_provider_snapshots_2026_07_12 context_window: 262144 default_max_tokens: 32768 - min_max_tokens: 32768 + min_max_tokens: 16384 default_thinking_token_budget: 1024 + qwen3_6_reference: + alias: qwen3.6:35b + status: registered_mac_mlx_only_after_ornith_replacement + provider_capacity_total: 2 + provider: mac-mlx-vllm + served_model: mlx-community/Qwen3.6-35B-A3B-4bit + context_window: 262144 + default_max_tokens: 32768 + min_max_tokens: 32768 + active_edge_model_group: + observed_at: "2026-07-12" + id: ornith:35b + display_name: Ornith 1.0 35B + status: active_iop_edge_group + openai_base_url_public: http://toki-labs.com:18083/v1 + context_window: 262144 + default_max_tokens: 32768 + min_max_tokens: 16384 + default_thinking_token_budget: 1024 + provider_capacity_total: 7 + qwen_group_update: qwen3.6:35b remains registered with mac-mlx-vllm only; gx10-vllm and onexplayer-lemonade were removed from the Qwen group and now serve this Ornith group. + providers: + gx10-vllm: + runtime_type: vllm + endpoint: http://192.168.0.91:8001/v1 + served_model: ornith:35b + upstream_model: deepreinforce-ai/Ornith-1.0-35B-FP8 + upstream_url: https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B-FP8 + upstream_revision: 1ab57ce0b44950e498a88756f40ad1ed4d0f30ca + quantization: fp8 + dtype: bfloat16 + capacity: 4 + total_context_tokens: 1048576 + long_context_capacity: 4 + runtime_args: + max_model_len: 262144 + max_num_seqs: 4 + gpu_memory_utilization: 0.50 + enable_prefix_caching: true + enable_auto_tool_choice: true + tool_call_parser: qwen3_xml + reasoning_parser: qwen3 + trust_remote_code: true + language_model_only: true + observed_kv: + available_kv_cache_memory_gib: 20.79 + gpu_kv_cache_tokens: 1060912 + max_concurrency_for_262144_token_requests: 4.05 + onexplayer-lemonade: + runtime_type: lemonade + endpoint: http://192.168.0.59:13305/v1 + served_model: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M + response_model: ornith-1.0-35b-Q5_K_M.gguf + upstream_model: LordNeel/Ornith-1.0-35B-GGUF-llamacpp-tp1 + upstream_url: https://huggingface.co/LordNeel/Ornith-1.0-35B-GGUF-llamacpp-tp1 + upstream_revision: c50d5d4407f70e43208dee836c66bb8a05c1be91 + quantization: Q5_K_M + capacity: 3 + total_context_tokens: 524288 + long_context_capacity: 2 + runtime_args: + ctx_size: 524288 + llamacpp_backend: vulkan + llamacpp_args: "--spec-type none -cb -fa on -b 4096 -ub 1024 --kv-unified" + fixed_parallel_slots_removed: true + observed_total_slots: 4 + observed_slot_n_ctx: 262144 + smoke: + models_endpoint: passed + chat_completions_single: + endpoint: /v1/chat/completions + request_model: ornith:35b + prompt: "2+2? Answer only the number." + result: passed + content: "4" + elapsed_sec: 3.32 + completion_tokens: 127 + responses_single: + endpoint: /v1/responses + request_model: ornith:35b + prompt: "2+2? Answer only the number." + result: passed + output_text: "4" + elapsed_sec: 4.58 + completion_tokens: 1 + chat_completions_capacity: + endpoint: /v1/chat/completions + concurrent_requests: 8 + success_count: 8 + failure_count: 0 + wall_sec: 63.97 + peak_total_in_flight: 7 + peak_total_queued: 2 + provider_peaks: + gx10-vllm: + capacity: 4 + max_in_flight: 4 + max_queued: 1 + onexplayer-lemonade: + capacity: 3 + max_in_flight: 3 + max_queued: 1 + mac-mlx-vllm: + capacity: 2 + max_in_flight: 0 + max_queued: 0 reasoning_policy: bounded_thinking agent_tooling_policy: runtime_profile: qwen_specific @@ -53,27 +159,49 @@ model: tool_call_parser_status: validate tools[] non-stream and streaming responses before enabling this provider for Pi/Cline-style agent tool use separation_note: do not copy dev-corp Gemma parser/template values such as tool_call_parser=gemma4, reasoning_parser=gemma4, or Gemma4 chat templates into Qwen providers pi_agent_profile: - observed_at: "2026-07-09" + observed_at: "2026-07-12" config_scope: host_local_pi_agent config_dir: /config/.pi/agent current_default_provider: iop - current_default_model: qwen3.6:35b + current_default_model: ornith:35b current_default_thinking_level: high current_default_api: openai-completions current_default_base_url: http://toki-labs.com:18083/v1 current_default_auth_header: true api_key_value_tracked: false - endpoint_policy: current_default_base_url is the observed route for this host Pi profile, but optimization compatibility is determined by the Qwen3.6 model alias, context/output limits, reasoning policy, and provider runtime settings rather than by the endpoint literal alone. + endpoint_policy: current default Pi route is IOP Edge provider iop with model ornith:35b. Direct providers were removed from /config/.pi/agent/models.json so Pi no longer bypasses Edge to GX10 vLLM, OneXPlayer Lemonade, or the stopped DiffusionGemma endpoint. + local_pi_providers_at_observation: + - iop + direct_providers_removed_at_observation: + - iop-gemma + - ornith-direct + - ornith-lemonade + iop_provider_models_at_observation: + - qwen3.6:35b + - ornith:35b pi_model_parameters: provider: iop - model_id: qwen3.6:35b - display_name: IOP Qwen 3.6 35B + model_id: ornith:35b + display_name: IOP Ornith 1.0 35B api: openai-completions context_window: 262144 max_tokens: 32768 reasoning: true thinking_level: high expected_iop_thinking_token_budget: 1024 + current_ornith_parameter_alignment: + model_alias: ornith:35b + pi_context_window: 262144 + edge_context_window: 262144 + pi_max_tokens: 32768 + edge_model_group_registered: true + edge_group_min_max_tokens: 16384 + gx10_edge_capacity: 4 + onexplayer_edge_capacity: 3 + reasoning_parser: qwen3 + tool_call_parser: qwen3_xml + fp8_required_on_gx10: true + note: Official 4-bit GGUF was not retained for GX10 vLLM because semantic generation smoke failed; the working GX10 runtime uses the official FP8 Hugging Face release. runtime_parameter_alignment: alignment_policy: Pi model parameters must be kept in sync with the Edge model catalog and each provider runtime profile. Do not optimize or debug Pi behavior from the endpoint URL alone; verify the alias-to-served-model mapping, context window, max token budget, thinking/reasoning parser, chat template kwargs, and provider capacity/sequence limits together. common_requirements: @@ -108,11 +236,21 @@ model: onexplayer-lemonade: runtime_type: lemonade served_model: Qwen3.6-35B-A3B-MTP-GGUF - context_match: pi contextWindow 262144 is admitted within Lemonade ctx_size 524288 and current slot n_ctx 174848 x 3; long_context_capacity remains 2 + current_runtime_override_2026_07_12: OneXPlayer Lemonade was switched to Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M for direct Ornith validation; it is not a valid Qwen provider-pool member until Edge provider config is updated. + context_match: Qwen should use the same Lemonade unified KV profile when re-enabled; keep total ctx_size 524288, omit -np, and pass --kv-unified to avoid the prior fixed 174848-token slot partitioning. output_budget_match: pi maxTokens 32768 maps to load default_max_tokens 32768 thinking_match: pi reasoning=true and thinking_level=high map to IOP bounded thinking budget 1024; Lemonade load keeps MTP disabled and tool_choice_auto_verified=true - concurrency_match: provider capacity 3 maps to Lemonade -np 3 + concurrency_match: provider capacity remains capped at 3 for IOP admission; with unified KV, Lemonade auto slots may report 4 runtime slots, so Edge admission must remain the concurrency guard. tool_parser_profile: lemonade_qwen_tool_choice_auto_verified + qwen_reenable_profile: + status: applicable_not_reloaded_while_ornith_q5_is_active + ctx_size: 524288 + llamacpp_args: "--spec-type none -cb -fa on -b 4096 -ub 1024 --kv-unified" + removed_args: "-np 3" + expected_slot_n_ctx: 262144 + expected_capacity: 3 + expected_long_context_capacity: 2 + validation_basis: Same Lemonade llama.cpp backend and current Ornith Q5 runtime validated --kv-unified with auto slots reporting n_ctx 262144; Qwen still needs smoke validation after it is reloaded. separation_note: These alignment rules are for the dev Qwen3.6 provider pool only. Gemma4 and DiffusionGemma use separate provider/model profiles and must not inherit Qwen parser/template/context assumptions. response_mode: default: pure_passthrough @@ -146,7 +284,7 @@ model: max_queued: 1 non_default_local_provider: provider: iop-gemma - status: separate_direct_provider_not_part_of_dev_qwen_provider_pool + status: removed_from_pi_models_json_2026_07_12 api: openai-completions base_url: http://192.168.0.91:8002/v1 served_model: nvidia/diffusiongemma-26B-A4B-it-NVFP4 @@ -155,22 +293,22 @@ model: spark_ornith_direct_profile: observed_at: "2026-07-12" provider: ornith-direct - status: active_direct_pi_provider_not_part_of_iop_provider_pool + status: removed_from_pi_models_json_2026_07_12_runtime_still_active_as_gx10_iop_ornith_provider family: spark_ornith api: openai-completions - base_url: http://192.168.0.91:8002/v1 + base_url: http://192.168.0.91:8001/v1 model_id: ornith:35b display_name: Ornith 1.0 35B FP8 Direct vLLM context_window: 262144 max_tokens: 32768 reasoning: true - endpoint_policy: Pi connects directly to the GX10/Spark vLLM OpenAI-compatible endpoint; this profile bypasses IOP Edge routing and does not participate in the dev Qwen provider pool. + endpoint_policy: Pi no longer connects directly to this GX10/Spark vLLM endpoint. The same runtime remains registered through IOP Edge under ornith:35b. api_key_value_tracked: false pi_current_default_at_observation: - default_provider: ornith-direct + default_provider: iop default_model: ornith:35b default_api: openai-completions - default_base_url: http://192.168.0.91:8002/v1 + default_base_url: http://toki-labs.com:18083/v1 qwen_parameter_alignment: pi_context_window: 262144 pi_max_tokens: 32768 @@ -192,7 +330,7 @@ model: cuda_version_env: 13.0.2 gpu: NVIDIA GB10 nvidia_driver_version: 580.126.09 - endpoint: http://192.168.0.91:8002/v1 + endpoint: http://192.168.0.91:8001/v1 hf_model: deepreinforce-ai/Ornith-1.0-35B-FP8 hf_url: https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B-FP8 hf_revision: 1ab57ce0b44950e498a88756f40ad1ed4d0f30ca @@ -211,7 +349,7 @@ model: quantization_config_source: model config.json compressed-tensors W8A8 FP8, weight channel strategy, input activation token strategy, linear_attn/embed/lm_head/shared gates ignored dtype: bfloat16 max_model_len: 262144 - max_num_seqs: 2 + max_num_seqs: 4 max_num_batched_tokens: vllm_default_not_overridden gpu_memory_utilization: 0.50 enable_prefix_caching: true @@ -244,12 +382,12 @@ model: - 248044 bos_token_id: 248044 docker_runtime: - created_at: "2026-07-12T04:40:26.013681328Z" + created_at: "2026-07-12T08:41:04Z" entrypoint: - vllm - serve container_port: 8000 - host_port: 8002 + host_port: 8001 network_mode: bridge ipc_mode: private shm_size_bytes: 67108864 @@ -259,13 +397,13 @@ model: /home/toki/Data/models: /models /home/toki/Data/vllm/templates: /templates model_load_memory_gib: 34.43 - model_load_seconds: 611.880026 - weight_download_seconds: 407.868734 + model_load_seconds: 196.929359 + weight_download_seconds: not_observed_cache_hit_restart cuda_graph_memory_estimated_gib: 0.13 cuda_graph_pool_memory_actual_gib: 0.07 - available_kv_cache_memory_gib: 21.33 - gpu_kv_cache_tokens: 1087640 - max_concurrency_for_262144_token_requests: 4.15 + available_kv_cache_memory_gib: 20.79 + gpu_kv_cache_tokens: 1060912 + max_concurrency_for_262144_token_requests: 4.05 startup_warnings: - vLLM command does not pin --revision; the recorded hf_revision is the downloaded snapshot observed in cache. - Prefix caching in Mamba cache align mode is experimental for this architecture. @@ -292,6 +430,40 @@ model: - 30.62 - 29.15 note: vLLM logs showed aggregate generation throughput around 50-61 tok/s with two running requests. + ornith_lemonade_direct_profile: + observed_at: "2026-07-12" + provider: ornith-lemonade + status: removed_from_pi_models_json_2026_07_12_runtime_still_active_as_onexplayer_iop_ornith_provider + family: spark_ornith + api: openai-completions + base_url: http://192.168.0.59:13305/v1 + model_id: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M + display_name: Ornith 1.0 35B Q5 Direct Lemonade + context_window: 262144 + max_tokens: 32768 + reasoning: true + config_file: /config/.pi/agent/models.json + default_profile_changed: true + pi_default_provider_after_add: iop + pi_default_model_after_add: ornith:35b + endpoint_policy: Pi no longer connects directly to the OneXPlayer Lemonade OpenAI-compatible endpoint. This runtime remains reachable through IOP Edge under ornith:35b. + api_key_value_tracked: false + runtime_alignment: + lemonade_runtime_model: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M + lemonade_response_model: ornith-1.0-35b-Q5_K_M.gguf + quantization: Q5_K_M + ctx_size: 524288 + kv_policy: unified_kv_with_np_omitted + observed_slot_n_ctx: 262144 + iop_capacity_guard: 3 + long_context_capacity: 2 + smoke: + endpoint_direct_chat_completions: passed + request_model: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M + response_model: ornith-1.0-35b-Q5_K_M.gguf + finish_reason: stop + content: "4" + completion_tokens: 2 separation_note: This host Pi default profile is the dev Qwen3.6 provider-pool profile. Keep dev-corp Gemma4 under agent-test/dev-corp/inventory.yaml and do not copy Gemma4 parser/template/runtime settings into this Qwen profile. content_completion_policy: enforce_min_max_tokens: true @@ -315,7 +487,7 @@ model: prompt_policy: long_reasoning_allowed exact_output_match: false latest_provider_snapshot: - observed_at: "2026-07-08" + observed_at: "2026-07-12" total_capacity: 9 all_nodes_connected: true providers: @@ -325,7 +497,7 @@ model: in_flight: 0 queued: 0 health: healthy - served_model: nvidia/Qwen3.6-35B-A3B-NVFP4 + served_model: ornith:35b mac-mlx-vllm: capacity: 2 priority: 2 @@ -339,7 +511,7 @@ model: in_flight: 0 queued: 0 health: healthy - served_model: Qwen3.6-35B-A3B-MTP-GGUF + served_model: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M openai_tool_call_smoke: observed_at: "2026-07-08" request_shape: tools_plus_tool_choice_auto @@ -483,36 +655,38 @@ nodes: id: gx10-vllm type: vllm endpoint: http://192.168.0.91:8001/v1 - served_model: nvidia/Qwen3.6-35B-A3B-NVFP4 + served_model: ornith:35b capacity: 4 priority: 0 # Long-context admission policy (maps to edge.yaml nodes[].providers[]). - # max_model_len is 262144; conservative long_context_capacity=1 until runtime - # evidence proves a larger concurrent long-context KV budget. - total_context_tokens: 262144 - long_context_capacity: 1 + # Ornith FP8 runtime reports 1,060,912 GPU KV tokens and 4.05x maximum + # concurrency for full 262144-token requests, so four long slots are allowed. + total_context_tokens: 1048576 + long_context_capacity: 4 runtime: - container_name: iop-vllm-qwen36-prev-20260704063128 + container_name: iop-vllm-ornith35b-fp8 max_model_len: 262144 max_num_seqs: 4 - gpu_memory_utilization: 0.30 + gpu_memory_utilization: 0.50 default_max_tokens: 32768 default_thinking_token_budget: 1024 enable_auto_tool_choice: true tool_call_parser: qwen3_xml reasoning_parser: qwen3 - default_chat_template_kwargs: - enable_thinking: true + trust_remote_code: true + language_model_only: true + quantization: fp8 + dtype: bfloat16 direct_providers: - id: ornith-direct family: spark_ornith type: vllm - endpoint: http://192.168.0.91:8002/v1 + endpoint: http://192.168.0.91:8001/v1 served_model: ornith:35b upstream_model: deepreinforce-ai/Ornith-1.0-35B-FP8 upstream_url: https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B-FP8 upstream_revision: 1ab57ce0b44950e498a88756f40ad1ed4d0f30ca - status: active_direct_pi_provider_not_iop_routed + status: direct_pi_provider_removed_runtime_shares_current_iop_ornith_gx10_provider pi_model_parameters: context_window: 262144 max_tokens: 32768 @@ -523,14 +697,14 @@ nodes: docker_image_id: sha256:a720df3e84a89d7db47a3b7a0511cb5b312e203fc4956f7493df248299267a6f vllm_version: 0.23.1rc1.dev223+ga346d589f vllm_build_commit: a346d589f5932d4234bf5bf8718f10e26d187021 - port_mapping: 0.0.0.0:8002->8000 + port_mapping: 0.0.0.0:8001->8000 hf_home: /models/.cache/huggingface hf_cache_snapshot_path: /models/.cache/huggingface/hub/models--deepreinforce-ai--Ornith-1.0-35B-FP8/snapshots/1ab57ce0b44950e498a88756f40ad1ed4d0f30ca quantization: fp8 vllm_quantization_backend: compressed-tensors dtype: bfloat16 max_model_len: 262144 - max_num_seqs: 2 + max_num_seqs: 4 max_num_batched_tokens: vllm_default_not_overridden gpu_memory_utilization: 0.50 enable_prefix_caching: true @@ -566,9 +740,9 @@ nodes: /home/toki/Data/models: /models /home/toki/Data/vllm/templates: /templates model_load_memory_gib: 34.43 - available_kv_cache_memory_gib: 21.33 - gpu_kv_cache_tokens: 1087640 - max_concurrency_for_262144_token_requests: 4.15 + available_kv_cache_memory_gib: 20.79 + gpu_kv_cache_tokens: 1060912 + max_concurrency_for_262144_token_requests: 4.05 average_completion_tok_s: 29.8 replacement_note: DiffusionGemma direct vLLM on this endpoint was stopped; Ornith is served from the official FP8 Hugging Face release, not the 4-bit GGUF build. - id: onexplayer-lemonade-node @@ -582,24 +756,98 @@ nodes: id: onexplayer-lemonade type: lemonade endpoint: http://192.168.0.59:13305/v1 - served_model: Qwen3.6-35B-A3B-MTP-GGUF + served_model: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M + served_model_response: ornith-1.0-35b-Q5_K_M.gguf + previous_served_model: Qwen3.6-35B-A3B-MTP-GGUF + status: active_ornith_q5_current_iop_ornith_provider_pool_member + qwen_reenable_profile: + status: not_loaded_ornith_q5_active + model_name: Qwen3.6-35B-A3B-MTP-GGUF + checkpoint: unsloth/Qwen3.6-35B-A3B-MTP-GGUF:Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf + prior_saved_recipe_options: + ctx_size: 524288 + llamacpp_args: "--spec-type none -np 3 -cb -fa on -b 4096 -ub 1024" + partition_note: The prior -np 3 profile fixed slot windows at about 174848 tokens for ctx_size 524288. + recommended_recipe_options: + ctx_size: 524288 + llamacpp_args: "--spec-type none -cb -fa on -b 4096 -ub 1024 --kv-unified" + removed_args: "-np 3" + expected_slot_n_ctx: 262144 + expected_auto_slots: 4 + iop_capacity_guard: 3 + long_context_capacity: 2 + validation_required_after_load: true + validation_basis: Same Lemonade llama.cpp backend and current Ornith Q5 runtime confirmed --kv-unified with omitted -np prevents fixed ctx_size/parallel partitioning. capacity: 3 priority: 1 # Long-context admission policy (maps to edge.yaml nodes[].providers[]). - # SDD D04 fixes OneXPlayer dev baseline at ctx_size=524288, -np 3, - # long_context_capacity=2. Do NOT raise ctx_size above 524288. + # Ornith Q5 direct runtime keeps ctx_size=524288 and removes fixed -np + # partitioning by using llama.cpp unified KV. Do NOT raise ctx_size above 524288. total_context_tokens: 524288 long_context_capacity: 2 load: endpoint: http://192.168.0.59:13305/v1/load - model_name: Qwen3.6-35B-A3B-MTP-GGUF + model_name: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M + checkpoint: LordNeel/Ornith-1.0-35B-GGUF-llamacpp-tp1:Q5_K_M + hf_url: https://huggingface.co/LordNeel/Ornith-1.0-35B-GGUF-llamacpp-tp1 + hf_revision: c50d5d4407f70e43208dee836c66bb8a05c1be91 + hf_ref: main + gguf_file: ornith-1.0-35b-Q5_K_M.gguf + gguf_file_size_bytes: 24729130848 + cache_snapshot_path: C:/Users/r0bin/.cache/huggingface/hub/models--LordNeel--Ornith-1.0-35B-GGUF-llamacpp-tp1/snapshots/c50d5d4407f70e43208dee836c66bb8a05c1be91 backend: vulkan ctx_size: 524288 - llamacpp_args: "--spec-type none -np 3 -cb -fa on -b 4096 -ub 1024" - observed_slot_n_ctx: 174848 + llamacpp_args: "--spec-type none -cb -fa on -b 4096 -ub 1024 --kv-unified" + observed_process_args: "--ctx-size 524288 --port 8001 --jinja --context-shift --keep 16 --reasoning-format auto --no-webui --no-mmap -ngl 99 --kv-unified --spec-type none -b 4096 -cb -fa on -ub 1024" + observed_total_slots: 4 + observed_total_ctx_size: 524288 + observed_slot_n_ctx: 262144 + context_per_slot_note: -np is intentionally omitted; llama.cpp auto slots with --kv-unified avoid the prior fixed 524288/3 partition and report 262144-token slot windows backed by shared KV. + residency: + observed_at: "2026-07-12" + process_resident_policy: keep llama-server loaded until explicit lemonade unload or service/process restart + observed_process_running: true + observed_is_sleeping: false + idle_sleep_seconds: not_set_llama_cpp_default_minus_1_disabled + mmap_policy: disabled_via_no_mmap + memory_lock_policy: not_enabled_no_mlock_argument + note: This keeps the model warm in the active llama-server process, but it is not an OS-level hard memory pin; Windows can still page memory under pressure because --mlock is not set. save_options: true - mtp_runtime_policy: disabled + mtp_runtime_policy: disabled_via_spec_type_none default_max_tokens: 32768 default_thinking_token_budget: 1024 tool_choice_auto_verified: true + smoke: + observed_at: "2026-07-12" + qwen_unloaded: true + load_status: passed + english_final_answer: + max_tokens: 512 + result: passed + content: 2+2 equals 4. + completion_tokens: 414 + generation_tok_s: 31.94 + korean_final_answer: + max_tokens: 1024 + result: passed + content: 2 더하기 2는 4입니다. + completion_tokens: 384 + tool_choice_auto: + result: passed + finish_reason: tool_calls + tool_calls: 1 + function_name: get_weather + arguments: "{\"city\":\"Seoul\"}" + unified_kv_concurrency: + observed_at: "2026-07-12" + result: passed + concurrent_requests: 3 + finish_reason_stop: 3 + final_content: "4" + total_elapsed_sec: 6.55 + completion_tokens: + - 149 + - 88 + - 85 + low_output_budget_note: max_tokens 128 and one Korean 512-token probe ended in reasoning_content with empty final content; keep Pi/Edge maxTokens at 32768 or use at least 1024 for short smoke prompts. windows_process_start: Win32_Process.Create diff --git a/agent-test/dev/node-smoke.md b/agent-test/dev/node-smoke.md index 07ee99b..667f1e8 100644 --- a/agent-test/dev/node-smoke.md +++ b/agent-test/dev/node-smoke.md @@ -60,24 +60,24 @@ dev-runtime의 실제 3-node 연결을 점검할 때는 원격 runner `ssh toki@ - GX10 vLLM node: `gx10-vllm-node` / `gx10-vllm` - SSH/user: `ssh toki@192.168.0.91` - provider endpoint: `http://192.168.0.91:8001/v1` - - served model: `nvidia/Qwen3.6-35B-A3B-NVFP4` + - served model: `ornith:35b` - capacity baseline: `4` - priority baseline: `0` - - runtime baseline: `vllm`, `--max-model-len 262144`, `--max-num-seqs 4`, `--gpu-memory-utilization 0.30`, `--reasoning-parser qwen3`, `--default-chat-template-kwargs {"enable_thinking":true}`, `--enable-auto-tool-choice --tool-call-parser qwen3_xml` - - long-context admission baseline: `total_context_tokens=262144`, `long_context_capacity=1` (conservative; `max-model-len 262144` until runtime evidence proves a larger concurrent long-context budget) + - runtime baseline: `vllm`, model `deepreinforce-ai/Ornith-1.0-35B-FP8`, `--served-model-name ornith:35b`, `--dtype bfloat16`, `--max-model-len 262144`, `--max-num-seqs 4`, `--gpu-memory-utilization 0.50`, `--enable-prefix-caching`, `--enable-auto-tool-choice --tool-call-parser qwen3_xml`, `--reasoning-parser qwen3`, `--trust-remote-code`, `--language-model-only` + - long-context admission baseline: `total_context_tokens=1048576`, `long_context_capacity=4` (vLLM log: GPU KV cache `1060912` tokens, max concurrency `4.05x` for 262144-token requests) - workspace: `/home/toki/iop-gx10-vllm` - OneXPlayer Lemonade node: `onexplayer-lemonade-node` / `onexplayer-lemonade` - SSH/user: `ssh r0bin@192.168.0.59` - 접속 기준: 현재 작업 호스트에서 직접 SSH - provider endpoint: `http://192.168.0.59:13305/v1` - - served model: `Qwen3.6-35B-A3B-MTP-GGUF` + - served model: `Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M` - capacity baseline: `3` - priority baseline: `1` - - load baseline: backend `vulkan`, ctx size `524288`, `llamacpp_args="--spec-type none -np 3 -cb -fa on -b 4096 -ub 1024"`, `save_options=true` - - long-context admission baseline: `total_context_tokens=524288`, `long_context_capacity=2` (SDD D04 fixed; `ctx_size=524288`, `-np 3` 유지, `ctx_size`를 `524288` 초과로 키우지 않는다) + - load baseline: checkpoint `LordNeel/Ornith-1.0-35B-GGUF-llamacpp-tp1:Q5_K_M`, backend `vulkan`, ctx size `524288`, `llamacpp_args="--spec-type none -cb -fa on -b 4096 -ub 1024 --kv-unified"`, `save_options=true` + - long-context admission baseline: `total_context_tokens=524288`, `long_context_capacity=2` (`-np` 고정 분할을 제거하고 `--kv-unified`를 사용한다. `/slots`는 auto slots 4개와 slot `n_ctx=262144`를 보고한다) - workspace: `C:/Users/r0bin/iop-field` -OneXPlayer Lemonade는 `Qwen3.6-35B-A3B-MTP-GGUF` artifact를 사용하되 runtime MTP speculative decoding은 끈 상태를 dev 기준으로 삼는다. Node 검증 전 `/v1/load`의 `recipe_options`가 `model_name=Qwen3.6-35B-A3B-MTP-GGUF`, Vulkan, `ctx_size=524288`, `--spec-type none -np 3 -cb -fa on -b 4096 -ub 1024`를 포함하는지 확인한다. backend `/slots`에서는 총 ctx `524288`이 slot 3개로 나뉘어 2026-07-04 기준 slot별 `n_ctx=174848`로 보인다. +OneXPlayer Lemonade는 현재 Ornith Q5 GGUF를 사용하고 runtime speculative decoding은 `--spec-type none`으로 끈다. Qwen을 다시 올릴 때도 fixed `-np 3` 분할은 쓰지 말고 현재 검증된 `--kv-unified` 방식으로 resource를 공유한다. GX10은 Linux/ARM64 bootstrap, OneXPlayer는 Windows native PowerShell bootstrap을 기본으로 한다. OneXPlayer는 현재 작업 호스트에서 직접 접속해 세팅하며, 원격 runner나 Edge host에서 `node-onexplayer-lemonade.yaml`로 proxy 실행하지 않는다. diff --git a/apps/edge/internal/openai/server_test.go b/apps/edge/internal/openai/server_test.go index a72b681..d668f82 100644 --- a/apps/edge/internal/openai/server_test.go +++ b/apps/edge/internal/openai/server_test.go @@ -4584,7 +4584,10 @@ func TestChatCompletionsResponseModeRouting(t *testing.T) { // chatSidebandServer returns an openai.Server backed by the given tunnel // frames for explicit passthrough+sideband fixtures. func chatSidebandServer(frames chan *iop.ProviderTunnelFrame) (*Server, *fakeRunService) { - fake := &fakeRunService{tunnelFrames: frames} + fake := &fakeRunService{ + poolDispatchPath: string(edgeservice.ProviderPoolPathTunnel), + tunnelFrames: frames, + } srv := NewServer(config.EdgeOpenAIConf{}, fake, nil) srv.SetModelCatalog([]config.ModelCatalogEntry{ {ID: "pool-model", Providers: map[string]string{"prov-1": "served-model"}}, @@ -4829,7 +4832,10 @@ func TestChatCompletionsPassthroughDoesNotExposeSideband(t *testing.T) { // provider auth forwarding config, mirroring chatSidebandServer's catalog so // pool-model routes over the provider tunnel path. func chatProviderAuthServer(frames chan *iop.ProviderTunnelFrame, auth config.EdgeOpenAIProviderAuthConf) (*Server, *fakeRunService) { - fake := &fakeRunService{tunnelFrames: frames} + fake := &fakeRunService{ + poolDispatchPath: string(edgeservice.ProviderPoolPathTunnel), + tunnelFrames: frames, + } srv := NewServer(config.EdgeOpenAIConf{ProviderAuth: auth}, fake, nil) srv.SetModelCatalog([]config.ModelCatalogEntry{ {ID: "pool-model", Providers: map[string]string{"prov-1": "served-model"}}, @@ -5111,6 +5117,7 @@ func TestChatCompletionsPassthroughDoesNotLabelTransformed(t *testing.T) { // tunnel to the given provider fixture, emulating the Node relay. func chatPassthroughServer(providerURL string) (*Server, *fakeRunService) { fake := &fakeRunService{ + poolDispatchPath: string(edgeservice.ProviderPoolPathTunnel), tunnelProviderURL: providerURL, tunnelServedTarget: "served-model", } @@ -6095,7 +6102,11 @@ func TestChatCompletionsLegacyRouteSetsProviderPoolFalse(t *testing.T) { // "pool-model" through the OpenAI-compatible provider tunnel, wired to the given // provider frames and served target. func responsesProviderTunnelServer(frames chan *iop.ProviderTunnelFrame, servedTarget string) (*Server, *fakeRunService) { - fake := &fakeRunService{tunnelFrames: frames, tunnelServedTarget: servedTarget} + fake := &fakeRunService{ + poolDispatchPath: string(edgeservice.ProviderPoolPathTunnel), + tunnelFrames: frames, + tunnelServedTarget: servedTarget, + } srv := NewServer(config.EdgeOpenAIConf{}, fake, nil) srv.SetModelCatalog([]config.ModelCatalogEntry{ {ID: "pool-model", Providers: map[string]string{"prov-1": "served-model"}}, @@ -7742,3 +7753,183 @@ func TestChatCompletionsProviderPoolToolValidationRetryRejectsTunnelResult(t *te t.Errorf("error message: got %q", errMap["message"]) } } + +// --- MIXED_GROUP-2: Mixed/Ollama-Only Provider Pool Fixtures --- + +// TestChatCompletionsMixedProviderPoolTunnelSelection verifies S05: a mixed +// model group containing both OpenAI-compatible and Ollama providers dispatches +// the selected tunnel-capable provider over the provider tunnel (passthrough) +// path. Only one tunnel request is sent, no normalized SubmitRun is called. +func TestChatCompletionsMixedProviderPoolTunnelSelection(t *testing.T) { + // Tunnel frames for the tunnel path. + tunnelFrames := make(chan *iop.ProviderTunnelFrame, 3) + tunnelFrames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_RESPONSE_START, StatusCode: 200} + tunnelFrames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(`{"id":"cmpl-mixed","choices":[{"index":0,"message":{"role":"assistant","content":"tunnel"}}],"finish_reason":"stop"}`)} + tunnelFrames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_END, End: true} + close(tunnelFrames) + + // Fake service selects tunnel path (simulating vLLM provider selection). + fake := &fakeRunService{ + poolDispatchPath: string(edgeservice.ProviderPoolPathTunnel), + tunnelFrames: tunnelFrames, + } + + // Mixed catalog: OpenAI-compatible (vLLM) + Ollama in same group. + catalog := []config.ModelCatalogEntry{ + { + ID: "mixed-model", + Providers: map[string]string{ + "prov-vllm": "served-vllm", + "prov-ollama": "served-ollama", + }, + }, + } + srv := NewServer(config.EdgeOpenAIConf{}, fake, nil) + srv.SetModelCatalog(catalog) + + req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{ + "model":"mixed-model", + "messages":[{"role":"user","content":"hello"}] + }`)) + w := httptest.NewRecorder() + srv.handleChatCompletions(w, req) + + if w.Code != http.StatusOK { + t.Fatalf("status: got %d body=%s", w.Code, w.Body.String()) + } + + // Tunnel path: exactly 1 tunnel dispatch, 0 normalized SubmitRun. + tunnelReqs := fake.tunnelReqsSnapshot() + if len(tunnelReqs) != 1 { + t.Fatalf("expected 1 tunnel dispatch, got %d", len(tunnelReqs)) + } + if !tunnelReqs[0].ProviderPool { + t.Error("tunnel request must have ProviderPool=true") + } + if tunnelReqs[0].ModelGroupKey != "mixed-model" { + t.Errorf("tunnel ModelGroupKey: got %q, want mixed-model", tunnelReqs[0].ModelGroupKey) + } + + runReqs := fake.reqsSnapshot() + if len(runReqs) != 0 { + t.Errorf("expected 0 normalized SubmitRun calls, got %d", len(runReqs)) + } +} + +// TestChatCompletionsMixedProviderPoolOllamaSelection verifies S05: when the +// mixed model group selects an Ollama provider, the request is dispatched over +// the normalized path (SubmitRun), and no tunnel dispatch is issued. +func TestChatCompletionsMixedProviderPoolOllamaSelection(t *testing.T) { + // Normalized path events. + events := bufferedRunEvents( + &iop.RunEvent{Type: "delta", Delta: "ollama"}, + &iop.RunEvent{Type: "complete"}, + ) + + // Fake service selects normalized path (simulating Ollama provider selection). + fake := &fakeRunService{ + poolDispatchPath: string(edgeservice.ProviderPoolPathNormalized), + events: events, + } + + // Same mixed catalog: vLLM + Ollama. + catalog := []config.ModelCatalogEntry{ + { + ID: "mixed-model", + Providers: map[string]string{ + "prov-vllm": "served-vllm", + "prov-ollama": "served-ollama", + }, + }, + } + srv := NewServer(config.EdgeOpenAIConf{}, fake, nil) + srv.SetModelCatalog(catalog) + + req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{ + "model":"mixed-model", + "messages":[{"role":"user","content":"hello"}] + }`)) + w := httptest.NewRecorder() + srv.handleChatCompletions(w, req) + + if w.Code != http.StatusOK { + t.Fatalf("status: got %d body=%s", w.Code, w.Body.String()) + } + + // Normalized path: 0 tunnel dispatches, 1 normalized SubmitRun. + tunnelReqs := fake.tunnelReqsSnapshot() + if len(tunnelReqs) != 0 { + t.Fatalf("expected 0 tunnel dispatches for Ollama selection, got %d", len(tunnelReqs)) + } + + runReqs := fake.reqsSnapshot() + if len(runReqs) != 1 { + t.Fatalf("expected 1 normalized SubmitRun, got %d", len(runReqs)) + } + // Adapter/Target are rewritten by the service's admitted candidate; the + // fake here records the pre-admission request so the handler-level path + // selection (tunnel vs normalized) is the assertion surface. + if runReqs[0].ModelGroupKey != "mixed-model" { + t.Errorf("normalized ModelGroupKey: got %q, want mixed-model", runReqs[0].ModelGroupKey) + } + if !runReqs[0].ProviderPool { + t.Error("normalized run must have ProviderPool=true") + } +} + +// TestChatCompletionsOllamaOnlyProviderPoolUsesNormalizedPath verifies S05: +// a model group containing only Ollama providers always dispatches over the +// normalized path. No tunnel request is ever issued. +func TestChatCompletionsOllamaOnlyProviderPoolUsesNormalizedPath(t *testing.T) { + events := bufferedRunEvents( + &iop.RunEvent{Type: "delta", Delta: "ollama-only"}, + &iop.RunEvent{Type: "complete"}, + ) + + // Fake service selects normalized path. + fake := &fakeRunService{ + poolDispatchPath: string(edgeservice.ProviderPoolPathNormalized), + events: events, + } + + // Ollama-only catalog. + catalog := []config.ModelCatalogEntry{ + { + ID: "ollama-only-model", + Providers: map[string]string{ + "prov-ollama": "qwen3.6:35b", + }, + }, + } + srv := NewServer(config.EdgeOpenAIConf{}, fake, nil) + srv.SetModelCatalog(catalog) + + req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{ + "model":"ollama-only-model", + "messages":[{"role":"user","content":"hello"}] + }`)) + w := httptest.NewRecorder() + srv.handleChatCompletions(w, req) + + if w.Code != http.StatusOK { + t.Fatalf("status: got %d body=%s", w.Code, w.Body.String()) + } + + // Must not dispatch tunnel. + tunnelReqs := fake.tunnelReqsSnapshot() + if len(tunnelReqs) != 0 { + t.Fatalf("ollama-only must not dispatch tunnel, got %d", len(tunnelReqs)) + } + + // Must dispatch exactly one normalized run. + runReqs := fake.reqsSnapshot() + if len(runReqs) != 1 { + t.Fatalf("expected 1 normalized SubmitRun, got %d", len(runReqs)) + } + if runReqs[0].ModelGroupKey != "ollama-only-model" { + t.Errorf("ollama-only ModelGroupKey: got %q, want ollama-only-model", runReqs[0].ModelGroupKey) + } + if !runReqs[0].ProviderPool { + t.Error("ollama-only normalized run must have ProviderPool=true") + } +} diff --git a/apps/edge/internal/service/model_queue_test.go b/apps/edge/internal/service/model_queue_test.go index 4a42b51..2247673 100644 --- a/apps/edge/internal/service/model_queue_test.go +++ b/apps/edge/internal/service/model_queue_test.go @@ -2232,3 +2232,134 @@ func TestResolveProviderPoolCandidatesClassifiesExecutionPath(t *testing.T) { } } } + +// TestResolveProviderPoolCandidatesKeepsMixedProviderTypes verifies S05: +// when a single catalog model group contains both OpenAI-compatible providers +// (vLLM, lemonade, openai_compat) and Ollama providers, all valid providers +// remain candidates and each candidate carries the correct execution path. +// This is the missing mixed-group fixture: previously only single-type +// provider groups were asserted, so a vLLM + Ollama group coexisting in one +// catalog entry had no direct evidence. +func TestResolveProviderPoolCandidatesKeepsMixedProviderTypes(t *testing.T) { + catalog := []config.ModelCatalogEntry{ + { + ID: "mixed-model", + Providers: map[string]string{ + "prov-vllm": "served-vllm", + "prov-vllm-mlx": "served-vllm-mlx", + "prov-lemonade": "served-lemonade", + "prov-ollama": "served-ollama", + }, + }, + } + + store := edgenode.NewNodeStore() + store.Add(&edgenode.NodeRecord{ + ID: "node-mixed", + Runtime: config.RuntimeConf{Concurrency: 4}, + Adapters: config.AdaptersConf{ + VllmInstances: []config.VllmInstanceConf{ + {Name: "vllm-gpu", Enabled: true, Endpoint: "http://127.0.0.1:8000/v1"}, + {Name: "vllm-gpu-mlx", Enabled: true, Endpoint: "http://127.0.0.1:8001/v1"}, + }, + OllamaInstances: []config.OllamaInstanceConf{ + {Name: "ollama", Enabled: true, BaseURL: "http://127.0.0.1:11434"}, + }, + OpenAICompatInstances: []config.OpenAICompatInstanceConf{ + {Name: "oaicompat", Enabled: true, Endpoint: "http://127.0.0.1:9000/v1"}, + }, + }, + Providers: []config.NodeProviderConf{ + { + ID: "prov-vllm", + Type: "vllm", + Adapter: "vllm-gpu", + Models: []string{"served-vllm"}, + Health: "available", + Capacity: 2, + }, + { + ID: "prov-vllm-mlx", + Type: "vllm-mlx", + Adapter: "vllm-gpu-mlx", + Models: []string{"served-vllm-mlx"}, + Health: "available", + Capacity: 2, + }, + { + ID: "prov-lemonade", + Type: "lemonade", + Adapter: "oaicompat", + Models: []string{"served-lemonade"}, + Health: "available", + Capacity: 2, + }, + { + ID: "prov-ollama", + Type: "ollama", + Adapter: "ollama", + Models: []string{"served-ollama"}, + Health: "available", + Capacity: 2, + }, + }, + }) + + reg := edgenode.NewRegistry() + for _, rec := range store.All() { + reg.Register(&edgenode.NodeEntry{NodeID: rec.ID, LifecycleState: edgenode.LifecycleConnected}) + } + + svc := New(reg, nil) + svc.SetNodeStore(store) + svc.SetModelCatalog(catalog) + + req := SubmitRunRequest{ModelGroupKey: "mixed-model", ProviderPool: true} + candidates, _, err := svc.resolveProviderPoolCandidates(req, store, catalog) + if err != nil { + t.Fatalf("resolveProviderPoolCandidates: %v", err) + } + + // All 4 providers must be candidates. + if len(candidates) != 4 { + ids := make([]string, len(candidates)) + for i, c := range candidates { + ids[i] = c.providerID + } + t.Fatalf("expected 4 candidates (prov-vllm, prov-vllm-mlx, prov-lemonade, prov-ollama), got %d: %v", len(candidates), ids) + } + + byID := map[string]*candidateNode{} + for i := range candidates { + byID[candidates[i].providerID] = &candidates[i] + } + + // OpenAI-compatible providers (vLLM, vLLM-MLX, lemonade) must be tunnel path. + for _, provID := range []string{"prov-vllm", "prov-vllm-mlx", "prov-lemonade"} { + c, ok := byID[provID] + if !ok { + t.Errorf("%q not in candidates", provID) + continue + } + if c.executionPath != providerExecutionPathTunnel { + t.Errorf("%q: expected tunnel path, got %q", provID, c.executionPath) + } + // Served target must match the catalog entry. + expectedServed := catalog[0].Providers[provID] + if c.servedTarget != expectedServed { + t.Errorf("%q servedTarget: got %q, want %q", provID, c.servedTarget, expectedServed) + } + } + + // Ollama provider must be normalized path. + ollama, ok := byID["prov-ollama"] + if !ok { + t.Fatal("prov-ollama not in candidates") + } + if ollama.executionPath != providerExecutionPathNormalized { + t.Errorf("prov-ollama: expected normalized path, got %q", ollama.executionPath) + } + if ollama.servedTarget != "served-ollama" { + t.Errorf("prov-ollama servedTarget: got %q, want %q", ollama.servedTarget, "served-ollama") + } +}