chore: archive task files and update tests
- Move plan/code_review to archive - Update test files (edge, node, server, model_queue)
This commit is contained in:
parent
3cfe3181c4
commit
ceefb85055
8 changed files with 778 additions and 109 deletions
|
|
@ -43,43 +43,56 @@ task=m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai, plan=0,
|
|||
|
||||
| 항목 | 완료 여부 |
|
||||
|------|---------|
|
||||
| [MIXED_GROUP-1] Service Mixed Candidate Retention | [ ] |
|
||||
| [MIXED_GROUP-2] OpenAI Mixed/Ollama-Only Fixtures | [ ] |
|
||||
| [MIXED_GROUP-3] Existing Passthrough Fixture Alignment | [ ] |
|
||||
| [MIXED_GROUP-1] Service Mixed Candidate Retention | ✅ 완료 |
|
||||
| [MIXED_GROUP-2] OpenAI Mixed/Ollama-Only Fixtures | ✅ 완료 |
|
||||
| [MIXED_GROUP-3] Existing Passthrough Fixture Alignment | ✅ 완료 |
|
||||
|
||||
## 구현 체크리스트
|
||||
|
||||
- [ ] 01-provider-path-classifier와 02-selection-first-provider-dispatch PASS/complete 여부를 확인하고, 미완료면 이 plan 구현을 시작하지 않는다.
|
||||
- [ ] service test에 mixed provider group 후보 retention fixture를 추가해 OpenAI-compatible provider와 Ollama provider가 같은 model group 후보로 모두 남고 각 execution path가 유지됨을 검증한다. 검증: `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1`.
|
||||
- [ ] OpenAI server test에 mixed group과 Ollama-only group fixtures를 추가해 selected OpenAI-compatible provider는 tunnel, selected Ollama provider는 normalized path를 실행함을 검증한다. 검증: `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1`.
|
||||
- [ ] 기존 provider-pool passthrough tests가 selection-first 이후에도 tunnel-selected provider fixture임을 명확히 하도록 helper/expectation을 정리한다.
|
||||
- [ ] CODE_REVIEW-*-G??.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채운다. 이 항목이 완료되기 전에는 구현이 완료된 것이 아니다.
|
||||
- [x] 01-provider-path-classifier와 02-selection-first-provider-dispatch PASS/complete 여부를 확인하고, 미완료면 이 plan 구현을 시작하지 않는다.
|
||||
- 리뷰 시점에 active 01/02 plan 디렉터리는 없고, `agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/01-provider-path-classifier/complete.log`와 `agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/02-selection-first-provider-dispatch/complete.log`에서 PASS 완료를 확인했다.
|
||||
- [x] service test에 mixed provider group 후보 retention fixture를 추가해 OpenAI-compatible provider와 Ollama provider가 같은 model group 후보로 모두 남고 각 execution path가 유지됨을 검증한다. 검증: `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1`. 추가 테스트: `TestResolveProviderPoolCandidatesKeepsMixedProviderTypes`.
|
||||
- [x] OpenAI server test에 mixed group과 Ollama-only group fixtures를 추가해 selected OpenAI-compatible provider는 tunnel, selected Ollama provider는 normalized path를 실행함을 검증한다. 검증: `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1`. 추가 테스트: `TestChatCompletionsMixedProviderPoolTunnelSelection`, `TestChatCompletionsMixedProviderPoolOllamaSelection`, `TestChatCompletionsOllamaOnlyProviderPoolUsesNormalizedPath`.
|
||||
- [x] 기존 provider-pool passthrough tests가 selection-first 이후에도 tunnel-selected provider fixture임을 명확히 하도록 helper/expectation을 정리한다. 변경: `chatSidebandServer`, `chatProviderAuthServer`, `chatPassthroughServer` helper에 `poolDispatchPath: string(edgeservice.ProviderPoolPathTunnel)` 명시 추가.
|
||||
- [x] CODE_REVIEW-*-G??.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채운다. 이 항목이 완료되기 전에는 구현이 완료된 것이 아니다.
|
||||
|
||||
## 코드리뷰 전용 체크리스트
|
||||
|
||||
> **[REVIEW AGENT ONLY]** 이 체크리스트는 코드리뷰 에이전트만 사용한다.
|
||||
> 구현 에이전트는 이 섹션을 수정하거나 체크하지 않는다.
|
||||
|
||||
- [ ] `코드리뷰 결과`에 `PASS`, `WARN`, `FAIL` 중 하나의 판정을 append한다.
|
||||
- [ ] 판정과 `차원별 평가`, Required/Suggested/Nit 분류가 서로 일치한다.
|
||||
- [ ] active `CODE_REVIEW-*-G??.md`를 `code_review_local_G06_N.log`로 아카이브한다.
|
||||
- [ ] active `PLAN-*-G??.md`를 `plan_local_G06_M.log`로 아카이브한다.
|
||||
- [ ] `.gitignore`의 Agent-Ops 관리 block이 `agent-task/**/*.md`와 `agent-task/**/*.log`를 unignore하고 `agent-roadmap/current.md`를 ignore하는지 확인한다.
|
||||
- [ ] PASS이면 `agent-ops/skills/common/code-review/templates/complete-log-template.md` 기준으로 `complete.log`를 작성하고 active `.md` 파일을 남기지 않는다.
|
||||
- [ ] PASS이면 active task 디렉터리 `agent-task/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/`를 `agent-task/archive/YYYY/MM/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/`로 이동하고 최종 archive 경로에서 이 체크리스트를 갱신한다.
|
||||
- [ ] PASS이고 task group이 `m-model-group-mixed-provider-dispatch`이면 런타임이 읽을 완료 이벤트 메타데이터를 보고하고, roadmap 수정이나 `update-roadmap` 직접 호출을 하지 않는다.
|
||||
- [ ] PASS split 작업이면 이동 후 빈 active parent `agent-task/m-model-group-mixed-provider-dispatch/`를 제거하거나, 남은 sibling/file이 있어 유지했다고 확인한다.
|
||||
- [x] `코드리뷰 결과`에 `PASS`, `WARN`, `FAIL` 중 하나의 판정을 append한다.
|
||||
- [x] 판정과 `차원별 평가`, Required/Suggested/Nit 분류가 서로 일치한다.
|
||||
- [x] active `CODE_REVIEW-*-G??.md`를 `code_review_local_G06_N.log`로 아카이브한다.
|
||||
- [x] active `PLAN-*-G??.md`를 `plan_local_G06_M.log`로 아카이브한다.
|
||||
- [x] `.gitignore`의 Agent-Ops 관리 block이 `agent-task/**/*.md`와 `agent-task/**/*.log`를 unignore하고 `agent-roadmap/current.md`를 ignore하는지 확인한다.
|
||||
- [x] PASS이면 `agent-ops/skills/common/code-review/templates/complete-log-template.md` 기준으로 `complete.log`를 작성하고 active `.md` 파일을 남기지 않는다.
|
||||
- [x] PASS이면 active task 디렉터리 `agent-task/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/`를 `agent-task/archive/YYYY/MM/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/`로 이동하고 최종 archive 경로에서 이 체크리스트를 갱신한다.
|
||||
- [x] PASS이고 task group이 `m-model-group-mixed-provider-dispatch`이면 런타임이 읽을 완료 이벤트 메타데이터를 보고하고, roadmap 수정이나 `update-roadmap` 직접 호출을 하지 않는다.
|
||||
- [x] PASS split 작업이면 이동 후 빈 active parent `agent-task/m-model-group-mixed-provider-dispatch/`를 제거하거나, 남은 sibling/file이 있어 유지했다고 확인한다.
|
||||
- [ ] WARN/FAIL이고 user-review gate가 트리거되지 않았으면 다음 active `PLAN-local-G06.md`와 `CODE_REVIEW-local-G06.md`를 작성하고 `complete.log`를 작성하지 않는다.
|
||||
- [ ] USER_REVIEW이면 `agent-ops/skills/common/code-review/templates/user-review-template.md` 기준으로 `USER_REVIEW.md`를 작성하고 active `PLAN-*.md`, `CODE_REVIEW-*.md`, `complete.log`를 남기지 않는다.
|
||||
- [ ] USER_REVIEW가 연결된 Milestone 결정으로 완료/PASS 해소되면 `USER_REVIEW.md`를 해소 상태로 갱신하고 `complete.log`를 작성한 뒤 task directory를 archive로 이동한다.
|
||||
|
||||
## 계획 대비 변경 사항
|
||||
|
||||
_구현 에이전트가 계획과 다르게 구현한 부분을 이유와 함께 기록한다._
|
||||
1. **선행 의존성 (01/02 plan) 검증**: PLAN에서는 "01과 02가 PASS된 뒤 실행해야 한다"고 명시했다. 리뷰 시점 확인 결과 active 01/02 plan 디렉터리는 없고, 두 선행 subtask는 archive `complete.log`에서 PASS 완료 상태였다.
|
||||
|
||||
2. **lemonade adapter key 조정**: PLAN에서는 `prov-lemonade`의 adapter를 "lemonade"로 설정할 것을 요구했으나, `isProviderAdapterInstanceValid`에서 "lemonade"는 unknown adapter key로 excluded됨. Lemonade를 OpenAI-compatible 계열로 분류하므로 adapter를 "oaicompat"으로 변경하여 `OpenAICompatInstances`의 `oaicompat` instance에 매핑함.
|
||||
|
||||
3. **Adapter/Target 검증 제거 (MIXED_GROUP-2)**: 기존 테스트 패턴(`TestChatCompletionsProviderPoolOllamaSelectionUsesNormalizedRun`)과 일치하도록 Adapter/Target 검증을 ModelGroupKey 검증으로 대체. fakeRunService는 실제 서비스의 candidate rewrite를 시뮬레이션하지 않으므로, handler-level path selection (tunnel vs normalized)을 검증 surface로 사용.
|
||||
|
||||
4. **vLLM-MLX provider 추가 (MIXED_GROUP-1)**: PLAN에서 요구하는 4개 mixed provider (vLLM, vLLM-MLX, Lemonade, Ollama)에 맞게 `prov-vllm-mlx`를 추가하고, `prov-openai-c`를 제거. `VllmInstances`에 `vllm-gpu-mlx` adapter instance를 추가.
|
||||
|
||||
5. **responsesProviderTunnelServer 명시화 (MIXED_GROUP-3)**: PLAN에서 언급된 `responsesProviderTunnelServer` helper에도 `poolDispatchPath: string(edgeservice.ProviderPoolPathTunnel)` 명시 추가.
|
||||
|
||||
## 주요 설계 결정
|
||||
|
||||
_구현 에이전트가 주요 설계 결정 사항을 기록한다._
|
||||
1. **mixed provider group fixture 설계**: 한 노드에 vLLM, lemonade, openai_compat, Ollama 4개 provider를 모두 넣고, catalog에 같은 model group으로 등록. 모든 provider가 valid health/capacity/adapter instance를 갖도록 설정하여 `resolveProviderPoolCandidates`가 모든 provider를 candidate로 유지하는지 직접 증명.
|
||||
|
||||
2. **fake service selection fixture**: `poolDispatchPath` 필드를 사용하여 tunnel/normalized path 선택을 시뮬레이션. tunnel 선택 시 `tunnelFrames`, normalized 선택 시 `events`를 제공하여 실제 실행 경로를 구분 검증.
|
||||
|
||||
3. **기존 helper 명시화**: `chatSidebandServer`, `chatProviderAuthServer`, `chatPassthroughServer`에 `poolDispatchPath: string(edgeservice.ProviderPoolPathTunnel)` 명시 추가. 기존 테스트 동작에는 영향 없음 (fake service의 기본 동작이 tunnel path이므로).
|
||||
|
||||
## 사용자 리뷰 요청
|
||||
|
||||
|
|
@ -113,22 +126,30 @@ _구현 에이전트가 각 중간 검증 및 최종 검증 명령 실행 후
|
|||
|
||||
### MIXED_GROUP-1 중간 검증
|
||||
```bash
|
||||
$ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1
|
||||
(output)
|
||||
$ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1 -run TestResolveProviderPoolCandidatesKeepsMixedProviderTypes
|
||||
ok iop/apps/edge/internal/service 0.004s
|
||||
```
|
||||
|
||||
### MIXED_GROUP-2/3 중간 검증
|
||||
```bash
|
||||
$ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1
|
||||
(output)
|
||||
$ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1 -run "TestChatCompletionsMixedProviderPool|TestChatCompletionsOllamaOnlyProviderPoolUsesNormalizedPath" -v
|
||||
=== RUN TestChatCompletionsMixedProviderPoolTunnelSelection
|
||||
--- PASS: TestChatCompletionsMixedProviderPoolTunnelSelection (0.00s)
|
||||
=== RUN TestChatCompletionsMixedProviderPoolOllamaSelection
|
||||
--- PASS: TestChatCompletionsMixedProviderPoolOllamaSelection (0.00s)
|
||||
=== RUN TestChatCompletionsOllamaOnlyProviderPoolUsesNormalizedPath
|
||||
--- PASS: TestChatCompletionsOllamaOnlyProviderPoolUsesNormalizedPath (0.00s)
|
||||
PASS
|
||||
ok iop/apps/edge/internal/openai 0.008s
|
||||
```
|
||||
|
||||
### 최종 검증
|
||||
```bash
|
||||
$ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1
|
||||
(output)
|
||||
ok iop/apps/edge/internal/service 1.035s
|
||||
|
||||
$ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1
|
||||
(output)
|
||||
ok iop/apps/edge/internal/openai 11.719s
|
||||
```
|
||||
|
||||
---
|
||||
|
|
@ -136,3 +157,31 @@ $ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1
|
|||
> **[IMPLEMENTING AGENT — BEFORE SAVING] Have you filled in every implementation-owned section: completion table, implementation checklist, changes from plan, design decisions, and verification output?**
|
||||
> If anything is blank, go back and fill it in before saving this file.
|
||||
> Leave review-agent-only sections unchanged.
|
||||
|
||||
## 코드리뷰 결과
|
||||
|
||||
- 종합 판정: PASS
|
||||
- 차원별 평가:
|
||||
- correctness: Pass
|
||||
- completeness: Pass
|
||||
- test coverage: Pass
|
||||
- API contract: Pass
|
||||
- code quality: Pass
|
||||
- implementation deviation: Pass
|
||||
- verification trust: Pass
|
||||
- spec conformance: Pass
|
||||
- 발견된 문제:
|
||||
- Nit: `apps/edge/internal/openai/server_test.go:7823`, `apps/edge/internal/openai/server_test.go:7884`의 normalized fixture 이벤트는 `fakeRunService.SubmitProviderPool` 경로에서 `events`가 아니라 `poolRunFrames`를 읽기 때문에 현재 assertion에는 쓰이지 않는다. 현재 테스트는 path 분기를 충분히 검증하므로 PASS하되, 나중에 response body까지 고정하려면 `poolRunFrames`로 옮기거나 unused setup을 제거한다.
|
||||
- 리뷰 중 직접 정리:
|
||||
- `gofmt -w apps/edge/internal/service/model_queue_test.go apps/edge/internal/openai/server_test.go`
|
||||
- 검증:
|
||||
- `gofmt -l apps/edge/internal/service/model_queue_test.go apps/edge/internal/openai/server_test.go`: PASS, no output
|
||||
- `git diff --check`: PASS, no output
|
||||
- `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1`: PASS, `ok iop/apps/edge/internal/service 1.027s`
|
||||
- `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1`: PASS, `ok iop/apps/edge/internal/openai 11.719s`
|
||||
- `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/...`: PASS, edge packages passed
|
||||
- SDD/Evidence 판단:
|
||||
- SDD S05와 Milestone task `mixed-model-group`에 필요한 mixed service candidate retention, selected tunnel path, selected normalized path, Ollama-only normalized path evidence가 확인됨.
|
||||
- 선행 split 작업 `01-provider-path-classifier`, `02-selection-first-provider-dispatch`는 `agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/*/complete.log`에서 PASS 완료를 확인함.
|
||||
- 다음 단계:
|
||||
- PASS 종결: active plan/review를 log로 아카이브하고 `complete.log` 작성 후 task directory를 `agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/`로 이동한다.
|
||||
|
|
@ -0,0 +1,47 @@
|
|||
# Complete - m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai
|
||||
|
||||
## 완료 일시
|
||||
|
||||
2026-07-12
|
||||
|
||||
## 요약
|
||||
|
||||
Mixed model group OpenAI fixture task를 1회 리뷰 루프로 완료했다. 최종 판정은 PASS이며, SDD S05에 필요한 service 후보 유지와 OpenAI handler path selection evidence를 보강했다.
|
||||
|
||||
## 루프 이력
|
||||
|
||||
| Plan | Review | Verdict | 메모 |
|
||||
|------|--------|---------|------|
|
||||
| `plan_local_G06_0.log` | `code_review_local_G06_0.log` | PASS | mixed provider candidate retention, selected tunnel path, selected normalized path, Ollama-only normalized path fixtures가 통과했다. |
|
||||
|
||||
## 구현/정리 내용
|
||||
|
||||
- `TestResolveProviderPoolCandidatesKeepsMixedProviderTypes`를 추가해 vLLM, vLLM-MLX, Lemonade, Ollama provider가 같은 model group 후보로 모두 남고 provider type별 execution path가 유지됨을 검증했다.
|
||||
- OpenAI Chat provider-pool 테스트에 mixed group tunnel-selected, mixed group Ollama-selected, Ollama-only normalized fixtures를 추가했다.
|
||||
- 기존 passthrough/sideband/provider-auth/Responses helper가 selection-first 이후에도 tunnel-selected fixture임을 `poolDispatchPath`로 명시했다.
|
||||
- 리뷰 중 `gofmt`를 적용해 변경된 Go 테스트 파일의 formatting drift를 정리했다.
|
||||
|
||||
## 최종 검증
|
||||
|
||||
- `gofmt -l apps/edge/internal/service/model_queue_test.go apps/edge/internal/openai/server_test.go` - PASS; no output.
|
||||
- `git diff --check` - PASS; no output.
|
||||
- `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1` - PASS; `ok iop/apps/edge/internal/service 1.027s`.
|
||||
- `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1` - PASS; `ok iop/apps/edge/internal/openai 11.719s`.
|
||||
- `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/...` - PASS; edge packages passed, with `apps/edge/internal/openai` and `apps/edge/internal/service` rerun uncached.
|
||||
- repo 내부 edge-node 진단, 보조 E2E smoke, full-cycle 실제 구동 - NOT RUN; 이번 변경은 runtime implementation이 아니라 S05 증거를 고정하는 test-only fixture 변경이다.
|
||||
|
||||
## Roadmap Completion
|
||||
|
||||
- Milestone: `agent-roadmap/phase/routing-policy-model-orchestration/milestones/model-group-mixed-provider-dispatch.md`
|
||||
- Milestone link: [Milestone 문서](agent-roadmap/phase/routing-policy-model-orchestration/milestones/model-group-mixed-provider-dispatch.md)
|
||||
- Completed task ids:
|
||||
- `mixed-model-group`: PASS; evidence=`agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/plan_local_G06_0.log`, `agent-task/archive/2026/07/m-model-group-mixed-provider-dispatch/03-mixed-model-group-openai/code_review_local_G06_0.log`; verification=`GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1`, `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1`, `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/...`
|
||||
- Not completed task ids: 없음
|
||||
|
||||
## 잔여 Nit
|
||||
|
||||
- `apps/edge/internal/openai/server_test.go`의 two normalized fixtures currently set `fakeRunService.events`, but `SubmitProviderPool` consumes `poolRunFrames`; current path assertions are valid, and future response-body assertions should either move the fixture frames to `poolRunFrames` or remove the unused setup.
|
||||
|
||||
## 후속 작업
|
||||
|
||||
- 없음
|
||||
|
|
@ -127,11 +127,11 @@
|
|||
|
||||
## 구현 체크리스트
|
||||
|
||||
- [ ] 01-provider-path-classifier와 02-selection-first-provider-dispatch PASS/complete 여부를 확인하고, 미완료면 이 plan 구현을 시작하지 않는다.
|
||||
- [ ] service test에 mixed provider group 후보 retention fixture를 추가해 OpenAI-compatible provider와 Ollama provider가 같은 model group 후보로 모두 남고 각 execution path가 유지됨을 검증한다. 검증: `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1`.
|
||||
- [ ] OpenAI server test에 mixed group과 Ollama-only group fixtures를 추가해 selected OpenAI-compatible provider는 tunnel, selected Ollama provider는 normalized path를 실행함을 검증한다. 검증: `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1`.
|
||||
- [ ] 기존 provider-pool passthrough tests가 selection-first 이후에도 tunnel-selected provider fixture임을 명확히 하도록 helper/expectation을 정리한다.
|
||||
- [ ] CODE_REVIEW-*-G??.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채운다. 이 항목이 완료되기 전에는 구현이 완료된 것이 아니다.
|
||||
- [x] 01-provider-path-classifier와 02-selection-first-provider-dispatch PASS/complete 여부를 확인하고, 미완료면 이 plan 구현을 시작하지 않는다.
|
||||
- [x] service test에 mixed provider group 후보 retention fixture를 추가해 OpenAI-compatible provider와 Ollama provider가 같은 model group 후보로 모두 남고 각 execution path가 유지됨을 검증한다. 검증: `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1`. 추가 테스트: `TestResolveProviderPoolCandidatesKeepsMixedProviderTypes`.
|
||||
- [x] OpenAI server test에 mixed group과 Ollama-only group fixtures를 추가해 selected OpenAI-compatible provider는 tunnel, selected Ollama provider는 normalized path를 실행함을 검증한다. 검증: `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1`. 추가 테스트: `TestChatCompletionsMixedProviderPoolTunnelSelection`, `TestChatCompletionsMixedProviderPoolOllamaSelection`, `TestChatCompletionsOllamaOnlyProviderPoolUsesNormalizedPath`.
|
||||
- [x] 기존 provider-pool passthrough tests가 selection-first 이후에도 tunnel-selected provider fixture임을 명확히 하도록 helper/expectation을 정리한다.
|
||||
- [x] CODE_REVIEW-local-G06.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채웠다.
|
||||
|
||||
## 의존 관계 및 구현 순서
|
||||
|
||||
|
|
@ -283,16 +283,18 @@ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1
|
|||
| 파일 | 항목 |
|
||||
| --- | --- |
|
||||
| `apps/edge/internal/service/model_queue_test.go` | MIXED_GROUP-1 |
|
||||
| `apps/edge/internal/service/run_dispatch_internal_test.go` | MIXED_GROUP-1 |
|
||||
| `apps/edge/internal/openai/server_test.go` | MIXED_GROUP-2, MIXED_GROUP-3 |
|
||||
| `apps/edge/internal/openai/chat_handler.go` | MIXED_GROUP-2 회귀 대응 필요 시 |
|
||||
| `apps/edge/internal/openai/server.go` | MIXED_GROUP-2 fake/interface 대응 필요 시 |
|
||||
|
||||
> 참고: `run_dispatch_internal_test.go`, `chat_handler.go`, `server.go`는 변경 필요 없었음.
|
||||
|
||||
## 최종 검증
|
||||
|
||||
```bash
|
||||
GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1
|
||||
GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1
|
||||
$ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/service -count=1
|
||||
ok iop/apps/edge/internal/service 1.035s
|
||||
|
||||
$ GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai -count=1
|
||||
ok iop/apps/edge/internal/openai 11.719s
|
||||
```
|
||||
|
||||
기대 결과: 모든 명령 PASS.
|
||||
|
|
@ -51,9 +51,10 @@ dev-runtime provider pool과 3-node 연결 상태를 점검할 때는 `agent-tes
|
|||
- bootstrap HTTP: `http://toki-labs.com:18082`
|
||||
- Edge OpenAI-compatible base URL: `http://toki-labs.com:18083/v1`
|
||||
- Edge-Node TCP transport: `toki-labs.com:18084`
|
||||
- model alias: `qwen3.6:35b`
|
||||
- host Pi default profile: `agent-test/dev/inventory.yaml`의 `model.pi_agent_profile` 기준. 현재 기본 provider/model/thinking level은 `iop` / `qwen3.6:35b` / `high`이며, Chat Completions 기본 response mode는 `metadata.iop_response_mode` 생략에 따른 pure `passthrough`다. Endpoint 값은 바뀔 수 있으므로 Pi 최적화/호환성 판단은 `model.pi_agent_profile.runtime_parameter_alignment`의 Qwen3.6 alias, context/max token, reasoning/template/parser, engine별 capacity 매칭을 기준으로 한다.
|
||||
- provider/model separation: host Pi의 non-default direct provider와 dev-corp `gemma4:26b` profile은 dev Qwen3.6 provider-pool 판정 근거로 섞지 않는다.
|
||||
- active model alias: `ornith:35b`
|
||||
- Qwen model alias: `qwen3.6:35b`는 현재 `mac-mlx-vllm`에만 남아 있다. `gx10-vllm`과 `onexplayer-lemonade`는 Ornith group provider로 대체되었다.
|
||||
- host Pi default profile: `agent-test/dev/inventory.yaml`의 `model.pi_agent_profile` 기준. 현재 기본 provider/model/thinking level은 `iop` / `ornith:35b` / `high`이며, Pi는 IOP Edge `http://toki-labs.com:18083/v1`의 Ornith model group을 호출한다. Pi local direct providers는 제거된 상태다.
|
||||
- provider/model separation: dev-corp `gemma4:26b` profile과 stopped DiffusionGemma direct provider는 dev Ornith/Qwen provider-pool 판정 근거로 섞지 않는다.
|
||||
|
||||
노드 후보:
|
||||
|
||||
|
|
@ -73,24 +74,24 @@ dev-runtime provider pool과 3-node 연결 상태를 점검할 때는 `agent-tes
|
|||
- GX10 vLLM node: `gx10-vllm-node` / `gx10-vllm`
|
||||
- SSH/user: `ssh toki@192.168.0.91`
|
||||
- provider endpoint: `http://192.168.0.91:8001/v1`
|
||||
- served model: `nvidia/Qwen3.6-35B-A3B-NVFP4`
|
||||
- served model: `ornith:35b`
|
||||
- capacity baseline: `4`
|
||||
- priority baseline: `0`
|
||||
- runtime baseline: `vllm`, `--max-model-len 262144`, `--max-num-seqs 4`, `--gpu-memory-utilization 0.30`, `--reasoning-parser qwen3`, `--default-chat-template-kwargs {"enable_thinking":true}`, `--enable-auto-tool-choice --tool-call-parser qwen3_xml`
|
||||
- long-context admission baseline: `total_context_tokens=262144`, `long_context_capacity=1` (conservative; `max-model-len 262144` until runtime evidence proves a larger concurrent long-context budget)
|
||||
- runtime baseline: `vllm`, model `deepreinforce-ai/Ornith-1.0-35B-FP8`, `--served-model-name ornith:35b`, `--dtype bfloat16`, `--max-model-len 262144`, `--max-num-seqs 4`, `--gpu-memory-utilization 0.50`, `--enable-prefix-caching`, `--enable-auto-tool-choice --tool-call-parser qwen3_xml`, `--reasoning-parser qwen3`, `--trust-remote-code`, `--language-model-only`
|
||||
- long-context admission baseline: `total_context_tokens=1048576`, `long_context_capacity=4` (vLLM log: GPU KV cache `1060912` tokens, max concurrency `4.05x` for 262144-token requests)
|
||||
- workspace: `/home/toki/iop-gx10-vllm`
|
||||
- OneXPlayer Lemonade node: `onexplayer-lemonade-node` / `onexplayer-lemonade`
|
||||
- SSH/user: `ssh r0bin@192.168.0.59`
|
||||
- 접속 기준: 현재 작업 호스트에서 직접 SSH
|
||||
- provider endpoint: `http://192.168.0.59:13305/v1`
|
||||
- served model: `Qwen3.6-35B-A3B-MTP-GGUF`
|
||||
- served model: `Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M`
|
||||
- capacity baseline: `3`
|
||||
- priority baseline: `1`
|
||||
- load baseline: backend `vulkan`, ctx size `524288`, `llamacpp_args="--spec-type none -np 3 -cb -fa on -b 4096 -ub 1024"`, `save_options=true`
|
||||
- long-context admission baseline: `total_context_tokens=524288`, `long_context_capacity=2` (SDD D04 fixed; `ctx_size=524288`, `-np 3` 유지, `ctx_size`를 `524288` 초과로 키우지 않는다)
|
||||
- load baseline: checkpoint `LordNeel/Ornith-1.0-35B-GGUF-llamacpp-tp1:Q5_K_M`, backend `vulkan`, ctx size `524288`, `llamacpp_args="--spec-type none -cb -fa on -b 4096 -ub 1024 --kv-unified"`, `save_options=true`
|
||||
- long-context admission baseline: `total_context_tokens=524288`, `long_context_capacity=2` (`-np` 고정 분할을 제거하고 `--kv-unified`를 사용한다. `/slots`는 auto slots 4개와 slot `n_ctx=262144`를 보고한다)
|
||||
- workspace: `C:/Users/r0bin/iop-field`
|
||||
|
||||
OneXPlayer Lemonade는 `Qwen3.6-35B-A3B-MTP-GGUF` artifact를 사용하되 runtime MTP speculative decoding은 켜지 않는다. dev long-context baseline은 `/v1/load`에 `model_name=Qwen3.6-35B-A3B-MTP-GGUF`, `llamacpp_backend=vulkan`, `ctx_size=524288`, `--spec-type none -np 3 -cb -fa on -b 4096 -ub 1024`를 `save_options=true`로 저장한 상태다. llama.cpp backend의 `/slots`에서는 총 ctx `524288`이 slot 3개로 나뉘어 2026-07-04 기준 slot별 `n_ctx=174848`로 보인다.
|
||||
OneXPlayer Lemonade는 현재 Ornith Q5 GGUF를 사용하고 runtime speculative decoding은 `--spec-type none`으로 끈다. Qwen을 다시 올릴 때도 fixed `-np 3` 분할은 쓰지 말고 현재 검증된 `--kv-unified` 방식으로 resource를 공유한다.
|
||||
|
||||
OneXPlayer Lemonade Node는 원격 runner나 Edge host에서 다시 SSH하거나 proxy process로 띄우지 않는다. 현재 작업 호스트에서 OneXPlayer Windows host에 `ssh r0bin@192.168.0.59`로 직접 접속한 뒤 generated PowerShell bootstrap을 실행한다.
|
||||
|
||||
|
|
@ -112,7 +113,7 @@ mac-mlx-vllm provider는 mac-codex-node 소속 resource로, Edge host와 같은
|
|||
- registry, service, transport, console, HTTP/A2A 입력 표면을 바꾼 경우 edge-node 메시지 2회 왕복과 command 응답을 확인한다.
|
||||
- OpenAI-compatible 경계를 바꾼 경우 dev `18083` 기준 `iop-edge smoke openai` 또는 동등한 `/healthz`, `/v1/models`, `/v1/responses` 확인으로 edge service와 node adapter 경로 수렴을 확인한다.
|
||||
- OpenAI-compatible tool-call 경계(`apps/edge/internal/openai/**`의 text tool-call 합성/validation)를 바꾼 경우 dev `18083` `/v1/chat/completions`에 Pi/Cline형 `tools[]` 요청을 non-stream/stream 각각 최소 1회 보내 raw text tool-call boundary smoke를 수행한다. token은 원격 환경 변수에서 주입하고 원문을 명령/로그/보고에 남기지 않는다.
|
||||
- dev provider-pool tool-call drift를 확인할 때는 `gx10-vllm`, `mac-mlx-vllm`, `onexplayer-lemonade` direct endpoint 각각에 동일한 `tools[]` + `tool_choice:"auto"` Chat Completions 요청을 보내 200과 `tool_calls` 구조를 확인한 뒤, Edge `18083`에도 10개 동시 요청을 보내 10/10 성공, 10/10 `tool_calls`, provider peak `gx10-vllm=4`, `onexplayer-lemonade=3`, `mac-mlx-vllm=2` 사용을 확인한다.
|
||||
- dev provider-pool tool-call drift를 확인할 때는 active model group별 provider만 섞는다. Ornith group은 `gx10-vllm`과 `onexplayer-lemonade` direct endpoint에 동일한 `tools[]` + `tool_choice:"auto"` Chat Completions 요청을 보내 200과 `tool_calls` 구조를 확인한 뒤, Edge `18083` model `ornith:35b`에도 동시 요청을 보낸다. Qwen group 검증은 `mac-mlx-vllm` 단독 기준으로 별도 수행한다.
|
||||
- bootstrap/artifact 경계를 바꾼 경우 dev artifact/base URL 후보 `18082`가 local/test `18080` field baseline을 덮어쓰지 않는지 확인한다.
|
||||
|
||||
## 보조 검증
|
||||
|
|
@ -129,7 +130,7 @@ mac-mlx-vllm provider는 mac-codex-node 소속 resource로, Edge host와 같은
|
|||
- raw text tool-call boundary smoke: Pi/Cline형 `tools[]` 요청에서 응답 body와 SSE delta 어디에도 `<tool_call>`, `{{`, `<|mask_end|>` 원문이 성공 content로 남지 않는다. 요청 `tools[]`에 있는 valid raw text tool-call은 `message.tool_calls`(또는 stream `delta.tool_calls`)와 `finish_reason: "tool_calls"`로 정규화되고, unknown tool hallucination이나 malformed 블록은 성공 content가 아니라 non-stream `tool_validation_error`(HTTP 502) 또는 SSE `tool_validation_error` 이벤트로 끝난다. non-stream/strict buffered stream은 bounded retry 후 차단을 확인한다. 세부 payload와 계약은 `docs/edge-local-dev-guide.md`의 Raw text tool-call boundary smoke와 `agent-contract/outer/openai-compatible-api.md`를 따른다.
|
||||
- raw boundary smoke evidence는 tracked 문서가 아니라 ignored run 위치(`agent-test/runs/**`)나 code-review output path에 저장하고, 저장물에도 token 원문을 남기지 않는다.
|
||||
- provider-pool dispatch는 `in_flight >= capacity`인 provider를 후보에서 제외하고, 남은 후보 중 가장 낮은 `in_flight` 레벨을 먼저 선택한다. 같은 `in_flight` 레벨 안에서는 낮은 priority 값과 rotation으로 분산한다.
|
||||
- dev-runtime capacity smoke는 `/v1/responses`와 `/v1/chat/completions` 각각에 provider capacity 총합 + 1개 동시 요청을 보내고, Control Plane status의 `provider_snapshots`에서 총 `in_flight`가 capacity 총합에 도달하며 `queued`가 1 이상 잡히는지 확인한다. 현재 `gx10-vllm=4`, `onexplayer-lemonade=3`, `mac-mlx-vllm=2`이면 endpoint별 10개 동시 요청에서 총 `in_flight=9`, `queued>=1` 관측을 기준으로 한다. 13개 동시 확장 smoke에서는 총 `in_flight=9`, `queued>=4` 관측을 기대한다.
|
||||
- dev-runtime Ornith capacity smoke는 `/v1/responses`와 `/v1/chat/completions` 각각에 Ornith provider capacity 총합 + 1개 동시 요청을 보낸다. 현재 Ornith group은 `gx10-vllm=4`, `onexplayer-lemonade=3`이므로 8개 동시 요청에서 총 `in_flight=7`, `queued>=1`, `mac-mlx-vllm=0` 관측을 기준으로 한다.
|
||||
- capacity smoke 완료 후 대상 provider의 `in_flight=0`, `queued=0` 회복을 확인한다.
|
||||
- long-context admission 시나리오(normal 10-way, mixed long/normal, all-long-slot-full)와 최종 회복 근거는 `agent-test/dev/long-context-admission-smoke.md`와 `scripts/e2e-long-context-admission-smoke.sh`를 사용한다.
|
||||
- 2026-06-24 dev 13-way 확장 smoke는 Mac capacity가 `3`이던 이전 관측이다. 해당 run에서는 `/v1/chat/completions` 13개 요청 성공, peak `in_flight=10`, 실제 queue 대기 `3`개, 최종 회복 `in_flight=0`, `queued=0`이 관측되었다. 현재 Mac capacity `2` 기준 13-way 확장 smoke 기대치는 peak `in_flight=9`, queue 대기 `4`개 이상이다.
|
||||
|
|
|
|||
|
|
@ -31,13 +31,119 @@ build:
|
|||
node_windows_amd64: build/dev-runtime/bin/iop-node-windows-amd64.exe
|
||||
|
||||
model:
|
||||
alias: qwen3.6:35b
|
||||
provider_capacity_total: 9
|
||||
provider_capacity_status: verified_with_control_plane_provider_snapshots_2026_07_08
|
||||
alias: ornith:35b
|
||||
provider_capacity_total: 7
|
||||
provider_capacity_status: verified_with_control_plane_provider_snapshots_2026_07_12
|
||||
context_window: 262144
|
||||
default_max_tokens: 32768
|
||||
min_max_tokens: 32768
|
||||
min_max_tokens: 16384
|
||||
default_thinking_token_budget: 1024
|
||||
qwen3_6_reference:
|
||||
alias: qwen3.6:35b
|
||||
status: registered_mac_mlx_only_after_ornith_replacement
|
||||
provider_capacity_total: 2
|
||||
provider: mac-mlx-vllm
|
||||
served_model: mlx-community/Qwen3.6-35B-A3B-4bit
|
||||
context_window: 262144
|
||||
default_max_tokens: 32768
|
||||
min_max_tokens: 32768
|
||||
active_edge_model_group:
|
||||
observed_at: "2026-07-12"
|
||||
id: ornith:35b
|
||||
display_name: Ornith 1.0 35B
|
||||
status: active_iop_edge_group
|
||||
openai_base_url_public: http://toki-labs.com:18083/v1
|
||||
context_window: 262144
|
||||
default_max_tokens: 32768
|
||||
min_max_tokens: 16384
|
||||
default_thinking_token_budget: 1024
|
||||
provider_capacity_total: 7
|
||||
qwen_group_update: qwen3.6:35b remains registered with mac-mlx-vllm only; gx10-vllm and onexplayer-lemonade were removed from the Qwen group and now serve this Ornith group.
|
||||
providers:
|
||||
gx10-vllm:
|
||||
runtime_type: vllm
|
||||
endpoint: http://192.168.0.91:8001/v1
|
||||
served_model: ornith:35b
|
||||
upstream_model: deepreinforce-ai/Ornith-1.0-35B-FP8
|
||||
upstream_url: https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B-FP8
|
||||
upstream_revision: 1ab57ce0b44950e498a88756f40ad1ed4d0f30ca
|
||||
quantization: fp8
|
||||
dtype: bfloat16
|
||||
capacity: 4
|
||||
total_context_tokens: 1048576
|
||||
long_context_capacity: 4
|
||||
runtime_args:
|
||||
max_model_len: 262144
|
||||
max_num_seqs: 4
|
||||
gpu_memory_utilization: 0.50
|
||||
enable_prefix_caching: true
|
||||
enable_auto_tool_choice: true
|
||||
tool_call_parser: qwen3_xml
|
||||
reasoning_parser: qwen3
|
||||
trust_remote_code: true
|
||||
language_model_only: true
|
||||
observed_kv:
|
||||
available_kv_cache_memory_gib: 20.79
|
||||
gpu_kv_cache_tokens: 1060912
|
||||
max_concurrency_for_262144_token_requests: 4.05
|
||||
onexplayer-lemonade:
|
||||
runtime_type: lemonade
|
||||
endpoint: http://192.168.0.59:13305/v1
|
||||
served_model: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M
|
||||
response_model: ornith-1.0-35b-Q5_K_M.gguf
|
||||
upstream_model: LordNeel/Ornith-1.0-35B-GGUF-llamacpp-tp1
|
||||
upstream_url: https://huggingface.co/LordNeel/Ornith-1.0-35B-GGUF-llamacpp-tp1
|
||||
upstream_revision: c50d5d4407f70e43208dee836c66bb8a05c1be91
|
||||
quantization: Q5_K_M
|
||||
capacity: 3
|
||||
total_context_tokens: 524288
|
||||
long_context_capacity: 2
|
||||
runtime_args:
|
||||
ctx_size: 524288
|
||||
llamacpp_backend: vulkan
|
||||
llamacpp_args: "--spec-type none -cb -fa on -b 4096 -ub 1024 --kv-unified"
|
||||
fixed_parallel_slots_removed: true
|
||||
observed_total_slots: 4
|
||||
observed_slot_n_ctx: 262144
|
||||
smoke:
|
||||
models_endpoint: passed
|
||||
chat_completions_single:
|
||||
endpoint: /v1/chat/completions
|
||||
request_model: ornith:35b
|
||||
prompt: "2+2? Answer only the number."
|
||||
result: passed
|
||||
content: "4"
|
||||
elapsed_sec: 3.32
|
||||
completion_tokens: 127
|
||||
responses_single:
|
||||
endpoint: /v1/responses
|
||||
request_model: ornith:35b
|
||||
prompt: "2+2? Answer only the number."
|
||||
result: passed
|
||||
output_text: "4"
|
||||
elapsed_sec: 4.58
|
||||
completion_tokens: 1
|
||||
chat_completions_capacity:
|
||||
endpoint: /v1/chat/completions
|
||||
concurrent_requests: 8
|
||||
success_count: 8
|
||||
failure_count: 0
|
||||
wall_sec: 63.97
|
||||
peak_total_in_flight: 7
|
||||
peak_total_queued: 2
|
||||
provider_peaks:
|
||||
gx10-vllm:
|
||||
capacity: 4
|
||||
max_in_flight: 4
|
||||
max_queued: 1
|
||||
onexplayer-lemonade:
|
||||
capacity: 3
|
||||
max_in_flight: 3
|
||||
max_queued: 1
|
||||
mac-mlx-vllm:
|
||||
capacity: 2
|
||||
max_in_flight: 0
|
||||
max_queued: 0
|
||||
reasoning_policy: bounded_thinking
|
||||
agent_tooling_policy:
|
||||
runtime_profile: qwen_specific
|
||||
|
|
@ -53,27 +159,49 @@ model:
|
|||
tool_call_parser_status: validate tools[] non-stream and streaming responses before enabling this provider for Pi/Cline-style agent tool use
|
||||
separation_note: do not copy dev-corp Gemma parser/template values such as tool_call_parser=gemma4, reasoning_parser=gemma4, or Gemma4 chat templates into Qwen providers
|
||||
pi_agent_profile:
|
||||
observed_at: "2026-07-09"
|
||||
observed_at: "2026-07-12"
|
||||
config_scope: host_local_pi_agent
|
||||
config_dir: /config/.pi/agent
|
||||
current_default_provider: iop
|
||||
current_default_model: qwen3.6:35b
|
||||
current_default_model: ornith:35b
|
||||
current_default_thinking_level: high
|
||||
current_default_api: openai-completions
|
||||
current_default_base_url: http://toki-labs.com:18083/v1
|
||||
current_default_auth_header: true
|
||||
api_key_value_tracked: false
|
||||
endpoint_policy: current_default_base_url is the observed route for this host Pi profile, but optimization compatibility is determined by the Qwen3.6 model alias, context/output limits, reasoning policy, and provider runtime settings rather than by the endpoint literal alone.
|
||||
endpoint_policy: current default Pi route is IOP Edge provider iop with model ornith:35b. Direct providers were removed from /config/.pi/agent/models.json so Pi no longer bypasses Edge to GX10 vLLM, OneXPlayer Lemonade, or the stopped DiffusionGemma endpoint.
|
||||
local_pi_providers_at_observation:
|
||||
- iop
|
||||
direct_providers_removed_at_observation:
|
||||
- iop-gemma
|
||||
- ornith-direct
|
||||
- ornith-lemonade
|
||||
iop_provider_models_at_observation:
|
||||
- qwen3.6:35b
|
||||
- ornith:35b
|
||||
pi_model_parameters:
|
||||
provider: iop
|
||||
model_id: qwen3.6:35b
|
||||
display_name: IOP Qwen 3.6 35B
|
||||
model_id: ornith:35b
|
||||
display_name: IOP Ornith 1.0 35B
|
||||
api: openai-completions
|
||||
context_window: 262144
|
||||
max_tokens: 32768
|
||||
reasoning: true
|
||||
thinking_level: high
|
||||
expected_iop_thinking_token_budget: 1024
|
||||
current_ornith_parameter_alignment:
|
||||
model_alias: ornith:35b
|
||||
pi_context_window: 262144
|
||||
edge_context_window: 262144
|
||||
pi_max_tokens: 32768
|
||||
edge_model_group_registered: true
|
||||
edge_group_min_max_tokens: 16384
|
||||
gx10_edge_capacity: 4
|
||||
onexplayer_edge_capacity: 3
|
||||
reasoning_parser: qwen3
|
||||
tool_call_parser: qwen3_xml
|
||||
fp8_required_on_gx10: true
|
||||
note: Official 4-bit GGUF was not retained for GX10 vLLM because semantic generation smoke failed; the working GX10 runtime uses the official FP8 Hugging Face release.
|
||||
runtime_parameter_alignment:
|
||||
alignment_policy: Pi model parameters must be kept in sync with the Edge model catalog and each provider runtime profile. Do not optimize or debug Pi behavior from the endpoint URL alone; verify the alias-to-served-model mapping, context window, max token budget, thinking/reasoning parser, chat template kwargs, and provider capacity/sequence limits together.
|
||||
common_requirements:
|
||||
|
|
@ -108,11 +236,21 @@ model:
|
|||
onexplayer-lemonade:
|
||||
runtime_type: lemonade
|
||||
served_model: Qwen3.6-35B-A3B-MTP-GGUF
|
||||
context_match: pi contextWindow 262144 is admitted within Lemonade ctx_size 524288 and current slot n_ctx 174848 x 3; long_context_capacity remains 2
|
||||
current_runtime_override_2026_07_12: OneXPlayer Lemonade was switched to Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M for direct Ornith validation; it is not a valid Qwen provider-pool member until Edge provider config is updated.
|
||||
context_match: Qwen should use the same Lemonade unified KV profile when re-enabled; keep total ctx_size 524288, omit -np, and pass --kv-unified to avoid the prior fixed 174848-token slot partitioning.
|
||||
output_budget_match: pi maxTokens 32768 maps to load default_max_tokens 32768
|
||||
thinking_match: pi reasoning=true and thinking_level=high map to IOP bounded thinking budget 1024; Lemonade load keeps MTP disabled and tool_choice_auto_verified=true
|
||||
concurrency_match: provider capacity 3 maps to Lemonade -np 3
|
||||
concurrency_match: provider capacity remains capped at 3 for IOP admission; with unified KV, Lemonade auto slots may report 4 runtime slots, so Edge admission must remain the concurrency guard.
|
||||
tool_parser_profile: lemonade_qwen_tool_choice_auto_verified
|
||||
qwen_reenable_profile:
|
||||
status: applicable_not_reloaded_while_ornith_q5_is_active
|
||||
ctx_size: 524288
|
||||
llamacpp_args: "--spec-type none -cb -fa on -b 4096 -ub 1024 --kv-unified"
|
||||
removed_args: "-np 3"
|
||||
expected_slot_n_ctx: 262144
|
||||
expected_capacity: 3
|
||||
expected_long_context_capacity: 2
|
||||
validation_basis: Same Lemonade llama.cpp backend and current Ornith Q5 runtime validated --kv-unified with auto slots reporting n_ctx 262144; Qwen still needs smoke validation after it is reloaded.
|
||||
separation_note: These alignment rules are for the dev Qwen3.6 provider pool only. Gemma4 and DiffusionGemma use separate provider/model profiles and must not inherit Qwen parser/template/context assumptions.
|
||||
response_mode:
|
||||
default: pure_passthrough
|
||||
|
|
@ -146,7 +284,7 @@ model:
|
|||
max_queued: 1
|
||||
non_default_local_provider:
|
||||
provider: iop-gemma
|
||||
status: separate_direct_provider_not_part_of_dev_qwen_provider_pool
|
||||
status: removed_from_pi_models_json_2026_07_12
|
||||
api: openai-completions
|
||||
base_url: http://192.168.0.91:8002/v1
|
||||
served_model: nvidia/diffusiongemma-26B-A4B-it-NVFP4
|
||||
|
|
@ -155,22 +293,22 @@ model:
|
|||
spark_ornith_direct_profile:
|
||||
observed_at: "2026-07-12"
|
||||
provider: ornith-direct
|
||||
status: active_direct_pi_provider_not_part_of_iop_provider_pool
|
||||
status: removed_from_pi_models_json_2026_07_12_runtime_still_active_as_gx10_iop_ornith_provider
|
||||
family: spark_ornith
|
||||
api: openai-completions
|
||||
base_url: http://192.168.0.91:8002/v1
|
||||
base_url: http://192.168.0.91:8001/v1
|
||||
model_id: ornith:35b
|
||||
display_name: Ornith 1.0 35B FP8 Direct vLLM
|
||||
context_window: 262144
|
||||
max_tokens: 32768
|
||||
reasoning: true
|
||||
endpoint_policy: Pi connects directly to the GX10/Spark vLLM OpenAI-compatible endpoint; this profile bypasses IOP Edge routing and does not participate in the dev Qwen provider pool.
|
||||
endpoint_policy: Pi no longer connects directly to this GX10/Spark vLLM endpoint. The same runtime remains registered through IOP Edge under ornith:35b.
|
||||
api_key_value_tracked: false
|
||||
pi_current_default_at_observation:
|
||||
default_provider: ornith-direct
|
||||
default_provider: iop
|
||||
default_model: ornith:35b
|
||||
default_api: openai-completions
|
||||
default_base_url: http://192.168.0.91:8002/v1
|
||||
default_base_url: http://toki-labs.com:18083/v1
|
||||
qwen_parameter_alignment:
|
||||
pi_context_window: 262144
|
||||
pi_max_tokens: 32768
|
||||
|
|
@ -192,7 +330,7 @@ model:
|
|||
cuda_version_env: 13.0.2
|
||||
gpu: NVIDIA GB10
|
||||
nvidia_driver_version: 580.126.09
|
||||
endpoint: http://192.168.0.91:8002/v1
|
||||
endpoint: http://192.168.0.91:8001/v1
|
||||
hf_model: deepreinforce-ai/Ornith-1.0-35B-FP8
|
||||
hf_url: https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B-FP8
|
||||
hf_revision: 1ab57ce0b44950e498a88756f40ad1ed4d0f30ca
|
||||
|
|
@ -211,7 +349,7 @@ model:
|
|||
quantization_config_source: model config.json compressed-tensors W8A8 FP8, weight channel strategy, input activation token strategy, linear_attn/embed/lm_head/shared gates ignored
|
||||
dtype: bfloat16
|
||||
max_model_len: 262144
|
||||
max_num_seqs: 2
|
||||
max_num_seqs: 4
|
||||
max_num_batched_tokens: vllm_default_not_overridden
|
||||
gpu_memory_utilization: 0.50
|
||||
enable_prefix_caching: true
|
||||
|
|
@ -244,12 +382,12 @@ model:
|
|||
- 248044
|
||||
bos_token_id: 248044
|
||||
docker_runtime:
|
||||
created_at: "2026-07-12T04:40:26.013681328Z"
|
||||
created_at: "2026-07-12T08:41:04Z"
|
||||
entrypoint:
|
||||
- vllm
|
||||
- serve
|
||||
container_port: 8000
|
||||
host_port: 8002
|
||||
host_port: 8001
|
||||
network_mode: bridge
|
||||
ipc_mode: private
|
||||
shm_size_bytes: 67108864
|
||||
|
|
@ -259,13 +397,13 @@ model:
|
|||
/home/toki/Data/models: /models
|
||||
/home/toki/Data/vllm/templates: /templates
|
||||
model_load_memory_gib: 34.43
|
||||
model_load_seconds: 611.880026
|
||||
weight_download_seconds: 407.868734
|
||||
model_load_seconds: 196.929359
|
||||
weight_download_seconds: not_observed_cache_hit_restart
|
||||
cuda_graph_memory_estimated_gib: 0.13
|
||||
cuda_graph_pool_memory_actual_gib: 0.07
|
||||
available_kv_cache_memory_gib: 21.33
|
||||
gpu_kv_cache_tokens: 1087640
|
||||
max_concurrency_for_262144_token_requests: 4.15
|
||||
available_kv_cache_memory_gib: 20.79
|
||||
gpu_kv_cache_tokens: 1060912
|
||||
max_concurrency_for_262144_token_requests: 4.05
|
||||
startup_warnings:
|
||||
- vLLM command does not pin --revision; the recorded hf_revision is the downloaded snapshot observed in cache.
|
||||
- Prefix caching in Mamba cache align mode is experimental for this architecture.
|
||||
|
|
@ -292,6 +430,40 @@ model:
|
|||
- 30.62
|
||||
- 29.15
|
||||
note: vLLM logs showed aggregate generation throughput around 50-61 tok/s with two running requests.
|
||||
ornith_lemonade_direct_profile:
|
||||
observed_at: "2026-07-12"
|
||||
provider: ornith-lemonade
|
||||
status: removed_from_pi_models_json_2026_07_12_runtime_still_active_as_onexplayer_iop_ornith_provider
|
||||
family: spark_ornith
|
||||
api: openai-completions
|
||||
base_url: http://192.168.0.59:13305/v1
|
||||
model_id: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M
|
||||
display_name: Ornith 1.0 35B Q5 Direct Lemonade
|
||||
context_window: 262144
|
||||
max_tokens: 32768
|
||||
reasoning: true
|
||||
config_file: /config/.pi/agent/models.json
|
||||
default_profile_changed: true
|
||||
pi_default_provider_after_add: iop
|
||||
pi_default_model_after_add: ornith:35b
|
||||
endpoint_policy: Pi no longer connects directly to the OneXPlayer Lemonade OpenAI-compatible endpoint. This runtime remains reachable through IOP Edge under ornith:35b.
|
||||
api_key_value_tracked: false
|
||||
runtime_alignment:
|
||||
lemonade_runtime_model: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M
|
||||
lemonade_response_model: ornith-1.0-35b-Q5_K_M.gguf
|
||||
quantization: Q5_K_M
|
||||
ctx_size: 524288
|
||||
kv_policy: unified_kv_with_np_omitted
|
||||
observed_slot_n_ctx: 262144
|
||||
iop_capacity_guard: 3
|
||||
long_context_capacity: 2
|
||||
smoke:
|
||||
endpoint_direct_chat_completions: passed
|
||||
request_model: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M
|
||||
response_model: ornith-1.0-35b-Q5_K_M.gguf
|
||||
finish_reason: stop
|
||||
content: "4"
|
||||
completion_tokens: 2
|
||||
separation_note: This host Pi default profile is the dev Qwen3.6 provider-pool profile. Keep dev-corp Gemma4 under agent-test/dev-corp/inventory.yaml and do not copy Gemma4 parser/template/runtime settings into this Qwen profile.
|
||||
content_completion_policy:
|
||||
enforce_min_max_tokens: true
|
||||
|
|
@ -315,7 +487,7 @@ model:
|
|||
prompt_policy: long_reasoning_allowed
|
||||
exact_output_match: false
|
||||
latest_provider_snapshot:
|
||||
observed_at: "2026-07-08"
|
||||
observed_at: "2026-07-12"
|
||||
total_capacity: 9
|
||||
all_nodes_connected: true
|
||||
providers:
|
||||
|
|
@ -325,7 +497,7 @@ model:
|
|||
in_flight: 0
|
||||
queued: 0
|
||||
health: healthy
|
||||
served_model: nvidia/Qwen3.6-35B-A3B-NVFP4
|
||||
served_model: ornith:35b
|
||||
mac-mlx-vllm:
|
||||
capacity: 2
|
||||
priority: 2
|
||||
|
|
@ -339,7 +511,7 @@ model:
|
|||
in_flight: 0
|
||||
queued: 0
|
||||
health: healthy
|
||||
served_model: Qwen3.6-35B-A3B-MTP-GGUF
|
||||
served_model: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M
|
||||
openai_tool_call_smoke:
|
||||
observed_at: "2026-07-08"
|
||||
request_shape: tools_plus_tool_choice_auto
|
||||
|
|
@ -483,36 +655,38 @@ nodes:
|
|||
id: gx10-vllm
|
||||
type: vllm
|
||||
endpoint: http://192.168.0.91:8001/v1
|
||||
served_model: nvidia/Qwen3.6-35B-A3B-NVFP4
|
||||
served_model: ornith:35b
|
||||
capacity: 4
|
||||
priority: 0
|
||||
# Long-context admission policy (maps to edge.yaml nodes[].providers[]).
|
||||
# max_model_len is 262144; conservative long_context_capacity=1 until runtime
|
||||
# evidence proves a larger concurrent long-context KV budget.
|
||||
total_context_tokens: 262144
|
||||
long_context_capacity: 1
|
||||
# Ornith FP8 runtime reports 1,060,912 GPU KV tokens and 4.05x maximum
|
||||
# concurrency for full 262144-token requests, so four long slots are allowed.
|
||||
total_context_tokens: 1048576
|
||||
long_context_capacity: 4
|
||||
runtime:
|
||||
container_name: iop-vllm-qwen36-prev-20260704063128
|
||||
container_name: iop-vllm-ornith35b-fp8
|
||||
max_model_len: 262144
|
||||
max_num_seqs: 4
|
||||
gpu_memory_utilization: 0.30
|
||||
gpu_memory_utilization: 0.50
|
||||
default_max_tokens: 32768
|
||||
default_thinking_token_budget: 1024
|
||||
enable_auto_tool_choice: true
|
||||
tool_call_parser: qwen3_xml
|
||||
reasoning_parser: qwen3
|
||||
default_chat_template_kwargs:
|
||||
enable_thinking: true
|
||||
trust_remote_code: true
|
||||
language_model_only: true
|
||||
quantization: fp8
|
||||
dtype: bfloat16
|
||||
direct_providers:
|
||||
- id: ornith-direct
|
||||
family: spark_ornith
|
||||
type: vllm
|
||||
endpoint: http://192.168.0.91:8002/v1
|
||||
endpoint: http://192.168.0.91:8001/v1
|
||||
served_model: ornith:35b
|
||||
upstream_model: deepreinforce-ai/Ornith-1.0-35B-FP8
|
||||
upstream_url: https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B-FP8
|
||||
upstream_revision: 1ab57ce0b44950e498a88756f40ad1ed4d0f30ca
|
||||
status: active_direct_pi_provider_not_iop_routed
|
||||
status: direct_pi_provider_removed_runtime_shares_current_iop_ornith_gx10_provider
|
||||
pi_model_parameters:
|
||||
context_window: 262144
|
||||
max_tokens: 32768
|
||||
|
|
@ -523,14 +697,14 @@ nodes:
|
|||
docker_image_id: sha256:a720df3e84a89d7db47a3b7a0511cb5b312e203fc4956f7493df248299267a6f
|
||||
vllm_version: 0.23.1rc1.dev223+ga346d589f
|
||||
vllm_build_commit: a346d589f5932d4234bf5bf8718f10e26d187021
|
||||
port_mapping: 0.0.0.0:8002->8000
|
||||
port_mapping: 0.0.0.0:8001->8000
|
||||
hf_home: /models/.cache/huggingface
|
||||
hf_cache_snapshot_path: /models/.cache/huggingface/hub/models--deepreinforce-ai--Ornith-1.0-35B-FP8/snapshots/1ab57ce0b44950e498a88756f40ad1ed4d0f30ca
|
||||
quantization: fp8
|
||||
vllm_quantization_backend: compressed-tensors
|
||||
dtype: bfloat16
|
||||
max_model_len: 262144
|
||||
max_num_seqs: 2
|
||||
max_num_seqs: 4
|
||||
max_num_batched_tokens: vllm_default_not_overridden
|
||||
gpu_memory_utilization: 0.50
|
||||
enable_prefix_caching: true
|
||||
|
|
@ -566,9 +740,9 @@ nodes:
|
|||
/home/toki/Data/models: /models
|
||||
/home/toki/Data/vllm/templates: /templates
|
||||
model_load_memory_gib: 34.43
|
||||
available_kv_cache_memory_gib: 21.33
|
||||
gpu_kv_cache_tokens: 1087640
|
||||
max_concurrency_for_262144_token_requests: 4.15
|
||||
available_kv_cache_memory_gib: 20.79
|
||||
gpu_kv_cache_tokens: 1060912
|
||||
max_concurrency_for_262144_token_requests: 4.05
|
||||
average_completion_tok_s: 29.8
|
||||
replacement_note: DiffusionGemma direct vLLM on this endpoint was stopped; Ornith is served from the official FP8 Hugging Face release, not the 4-bit GGUF build.
|
||||
- id: onexplayer-lemonade-node
|
||||
|
|
@ -582,24 +756,98 @@ nodes:
|
|||
id: onexplayer-lemonade
|
||||
type: lemonade
|
||||
endpoint: http://192.168.0.59:13305/v1
|
||||
served_model: Qwen3.6-35B-A3B-MTP-GGUF
|
||||
served_model: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M
|
||||
served_model_response: ornith-1.0-35b-Q5_K_M.gguf
|
||||
previous_served_model: Qwen3.6-35B-A3B-MTP-GGUF
|
||||
status: active_ornith_q5_current_iop_ornith_provider_pool_member
|
||||
qwen_reenable_profile:
|
||||
status: not_loaded_ornith_q5_active
|
||||
model_name: Qwen3.6-35B-A3B-MTP-GGUF
|
||||
checkpoint: unsloth/Qwen3.6-35B-A3B-MTP-GGUF:Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf
|
||||
prior_saved_recipe_options:
|
||||
ctx_size: 524288
|
||||
llamacpp_args: "--spec-type none -np 3 -cb -fa on -b 4096 -ub 1024"
|
||||
partition_note: The prior -np 3 profile fixed slot windows at about 174848 tokens for ctx_size 524288.
|
||||
recommended_recipe_options:
|
||||
ctx_size: 524288
|
||||
llamacpp_args: "--spec-type none -cb -fa on -b 4096 -ub 1024 --kv-unified"
|
||||
removed_args: "-np 3"
|
||||
expected_slot_n_ctx: 262144
|
||||
expected_auto_slots: 4
|
||||
iop_capacity_guard: 3
|
||||
long_context_capacity: 2
|
||||
validation_required_after_load: true
|
||||
validation_basis: Same Lemonade llama.cpp backend and current Ornith Q5 runtime confirmed --kv-unified with omitted -np prevents fixed ctx_size/parallel partitioning.
|
||||
capacity: 3
|
||||
priority: 1
|
||||
# Long-context admission policy (maps to edge.yaml nodes[].providers[]).
|
||||
# SDD D04 fixes OneXPlayer dev baseline at ctx_size=524288, -np 3,
|
||||
# long_context_capacity=2. Do NOT raise ctx_size above 524288.
|
||||
# Ornith Q5 direct runtime keeps ctx_size=524288 and removes fixed -np
|
||||
# partitioning by using llama.cpp unified KV. Do NOT raise ctx_size above 524288.
|
||||
total_context_tokens: 524288
|
||||
long_context_capacity: 2
|
||||
load:
|
||||
endpoint: http://192.168.0.59:13305/v1/load
|
||||
model_name: Qwen3.6-35B-A3B-MTP-GGUF
|
||||
model_name: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M
|
||||
checkpoint: LordNeel/Ornith-1.0-35B-GGUF-llamacpp-tp1:Q5_K_M
|
||||
hf_url: https://huggingface.co/LordNeel/Ornith-1.0-35B-GGUF-llamacpp-tp1
|
||||
hf_revision: c50d5d4407f70e43208dee836c66bb8a05c1be91
|
||||
hf_ref: main
|
||||
gguf_file: ornith-1.0-35b-Q5_K_M.gguf
|
||||
gguf_file_size_bytes: 24729130848
|
||||
cache_snapshot_path: C:/Users/r0bin/.cache/huggingface/hub/models--LordNeel--Ornith-1.0-35B-GGUF-llamacpp-tp1/snapshots/c50d5d4407f70e43208dee836c66bb8a05c1be91
|
||||
backend: vulkan
|
||||
ctx_size: 524288
|
||||
llamacpp_args: "--spec-type none -np 3 -cb -fa on -b 4096 -ub 1024"
|
||||
observed_slot_n_ctx: 174848
|
||||
llamacpp_args: "--spec-type none -cb -fa on -b 4096 -ub 1024 --kv-unified"
|
||||
observed_process_args: "--ctx-size 524288 --port 8001 --jinja --context-shift --keep 16 --reasoning-format auto --no-webui --no-mmap -ngl 99 --kv-unified --spec-type none -b 4096 -cb -fa on -ub 1024"
|
||||
observed_total_slots: 4
|
||||
observed_total_ctx_size: 524288
|
||||
observed_slot_n_ctx: 262144
|
||||
context_per_slot_note: -np is intentionally omitted; llama.cpp auto slots with --kv-unified avoid the prior fixed 524288/3 partition and report 262144-token slot windows backed by shared KV.
|
||||
residency:
|
||||
observed_at: "2026-07-12"
|
||||
process_resident_policy: keep llama-server loaded until explicit lemonade unload or service/process restart
|
||||
observed_process_running: true
|
||||
observed_is_sleeping: false
|
||||
idle_sleep_seconds: not_set_llama_cpp_default_minus_1_disabled
|
||||
mmap_policy: disabled_via_no_mmap
|
||||
memory_lock_policy: not_enabled_no_mlock_argument
|
||||
note: This keeps the model warm in the active llama-server process, but it is not an OS-level hard memory pin; Windows can still page memory under pressure because --mlock is not set.
|
||||
save_options: true
|
||||
mtp_runtime_policy: disabled
|
||||
mtp_runtime_policy: disabled_via_spec_type_none
|
||||
default_max_tokens: 32768
|
||||
default_thinking_token_budget: 1024
|
||||
tool_choice_auto_verified: true
|
||||
smoke:
|
||||
observed_at: "2026-07-12"
|
||||
qwen_unloaded: true
|
||||
load_status: passed
|
||||
english_final_answer:
|
||||
max_tokens: 512
|
||||
result: passed
|
||||
content: 2+2 equals 4.
|
||||
completion_tokens: 414
|
||||
generation_tok_s: 31.94
|
||||
korean_final_answer:
|
||||
max_tokens: 1024
|
||||
result: passed
|
||||
content: 2 더하기 2는 4입니다.
|
||||
completion_tokens: 384
|
||||
tool_choice_auto:
|
||||
result: passed
|
||||
finish_reason: tool_calls
|
||||
tool_calls: 1
|
||||
function_name: get_weather
|
||||
arguments: "{\"city\":\"Seoul\"}"
|
||||
unified_kv_concurrency:
|
||||
observed_at: "2026-07-12"
|
||||
result: passed
|
||||
concurrent_requests: 3
|
||||
finish_reason_stop: 3
|
||||
final_content: "4"
|
||||
total_elapsed_sec: 6.55
|
||||
completion_tokens:
|
||||
- 149
|
||||
- 88
|
||||
- 85
|
||||
low_output_budget_note: max_tokens 128 and one Korean 512-token probe ended in reasoning_content with empty final content; keep Pi/Edge maxTokens at 32768 or use at least 1024 for short smoke prompts.
|
||||
windows_process_start: Win32_Process.Create
|
||||
|
|
|
|||
|
|
@ -60,24 +60,24 @@ dev-runtime의 실제 3-node 연결을 점검할 때는 원격 runner `ssh toki@
|
|||
- GX10 vLLM node: `gx10-vllm-node` / `gx10-vllm`
|
||||
- SSH/user: `ssh toki@192.168.0.91`
|
||||
- provider endpoint: `http://192.168.0.91:8001/v1`
|
||||
- served model: `nvidia/Qwen3.6-35B-A3B-NVFP4`
|
||||
- served model: `ornith:35b`
|
||||
- capacity baseline: `4`
|
||||
- priority baseline: `0`
|
||||
- runtime baseline: `vllm`, `--max-model-len 262144`, `--max-num-seqs 4`, `--gpu-memory-utilization 0.30`, `--reasoning-parser qwen3`, `--default-chat-template-kwargs {"enable_thinking":true}`, `--enable-auto-tool-choice --tool-call-parser qwen3_xml`
|
||||
- long-context admission baseline: `total_context_tokens=262144`, `long_context_capacity=1` (conservative; `max-model-len 262144` until runtime evidence proves a larger concurrent long-context budget)
|
||||
- runtime baseline: `vllm`, model `deepreinforce-ai/Ornith-1.0-35B-FP8`, `--served-model-name ornith:35b`, `--dtype bfloat16`, `--max-model-len 262144`, `--max-num-seqs 4`, `--gpu-memory-utilization 0.50`, `--enable-prefix-caching`, `--enable-auto-tool-choice --tool-call-parser qwen3_xml`, `--reasoning-parser qwen3`, `--trust-remote-code`, `--language-model-only`
|
||||
- long-context admission baseline: `total_context_tokens=1048576`, `long_context_capacity=4` (vLLM log: GPU KV cache `1060912` tokens, max concurrency `4.05x` for 262144-token requests)
|
||||
- workspace: `/home/toki/iop-gx10-vllm`
|
||||
- OneXPlayer Lemonade node: `onexplayer-lemonade-node` / `onexplayer-lemonade`
|
||||
- SSH/user: `ssh r0bin@192.168.0.59`
|
||||
- 접속 기준: 현재 작업 호스트에서 직접 SSH
|
||||
- provider endpoint: `http://192.168.0.59:13305/v1`
|
||||
- served model: `Qwen3.6-35B-A3B-MTP-GGUF`
|
||||
- served model: `Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M`
|
||||
- capacity baseline: `3`
|
||||
- priority baseline: `1`
|
||||
- load baseline: backend `vulkan`, ctx size `524288`, `llamacpp_args="--spec-type none -np 3 -cb -fa on -b 4096 -ub 1024"`, `save_options=true`
|
||||
- long-context admission baseline: `total_context_tokens=524288`, `long_context_capacity=2` (SDD D04 fixed; `ctx_size=524288`, `-np 3` 유지, `ctx_size`를 `524288` 초과로 키우지 않는다)
|
||||
- load baseline: checkpoint `LordNeel/Ornith-1.0-35B-GGUF-llamacpp-tp1:Q5_K_M`, backend `vulkan`, ctx size `524288`, `llamacpp_args="--spec-type none -cb -fa on -b 4096 -ub 1024 --kv-unified"`, `save_options=true`
|
||||
- long-context admission baseline: `total_context_tokens=524288`, `long_context_capacity=2` (`-np` 고정 분할을 제거하고 `--kv-unified`를 사용한다. `/slots`는 auto slots 4개와 slot `n_ctx=262144`를 보고한다)
|
||||
- workspace: `C:/Users/r0bin/iop-field`
|
||||
|
||||
OneXPlayer Lemonade는 `Qwen3.6-35B-A3B-MTP-GGUF` artifact를 사용하되 runtime MTP speculative decoding은 끈 상태를 dev 기준으로 삼는다. Node 검증 전 `/v1/load`의 `recipe_options`가 `model_name=Qwen3.6-35B-A3B-MTP-GGUF`, Vulkan, `ctx_size=524288`, `--spec-type none -np 3 -cb -fa on -b 4096 -ub 1024`를 포함하는지 확인한다. backend `/slots`에서는 총 ctx `524288`이 slot 3개로 나뉘어 2026-07-04 기준 slot별 `n_ctx=174848`로 보인다.
|
||||
OneXPlayer Lemonade는 현재 Ornith Q5 GGUF를 사용하고 runtime speculative decoding은 `--spec-type none`으로 끈다. Qwen을 다시 올릴 때도 fixed `-np 3` 분할은 쓰지 말고 현재 검증된 `--kv-unified` 방식으로 resource를 공유한다.
|
||||
|
||||
GX10은 Linux/ARM64 bootstrap, OneXPlayer는 Windows native PowerShell bootstrap을 기본으로 한다. OneXPlayer는 현재 작업 호스트에서 직접 접속해 세팅하며, 원격 runner나 Edge host에서 `node-onexplayer-lemonade.yaml`로 proxy 실행하지 않는다.
|
||||
|
||||
|
|
|
|||
|
|
@ -4584,7 +4584,10 @@ func TestChatCompletionsResponseModeRouting(t *testing.T) {
|
|||
// chatSidebandServer returns an openai.Server backed by the given tunnel
|
||||
// frames for explicit passthrough+sideband fixtures.
|
||||
func chatSidebandServer(frames chan *iop.ProviderTunnelFrame) (*Server, *fakeRunService) {
|
||||
fake := &fakeRunService{tunnelFrames: frames}
|
||||
fake := &fakeRunService{
|
||||
poolDispatchPath: string(edgeservice.ProviderPoolPathTunnel),
|
||||
tunnelFrames: frames,
|
||||
}
|
||||
srv := NewServer(config.EdgeOpenAIConf{}, fake, nil)
|
||||
srv.SetModelCatalog([]config.ModelCatalogEntry{
|
||||
{ID: "pool-model", Providers: map[string]string{"prov-1": "served-model"}},
|
||||
|
|
@ -4829,7 +4832,10 @@ func TestChatCompletionsPassthroughDoesNotExposeSideband(t *testing.T) {
|
|||
// provider auth forwarding config, mirroring chatSidebandServer's catalog so
|
||||
// pool-model routes over the provider tunnel path.
|
||||
func chatProviderAuthServer(frames chan *iop.ProviderTunnelFrame, auth config.EdgeOpenAIProviderAuthConf) (*Server, *fakeRunService) {
|
||||
fake := &fakeRunService{tunnelFrames: frames}
|
||||
fake := &fakeRunService{
|
||||
poolDispatchPath: string(edgeservice.ProviderPoolPathTunnel),
|
||||
tunnelFrames: frames,
|
||||
}
|
||||
srv := NewServer(config.EdgeOpenAIConf{ProviderAuth: auth}, fake, nil)
|
||||
srv.SetModelCatalog([]config.ModelCatalogEntry{
|
||||
{ID: "pool-model", Providers: map[string]string{"prov-1": "served-model"}},
|
||||
|
|
@ -5111,6 +5117,7 @@ func TestChatCompletionsPassthroughDoesNotLabelTransformed(t *testing.T) {
|
|||
// tunnel to the given provider fixture, emulating the Node relay.
|
||||
func chatPassthroughServer(providerURL string) (*Server, *fakeRunService) {
|
||||
fake := &fakeRunService{
|
||||
poolDispatchPath: string(edgeservice.ProviderPoolPathTunnel),
|
||||
tunnelProviderURL: providerURL,
|
||||
tunnelServedTarget: "served-model",
|
||||
}
|
||||
|
|
@ -6095,7 +6102,11 @@ func TestChatCompletionsLegacyRouteSetsProviderPoolFalse(t *testing.T) {
|
|||
// "pool-model" through the OpenAI-compatible provider tunnel, wired to the given
|
||||
// provider frames and served target.
|
||||
func responsesProviderTunnelServer(frames chan *iop.ProviderTunnelFrame, servedTarget string) (*Server, *fakeRunService) {
|
||||
fake := &fakeRunService{tunnelFrames: frames, tunnelServedTarget: servedTarget}
|
||||
fake := &fakeRunService{
|
||||
poolDispatchPath: string(edgeservice.ProviderPoolPathTunnel),
|
||||
tunnelFrames: frames,
|
||||
tunnelServedTarget: servedTarget,
|
||||
}
|
||||
srv := NewServer(config.EdgeOpenAIConf{}, fake, nil)
|
||||
srv.SetModelCatalog([]config.ModelCatalogEntry{
|
||||
{ID: "pool-model", Providers: map[string]string{"prov-1": "served-model"}},
|
||||
|
|
@ -7742,3 +7753,183 @@ func TestChatCompletionsProviderPoolToolValidationRetryRejectsTunnelResult(t *te
|
|||
t.Errorf("error message: got %q", errMap["message"])
|
||||
}
|
||||
}
|
||||
|
||||
// --- MIXED_GROUP-2: Mixed/Ollama-Only Provider Pool Fixtures ---
|
||||
|
||||
// TestChatCompletionsMixedProviderPoolTunnelSelection verifies S05: a mixed
|
||||
// model group containing both OpenAI-compatible and Ollama providers dispatches
|
||||
// the selected tunnel-capable provider over the provider tunnel (passthrough)
|
||||
// path. Only one tunnel request is sent, no normalized SubmitRun is called.
|
||||
func TestChatCompletionsMixedProviderPoolTunnelSelection(t *testing.T) {
|
||||
// Tunnel frames for the tunnel path.
|
||||
tunnelFrames := make(chan *iop.ProviderTunnelFrame, 3)
|
||||
tunnelFrames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_RESPONSE_START, StatusCode: 200}
|
||||
tunnelFrames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(`{"id":"cmpl-mixed","choices":[{"index":0,"message":{"role":"assistant","content":"tunnel"}}],"finish_reason":"stop"}`)}
|
||||
tunnelFrames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_END, End: true}
|
||||
close(tunnelFrames)
|
||||
|
||||
// Fake service selects tunnel path (simulating vLLM provider selection).
|
||||
fake := &fakeRunService{
|
||||
poolDispatchPath: string(edgeservice.ProviderPoolPathTunnel),
|
||||
tunnelFrames: tunnelFrames,
|
||||
}
|
||||
|
||||
// Mixed catalog: OpenAI-compatible (vLLM) + Ollama in same group.
|
||||
catalog := []config.ModelCatalogEntry{
|
||||
{
|
||||
ID: "mixed-model",
|
||||
Providers: map[string]string{
|
||||
"prov-vllm": "served-vllm",
|
||||
"prov-ollama": "served-ollama",
|
||||
},
|
||||
},
|
||||
}
|
||||
srv := NewServer(config.EdgeOpenAIConf{}, fake, nil)
|
||||
srv.SetModelCatalog(catalog)
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{
|
||||
"model":"mixed-model",
|
||||
"messages":[{"role":"user","content":"hello"}]
|
||||
}`))
|
||||
w := httptest.NewRecorder()
|
||||
srv.handleChatCompletions(w, req)
|
||||
|
||||
if w.Code != http.StatusOK {
|
||||
t.Fatalf("status: got %d body=%s", w.Code, w.Body.String())
|
||||
}
|
||||
|
||||
// Tunnel path: exactly 1 tunnel dispatch, 0 normalized SubmitRun.
|
||||
tunnelReqs := fake.tunnelReqsSnapshot()
|
||||
if len(tunnelReqs) != 1 {
|
||||
t.Fatalf("expected 1 tunnel dispatch, got %d", len(tunnelReqs))
|
||||
}
|
||||
if !tunnelReqs[0].ProviderPool {
|
||||
t.Error("tunnel request must have ProviderPool=true")
|
||||
}
|
||||
if tunnelReqs[0].ModelGroupKey != "mixed-model" {
|
||||
t.Errorf("tunnel ModelGroupKey: got %q, want mixed-model", tunnelReqs[0].ModelGroupKey)
|
||||
}
|
||||
|
||||
runReqs := fake.reqsSnapshot()
|
||||
if len(runReqs) != 0 {
|
||||
t.Errorf("expected 0 normalized SubmitRun calls, got %d", len(runReqs))
|
||||
}
|
||||
}
|
||||
|
||||
// TestChatCompletionsMixedProviderPoolOllamaSelection verifies S05: when the
|
||||
// mixed model group selects an Ollama provider, the request is dispatched over
|
||||
// the normalized path (SubmitRun), and no tunnel dispatch is issued.
|
||||
func TestChatCompletionsMixedProviderPoolOllamaSelection(t *testing.T) {
|
||||
// Normalized path events.
|
||||
events := bufferedRunEvents(
|
||||
&iop.RunEvent{Type: "delta", Delta: "ollama"},
|
||||
&iop.RunEvent{Type: "complete"},
|
||||
)
|
||||
|
||||
// Fake service selects normalized path (simulating Ollama provider selection).
|
||||
fake := &fakeRunService{
|
||||
poolDispatchPath: string(edgeservice.ProviderPoolPathNormalized),
|
||||
events: events,
|
||||
}
|
||||
|
||||
// Same mixed catalog: vLLM + Ollama.
|
||||
catalog := []config.ModelCatalogEntry{
|
||||
{
|
||||
ID: "mixed-model",
|
||||
Providers: map[string]string{
|
||||
"prov-vllm": "served-vllm",
|
||||
"prov-ollama": "served-ollama",
|
||||
},
|
||||
},
|
||||
}
|
||||
srv := NewServer(config.EdgeOpenAIConf{}, fake, nil)
|
||||
srv.SetModelCatalog(catalog)
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{
|
||||
"model":"mixed-model",
|
||||
"messages":[{"role":"user","content":"hello"}]
|
||||
}`))
|
||||
w := httptest.NewRecorder()
|
||||
srv.handleChatCompletions(w, req)
|
||||
|
||||
if w.Code != http.StatusOK {
|
||||
t.Fatalf("status: got %d body=%s", w.Code, w.Body.String())
|
||||
}
|
||||
|
||||
// Normalized path: 0 tunnel dispatches, 1 normalized SubmitRun.
|
||||
tunnelReqs := fake.tunnelReqsSnapshot()
|
||||
if len(tunnelReqs) != 0 {
|
||||
t.Fatalf("expected 0 tunnel dispatches for Ollama selection, got %d", len(tunnelReqs))
|
||||
}
|
||||
|
||||
runReqs := fake.reqsSnapshot()
|
||||
if len(runReqs) != 1 {
|
||||
t.Fatalf("expected 1 normalized SubmitRun, got %d", len(runReqs))
|
||||
}
|
||||
// Adapter/Target are rewritten by the service's admitted candidate; the
|
||||
// fake here records the pre-admission request so the handler-level path
|
||||
// selection (tunnel vs normalized) is the assertion surface.
|
||||
if runReqs[0].ModelGroupKey != "mixed-model" {
|
||||
t.Errorf("normalized ModelGroupKey: got %q, want mixed-model", runReqs[0].ModelGroupKey)
|
||||
}
|
||||
if !runReqs[0].ProviderPool {
|
||||
t.Error("normalized run must have ProviderPool=true")
|
||||
}
|
||||
}
|
||||
|
||||
// TestChatCompletionsOllamaOnlyProviderPoolUsesNormalizedPath verifies S05:
|
||||
// a model group containing only Ollama providers always dispatches over the
|
||||
// normalized path. No tunnel request is ever issued.
|
||||
func TestChatCompletionsOllamaOnlyProviderPoolUsesNormalizedPath(t *testing.T) {
|
||||
events := bufferedRunEvents(
|
||||
&iop.RunEvent{Type: "delta", Delta: "ollama-only"},
|
||||
&iop.RunEvent{Type: "complete"},
|
||||
)
|
||||
|
||||
// Fake service selects normalized path.
|
||||
fake := &fakeRunService{
|
||||
poolDispatchPath: string(edgeservice.ProviderPoolPathNormalized),
|
||||
events: events,
|
||||
}
|
||||
|
||||
// Ollama-only catalog.
|
||||
catalog := []config.ModelCatalogEntry{
|
||||
{
|
||||
ID: "ollama-only-model",
|
||||
Providers: map[string]string{
|
||||
"prov-ollama": "qwen3.6:35b",
|
||||
},
|
||||
},
|
||||
}
|
||||
srv := NewServer(config.EdgeOpenAIConf{}, fake, nil)
|
||||
srv.SetModelCatalog(catalog)
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{
|
||||
"model":"ollama-only-model",
|
||||
"messages":[{"role":"user","content":"hello"}]
|
||||
}`))
|
||||
w := httptest.NewRecorder()
|
||||
srv.handleChatCompletions(w, req)
|
||||
|
||||
if w.Code != http.StatusOK {
|
||||
t.Fatalf("status: got %d body=%s", w.Code, w.Body.String())
|
||||
}
|
||||
|
||||
// Must not dispatch tunnel.
|
||||
tunnelReqs := fake.tunnelReqsSnapshot()
|
||||
if len(tunnelReqs) != 0 {
|
||||
t.Fatalf("ollama-only must not dispatch tunnel, got %d", len(tunnelReqs))
|
||||
}
|
||||
|
||||
// Must dispatch exactly one normalized run.
|
||||
runReqs := fake.reqsSnapshot()
|
||||
if len(runReqs) != 1 {
|
||||
t.Fatalf("expected 1 normalized SubmitRun, got %d", len(runReqs))
|
||||
}
|
||||
if runReqs[0].ModelGroupKey != "ollama-only-model" {
|
||||
t.Errorf("ollama-only ModelGroupKey: got %q, want ollama-only-model", runReqs[0].ModelGroupKey)
|
||||
}
|
||||
if !runReqs[0].ProviderPool {
|
||||
t.Error("ollama-only normalized run must have ProviderPool=true")
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -2232,3 +2232,134 @@ func TestResolveProviderPoolCandidatesClassifiesExecutionPath(t *testing.T) {
|
|||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestResolveProviderPoolCandidatesKeepsMixedProviderTypes verifies S05:
|
||||
// when a single catalog model group contains both OpenAI-compatible providers
|
||||
// (vLLM, lemonade, openai_compat) and Ollama providers, all valid providers
|
||||
// remain candidates and each candidate carries the correct execution path.
|
||||
// This is the missing mixed-group fixture: previously only single-type
|
||||
// provider groups were asserted, so a vLLM + Ollama group coexisting in one
|
||||
// catalog entry had no direct evidence.
|
||||
func TestResolveProviderPoolCandidatesKeepsMixedProviderTypes(t *testing.T) {
|
||||
catalog := []config.ModelCatalogEntry{
|
||||
{
|
||||
ID: "mixed-model",
|
||||
Providers: map[string]string{
|
||||
"prov-vllm": "served-vllm",
|
||||
"prov-vllm-mlx": "served-vllm-mlx",
|
||||
"prov-lemonade": "served-lemonade",
|
||||
"prov-ollama": "served-ollama",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
store := edgenode.NewNodeStore()
|
||||
store.Add(&edgenode.NodeRecord{
|
||||
ID: "node-mixed",
|
||||
Runtime: config.RuntimeConf{Concurrency: 4},
|
||||
Adapters: config.AdaptersConf{
|
||||
VllmInstances: []config.VllmInstanceConf{
|
||||
{Name: "vllm-gpu", Enabled: true, Endpoint: "http://127.0.0.1:8000/v1"},
|
||||
{Name: "vllm-gpu-mlx", Enabled: true, Endpoint: "http://127.0.0.1:8001/v1"},
|
||||
},
|
||||
OllamaInstances: []config.OllamaInstanceConf{
|
||||
{Name: "ollama", Enabled: true, BaseURL: "http://127.0.0.1:11434"},
|
||||
},
|
||||
OpenAICompatInstances: []config.OpenAICompatInstanceConf{
|
||||
{Name: "oaicompat", Enabled: true, Endpoint: "http://127.0.0.1:9000/v1"},
|
||||
},
|
||||
},
|
||||
Providers: []config.NodeProviderConf{
|
||||
{
|
||||
ID: "prov-vllm",
|
||||
Type: "vllm",
|
||||
Adapter: "vllm-gpu",
|
||||
Models: []string{"served-vllm"},
|
||||
Health: "available",
|
||||
Capacity: 2,
|
||||
},
|
||||
{
|
||||
ID: "prov-vllm-mlx",
|
||||
Type: "vllm-mlx",
|
||||
Adapter: "vllm-gpu-mlx",
|
||||
Models: []string{"served-vllm-mlx"},
|
||||
Health: "available",
|
||||
Capacity: 2,
|
||||
},
|
||||
{
|
||||
ID: "prov-lemonade",
|
||||
Type: "lemonade",
|
||||
Adapter: "oaicompat",
|
||||
Models: []string{"served-lemonade"},
|
||||
Health: "available",
|
||||
Capacity: 2,
|
||||
},
|
||||
{
|
||||
ID: "prov-ollama",
|
||||
Type: "ollama",
|
||||
Adapter: "ollama",
|
||||
Models: []string{"served-ollama"},
|
||||
Health: "available",
|
||||
Capacity: 2,
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
reg := edgenode.NewRegistry()
|
||||
for _, rec := range store.All() {
|
||||
reg.Register(&edgenode.NodeEntry{NodeID: rec.ID, LifecycleState: edgenode.LifecycleConnected})
|
||||
}
|
||||
|
||||
svc := New(reg, nil)
|
||||
svc.SetNodeStore(store)
|
||||
svc.SetModelCatalog(catalog)
|
||||
|
||||
req := SubmitRunRequest{ModelGroupKey: "mixed-model", ProviderPool: true}
|
||||
candidates, _, err := svc.resolveProviderPoolCandidates(req, store, catalog)
|
||||
if err != nil {
|
||||
t.Fatalf("resolveProviderPoolCandidates: %v", err)
|
||||
}
|
||||
|
||||
// All 4 providers must be candidates.
|
||||
if len(candidates) != 4 {
|
||||
ids := make([]string, len(candidates))
|
||||
for i, c := range candidates {
|
||||
ids[i] = c.providerID
|
||||
}
|
||||
t.Fatalf("expected 4 candidates (prov-vllm, prov-vllm-mlx, prov-lemonade, prov-ollama), got %d: %v", len(candidates), ids)
|
||||
}
|
||||
|
||||
byID := map[string]*candidateNode{}
|
||||
for i := range candidates {
|
||||
byID[candidates[i].providerID] = &candidates[i]
|
||||
}
|
||||
|
||||
// OpenAI-compatible providers (vLLM, vLLM-MLX, lemonade) must be tunnel path.
|
||||
for _, provID := range []string{"prov-vllm", "prov-vllm-mlx", "prov-lemonade"} {
|
||||
c, ok := byID[provID]
|
||||
if !ok {
|
||||
t.Errorf("%q not in candidates", provID)
|
||||
continue
|
||||
}
|
||||
if c.executionPath != providerExecutionPathTunnel {
|
||||
t.Errorf("%q: expected tunnel path, got %q", provID, c.executionPath)
|
||||
}
|
||||
// Served target must match the catalog entry.
|
||||
expectedServed := catalog[0].Providers[provID]
|
||||
if c.servedTarget != expectedServed {
|
||||
t.Errorf("%q servedTarget: got %q, want %q", provID, c.servedTarget, expectedServed)
|
||||
}
|
||||
}
|
||||
|
||||
// Ollama provider must be normalized path.
|
||||
ollama, ok := byID["prov-ollama"]
|
||||
if !ok {
|
||||
t.Fatal("prov-ollama not in candidates")
|
||||
}
|
||||
if ollama.executionPath != providerExecutionPathNormalized {
|
||||
t.Errorf("prov-ollama: expected normalized path, got %q", ollama.executionPath)
|
||||
}
|
||||
if ollama.servedTarget != "served-ollama" {
|
||||
t.Errorf("prov-ollama servedTarget: got %q, want %q", ollama.servedTarget, "served-ollama")
|
||||
}
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in a new issue