toki/iop
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

refactor(inventory): 에이전트 host 프로필을 분리한다

Browse source 
환경별 노드와 provider runtime의 원본을 유지하면서 에이전트별 host 설정과 Pi 호환성 관측을 별도 인벤토리로 관리한다.
This commit is contained in:
toki 2026-08-02 11:06:54 +09:00
parent 3fb4ac2cdd
commit 32c0754f91
6 changed files with 486 additions and 427 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
agent-test/README.md
View file

@ -36,7 +36,7 @@
## 확장 원칙
- 환경 공통값은 `rules.md`에 둔다.
- 공통 인벤토리 routing과 환경 선택/secret 정책은 `agent-test/inventory.yaml`에 두고, 실제 노드, provider, bootstrap, model endpoint, workspace, SSH user/IP 같은 환경별 상세 인벤토리는 `agent-test/inventory-<env>.yaml`에 둔다. 사람용 설명과 도메인별 판정 기준은 해당 환경의 domain profile에 둔다.
- 공통 인벤토리 routing과 환경 선택/secret 정책은 `agent-test/inventory.yaml`에 두고, Claude/Codex/Pi/agy 같은 에이전트별 host profile은 `agent-test/inventory-agent.yaml`에 둔다. 실제 노드, provider, bootstrap, model endpoint, workspace, SSH user/IP 같은 환경별 상세 인벤토리는 `agent-test/inventory-<env>.yaml`에 둔다. 에이전트 인벤토리는 가능한 경우 `node_ref`로 환경별 노드를 참조하며, provider runtime의 source of truth는 환경별 인벤토리에 유지한다. Pi처럼 에이전트 동작 확인에 필요한 provider 호환성 관측값은 에이전트 인벤토리에 둘 수 있다. 사람용 설명과 도메인별 판정 기준은 해당 환경의 domain profile에 둔다.
- 상위 `rules.md`에는 상세값을 복제하지 말고 `노드/Provider 인벤토리 위치`와 `라우팅`으로 연결한다.
- 실제 노드, provider, model, endpoint 상세는 환경별 inventory 전체를 읽지 말고 먼저 `go build -o /tmp/iop-inventory-query ./scripts/inventory-query` 로 binary를 빌드한 뒤, `/tmp/iop-inventory-query --env <env> [selector]` 로 bounded 결과를 조회한다. 조회 도구는 `agent-test/inventory.yaml` manifest에서 환경별 파일을 resolve한다. selector는 `--model`, `--node`, `--provider` 중 최대 하나만 허용한다. selector 없이 실행하면 `inventory_id`, `common_inventory`, `test_env`, `profile`, `last_updated_at`, `source`, `edge`, `build` top-level key만 포함한 env projection을 반환하고 `model`, `nodes` 전체는 제외한다. selector 결과는 `{"env","kind","selector","matches":[{"path","value"}]}`이고 `path`는 오름차순으로 정렬한다. exit code는 성공 0, zero match 1, flag/YAML/schema 오류 2를 사용한다. substring/정규식/대소문자 접기는 하지 않고 YAML path로 중복 제거한다.
- compose, native Edge, dev-runtime provider pool처럼 같은 환경 안의 실행 모드는 `런타임 프로필`에서 분리한다.

4
agent-test/dev/edge-smoke.md
View file

@ -3,7 +3,7 @@ test_env: dev
test_profile: edge-smoke
domain: edge
verification_type: smoke
last_rule_updated_at: 2026-07-30
last_rule_updated_at: 2026-08-02
---
# edge-smoke dev 테스트
@ -52,7 +52,7 @@ dev-runtime provider pool과 4-node 연결 상태를 점검할 때는 `agent-tes
- Edge OpenAI-compatible base URL: `http://toki-labs.com:18083/v1`
- Edge-Node TCP transport: `toki-labs.com:18084`
- active model aliases: `laguna-s:2.1`(GX10), `ornith:35b`/`ornith-fast`(OneXPlayer/RTX5090), `qwen3.6:35b`(mac-mlx-vllm)
- host Pi default profile: `agent-test/inventory-dev.yaml`의 `model.pi_agent_profile` 기준. 현재 기본 provider/model/thinking level은 `iop` / `laguna-s:2.1` / `high`이며, Pi는 IOP Edge `http://toki-labs.com:18083/v1`의 Laguna model group을 호출한다. Pi local direct providers는 제거된 상태다.
- host Pi default profile: `agent-test/inventory-agent.yaml`의 `environments.dev.agents.pi` 기준. 현재 기본 provider/model/thinking level은 `iop` / `laguna-s:2.1` / `high`이며, Pi는 IOP Edge `http://toki-labs.com:18083/v1`의 Laguna model group을 호출한다. Pi local direct providers는 제거된 상태다.
- provider/model separation: Laguna는 GX10 `poolside_v1`, Qwen은 mac `qwen`/`qwen3`, Ornith는 Lemonade runtime profile을 사용한다. dev-corp `gemma4:26b`와 stopped DiffusionGemma 설정을 이들 profile에 섞지 않는다.
노드 후보:

3
agent-test/dev/rules.md
View file

@ -1,6 +1,6 @@
---
test_env: dev
last_rule_updated_at: 2026-07-30
last_rule_updated_at: 2026-08-02
---
# dev 테스트 규칙
@ -81,6 +81,7 @@ dev-runtime provider pool은 compose Edge-Node TCP `19003`이 아니라 native E
## 노드/Provider 인벤토리 위치
- 공통 machine-readable 기준과 환경 routing: `agent-test/inventory.yaml`
- 에이전트 host profile 기준: `agent-test/inventory-agent.yaml`의 `environments.dev.agents`
- dev-runtime provider pool machine-readable 기준: `agent-test/inventory-dev.yaml`
- dev-runtime provider pool과 Edge/OpenAI-compatible 입력 표면 상세 기준: `agent-test/dev/edge-smoke.md`
- dev-runtime Node 접속과 bootstrap 상세 기준: `agent-test/dev/node-smoke.md`

468
agent-test/inventory-agent.yaml Normal file
View file

@ -0,0 +1,468 @@
inventory_id: inventory-agent
schema_version: 1
last_updated_at: "2026-08-02"
scope:
type: shared_agent_host_profiles
description: Agent-specific host configuration, observed defaults, and environment-scoped status. Node, provider, endpoint, workspace, and SSH details remain in the referenced environment inventory.
policy:
secret_values_tracked: false
host_connection_values_owned_by: environment_inventory
provider_runtime_source_of_truth: environment_inventory
agent_provider_compatibility_observations: allowed
node_reference_policy: reference_only
source_inventories:
common: agent-test/inventory.yaml
dev: agent-test/inventory-dev.yaml
dev-corp: agent-test/inventory-dev-corp.yaml
environments:
dev:
source_inventory: agent-test/inventory-dev.yaml
agents:
claude:
status: not_configured
note: No Claude host profile is recorded in the dev inventory.
codex:
status: node_reference_only
node_ref: mac-codex-node
note: The CLI adapter and MLX provider runtime remain owned by the referenced dev node inventory.
pi:
status: configured
host_binding: not_recorded
observed_at: "2026-07-24"
config_scope: host_local_pi_agent
config_dir: /config/.pi/agent
current_default_provider: iop
current_default_model: laguna-s:2.1
current_default_thinking_level: high
current_default_api: openai-completions
current_default_base_url: http://toki-labs.com:18083/v1
current_default_auth_header: true
sampling_policy:
explicit_parameters: false
source: dev_laguna_provider_defaults
provider_default:
temperature: 0.7
top_p: 0.95
top_k: 20
omitted_fields:
- temperature
- top_p
- top_k
- repeat_penalty
note: Pi omits these fields and the selected GX10 Laguna provider applies the model baseline.
api_key_value_tracked: false
endpoint_policy: current default Pi route is IOP Edge provider iop with model laguna-s:2.1. Pi does not bypass Edge to the GX10 vLLM endpoint.
local_pi_providers_at_observation:
- iop
direct_providers_removed_at_observation:
- iop-gemma
- ornith-direct
- ornith-lemonade
iop_provider_models_at_observation:
- qwen3.6:35b
- ornith:35b
- ornith-fast
- laguna-s:2.1
pi_model_parameters:
provider: iop
model_id: laguna-s:2.1
display_name: IOP Poolside Laguna S 2.1
api: openai-completions
context_window: 262144
max_tokens: 65536
reasoning: true
thinking_level: high
thinking_format: chat-template
enable_thinking_source: thinking.enabled
preserve_thinking: true
current_laguna_parameter_alignment:
model_alias: laguna-s:2.1
pi_context_window: 262144
edge_context_window: 262144
pi_max_tokens: 65536
edge_default_max_tokens: 65536
pi_output_budget_within_edge_limit: true
edge_model_group_registered: true
edge_group_min_max_tokens: 0
gx10_edge_capacity: 4
gx10_long_context_capacity: 1
reasoning_parser: poolside_v1
tool_call_parser: poolside_v1
quantization: nvfp4
dflash_speculative_tokens: 7
chat_template_container_path: /run/iop/laguna-s-2.1-thinking.jinja
thinking_generation_prefix: "<think>\n"
note: The active GX10 runtime uses the official Laguna S 2.1 NVFP4 checkpoint and quantization-matched DFlash NVFP4 draft. Its local chat-template override adds a newline after the generation-prefix think tag so enabled requests do not immediately emit the closing think tag.
qwen_runtime_parameter_alignment:
alignment_policy: Pi model parameters must be kept in sync with the Edge model catalog and each provider runtime profile. Do not optimize or debug Pi behavior from the endpoint URL alone; verify the alias-to-served-model mapping, context window, max token budget, thinking/reasoning parser, chat template kwargs, and provider capacity/sequence limits together.
common_requirements:
model_alias: qwen3.6:35b
pi_context_window: 262144
edge_context_window: 262144
pi_max_tokens: 32768
edge_default_max_tokens: 32768
edge_min_max_tokens: 32768
pi_reasoning: true
edge_reasoning_policy: model_native_thinking
required_template_policy: Qwen providers keep default_chat_template_kwargs.enable_thinking=true
required_reasoning_parser: qwen3
engine_matrix:
previous_gx10-vllm:
status: historical_replaced_by_laguna_s_2_1
runtime_type: vllm
served_model: nvidia/Qwen3.6-35B-A3B-NVFP4
context_match: pi contextWindow 262144 maps to vLLM max_model_len 262144
output_budget_match: pi maxTokens 32768 maps to provider default_max_tokens 32768
thinking_match: pi reasoning=true and thinking_level=high map to reasoning_parser=qwen3 and default_chat_template_kwargs.enable_thinking=true
concurrency_match: provider capacity 4 maps to runtime max_num_seqs 4
tool_parser_profile: qwen3_xml
mac-mlx-vllm:
runtime_type: vllm-mlx
served_model: mlx-community/Qwen3.6-35B-A3B-4bit
context_match: pi contextWindow 262144 maps to vLLM-MLX max_request_tokens 262144 and max_kv_size 262144
output_budget_match: pi maxTokens 32768 maps to provider default_max_tokens 32768
thinking_match: pi reasoning=true and thinking_level=high map to reasoning_parser=qwen3 and default_chat_template_kwargs.enable_thinking=true
concurrency_match: provider capacity 2 maps to runtime max_num_seqs 2
tool_parser_profile: qwen
onexplayer-lemonade:
runtime_type: lemonade
served_model: Qwen3.6-35B-A3B-MTP-GGUF
current_runtime_override_2026_07_12: OneXPlayer Lemonade was switched to Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M for direct Ornith validation; it is not a valid Qwen provider-pool member until Edge provider config is updated.
context_match: Qwen should use the same Lemonade unified KV profile when re-enabled; keep total ctx_size 524288, omit -np, and pass --kv-unified to avoid the prior fixed 174848-token slot partitioning.
output_budget_match: pi maxTokens 32768 maps to load default_max_tokens 32768
thinking_match: pi reasoning=true and thinking_level=high uses model-native thinking; Lemonade load keeps MTP disabled and tool_choice_auto_verified=true
concurrency_match: provider capacity remains capped at 3 for IOP admission; with unified KV, Lemonade auto slots may report 4 runtime slots, so Edge admission must remain the concurrency guard.
tool_parser_profile: lemonade_qwen_tool_choice_auto_verified
qwen_reenable_profile:
status: applicable_not_reloaded_while_ornith_q5_is_active
ctx_size: 524288
llamacpp_args: "--spec-type none -cb -fa on -b 4096 -ub 1024 --kv-unified"
removed_args: "-np 3"
expected_slot_n_ctx: 262144
expected_capacity: 3
expected_long_context_capacity: 2
validation_basis: Same Lemonade llama.cpp backend and current Ornith Q5 runtime validated --kv-unified with auto slots reporting n_ctx 262144; Qwen still needs smoke validation after it is reloaded.
separation_note: These alignment rules are for the dev Qwen3.6 provider pool only. Gemma4 and DiffusionGemma use separate provider/model profiles and must not inherit Qwen parser/template/context assumptions.
provider_passthrough:
default: pure_passthrough
selector: none; request model/provider capability determines passthrough vs normalized execution
verification: Chat Completions streaming probe returned HTTP 200 through the provider tunnel with provider body preserved and no IOP response envelope injected.
previous_qwen_provider_pool_routing:
status: historical_gx10_and_onexplayer_no_longer_in_qwen_group
observed_at: "2026-07-09"
caller_shape: Pi-compatible OpenAI Chat Completions streaming requests to provider/model iop/qwen3.6:35b
concurrent_requests: 10
peak_total_in_flight: 9
peak_total_queued_snapshot_units: 3
final_recovery: in_flight_0_queued_0
provider_peaks:
gx10-vllm:
runtime_type: vllm
served_model: nvidia/Qwen3.6-35B-A3B-NVFP4
capacity: 4
max_in_flight: 4
max_queued: 1
mac-mlx-vllm:
runtime_type: vllm-mlx
served_model: mlx-community/Qwen3.6-35B-A3B-4bit
capacity: 2
max_in_flight: 2
max_queued: 1
onexplayer-lemonade:
runtime_type: lemonade
served_model: Qwen3.6-35B-A3B-MTP-GGUF
capacity: 3
max_in_flight: 3
max_queued: 1
non_default_local_provider:
provider: iop-gemma
status: removed_from_pi_models_json_2026_07_12
api: openai-completions
base_url: http://192.168.0.91:8002/v1
served_model: nvidia/diffusiongemma-26B-A4B-it-NVFP4
reasoning: false
current_runtime_status: stopped_2026_07_12_endpoint_now_serves_laguna_s_2_1
spark_laguna_direct_profile:
observed_at: "2026-07-24"
provider: gx10-vllm
status: active_through_iop_edge
family: poolside_laguna
api: openai-completions
base_url: http://192.168.0.91:8001/v1
model_id: laguna-s:2.1
display_name: Poolside Laguna S 2.1 NVFP4
context_window: 262144
max_tokens: 65536
reasoning: true
endpoint_policy: Pi connects through IOP Edge under laguna-s:2.1 and does not use the direct GX10 endpoint.
api_key_value_tracked: false
pi_current_default_at_observation:
default_provider: iop
default_model: laguna-s:2.1
default_api: openai-completions
default_base_url: http://toki-labs.com:18083/v1
default_thinking_level: high
thinking_format: chat-template
enable_thinking_source: thinking.enabled
preserve_thinking: true
runtime_summary:
host: toki@192.168.0.91
container_name: iop-vllm-laguna-s21
docker_image: iop-vllm-laguna-s21:v0.25.1-cu130
docker_image_id: sha256:4eb2bb71c0ffc6ceb54dd3381030169f5e24ac1477fb1faa1ba90e57e6996331
vllm_version: 0.25.1
torch_version: 2.11.0+cu130
transformers_version: 5.12.1
compressed_tensors_version: 0.17.0
flashinfer_python_version: 0.6.15.dev20260712
gpu: NVIDIA GB10
endpoint: http://192.168.0.91:8001/v1
hf_model: poolside/Laguna-S-2.1-NVFP4
hf_url: https://huggingface.co/poolside/Laguna-S-2.1-NVFP4
hf_revision: 07614121b31898586430f189d27a25a0be310843
hf_cache_snapshot_path: /models/.cache/huggingface/hub/models--poolside--Laguna-S-2.1-NVFP4/snapshots/07614121b31898586430f189d27a25a0be310843
draft_model: poolside/Laguna-S-2.1-DFlash-NVFP4
draft_url: https://huggingface.co/poolside/Laguna-S-2.1-DFlash-NVFP4
draft_revision: 723794750422b3efbf3a7b3af76dffb4ba035943
served_model_name: laguna-s:2.1
quantization: nvfp4
vllm_quantization_backend: compressed-tensors
dtype: bfloat16
max_model_len: 262144
max_num_seqs: 4
gpu_memory_utilization: 0.70
enable_prefix_caching: true
enable_chunked_prefill: true
enable_auto_tool_choice: true
tool_call_parser: poolside_v1
reasoning_parser: poolside_v1
chat_template_host_path: /home/toki/iop-gx10-vllm/laguna-s-2.1-thinking.jinja
chat_template_container_path: /run/iop/laguna-s-2.1-thinking.jinja
thinking_generation_prefix: "<think>\n"
default_chat_template_kwargs:
enable_thinking: true
generation_config_override:
temperature: 0.7
top_p: 0.95
speculative_config:
method: dflash
num_speculative_tokens: 7
model: poolside/Laguna-S-2.1-DFlash-NVFP4
attention_backend: FLASHINFER
kv_cache_dtype: float8_e4m3fn
available_kv_cache_memory_gib: 13.08
gpu_kv_cache_tokens: 373711
max_concurrency_for_262144_token_requests: 1.43
docker_runtime:
recreated_at: "2026-07-24"
host_port: 8001
container_port: 8000
network_mode: bridge
ipc_mode: host
restart_policy: "no"
bind_mounts:
/home/toki/Data/models: /models
/home/toki/Data/vllm/cache/laguna-s21: /root/.cache
/home/toki/iop-gx10-vllm/laguna-s-2.1-thinking.jinja: /run/iop/laguna-s-2.1-thinking.jinja
smoke:
direct_models: passed
direct_chat_completions: passed
edge_chat_completions: passed
pi_agentic_tool_reasoning_stream: passed
reasoning_field: reasoning
pi_high_result: thinking_start, thinking_delta, and thinking_end observed
pi_off_result: no thinking event and final text observed
simple_no_tool_reasoning_note: The previous stock generation prefix closed the think block immediately. The current local template uses a trailing newline and a simple Pi high request emits a non-empty thinking block.
previous_spark_ornith_direct_profile:
observed_at: "2026-07-12"
provider: ornith-direct
status: historical_runtime_replaced_by_laguna_s_2_1
family: spark_ornith
api: openai-completions
base_url: http://192.168.0.91:8001/v1
model_id: ornith:35b
display_name: Ornith 1.0 35B FP8 Direct vLLM
context_window: 262144
max_tokens: 32768
reasoning: true
endpoint_policy: Historical observation only. The endpoint now serves laguna-s:2.1 through IOP Edge.
api_key_value_tracked: false
pi_current_default_at_observation:
default_provider: iop
default_model: ornith:35b
default_api: openai-completions
default_base_url: http://toki-labs.com:18083/v1
qwen_parameter_alignment:
pi_context_window: 262144
pi_max_tokens: 32768
context_match: matches the dev Qwen Pi contextWindow and vLLM max_model_len value
output_budget_match: matches the dev Qwen Pi maxTokens/default_max_tokens value
runtime_summary:
host: toki@192.168.0.91
container_name: iop-vllm-ornith35b-fp8
docker_image: vllm/vllm-openai:nightly-aarch64
docker_image_id: sha256:a720df3e84a89d7db47a3b7a0511cb5b312e203fc4956f7493df248299267a6f
vllm_image_tag_env: vllm/vllm-openai:nightly-a346d589f5932d4234bf5bf8718f10e26d187021
vllm_version: 0.23.1rc1.dev223+ga346d589f
vllm_build_commit: a346d589f5932d4234bf5bf8718f10e26d187021
vllm_build_url: https://buildkite.com/vllm/release-v2/builds/2923
torch_version: 2.11.0+cu130
transformers_version: 5.12.1
compressed_tensors_version: 0.17.0
flashinfer_python_version: 0.6.12
cuda_version_env: 13.0.2
gpu: NVIDIA GB10
nvidia_driver_version: 580.126.09
endpoint: http://192.168.0.91:8001/v1
hf_model: deepreinforce-ai/Ornith-1.0-35B-FP8
hf_url: https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B-FP8
hf_revision: 1ab57ce0b44950e498a88756f40ad1ed4d0f30ca
hf_ref: main
revision_argument: not_set_uses_repo_default_main_at_container_start
tokenizer_revision_argument: not_set_uses_model_revision
hf_home: /models/.cache/huggingface
hf_cache_snapshot_path: /models/.cache/huggingface/hub/models--deepreinforce-ai--Ornith-1.0-35B-FP8/snapshots/1ab57ce0b44950e498a88756f40ad1ed4d0f30ca
served_model_name: ornith:35b
openai_models_root: deepreinforce-ai/Ornith-1.0-35B-FP8
openai_models_max_model_len: 262144
quantization: fp8
vllm_quantization_backend: compressed-tensors
model_type: qwen3_5_moe
model_architecture: Qwen3_5MoeForConditionalGeneration
quantization_config_source: model config.json compressed-tensors W8A8 FP8, weight channel strategy, input activation token strategy, linear_attn/embed/lm_head/shared gates ignored
dtype: bfloat16
max_model_len: 262144
max_num_seqs: 4
max_num_batched_tokens: vllm_default_not_overridden
gpu_memory_utilization: 0.50
enable_prefix_caching: true
enable_chunked_prefill: true
async_scheduling: true
tensor_parallel_size: 1
pipeline_parallel_size: 1
data_parallel_size: 1
load_format: auto
download_dir: null
kv_cache_dtype: auto
enforce_eager: false
seed: 0
enable_auto_tool_choice: true
tool_call_parser: qwen3_xml
reasoning_parser: qwen3
reasoning_parser_enable_in_reasoning: false
trust_remote_code: true
language_model_only: true
attention_backend: FLASH_ATTN
sampling_backend: FlashInfer top-p/top-k
fp8_linear_kernel: CutlassFP8ScaledMMLinearKernel for CompressedTensorsW8A8Fp8
fp8_moe_backend: TRITON
generation_config_override:
temperature: 0.6
top_k: 20
top_p: 0.95
eos_token_id:
- 248046
- 248044
bos_token_id: 248044
docker_runtime:
created_at: "2026-07-12T08:41:04Z"
entrypoint:
- vllm
- serve
container_port: 8000
host_port: 8001
network_mode: bridge
ipc_mode: private
shm_size_bytes: 67108864
restart_policy: "no"
gpu_device_request: all
bind_mounts:
/home/toki/Data/models: /models
/home/toki/Data/vllm/templates: /templates
model_load_memory_gib: 34.43
model_load_seconds: 196.929359
weight_download_seconds: not_observed_cache_hit_restart
cuda_graph_memory_estimated_gib: 0.13
cuda_graph_pool_memory_actual_gib: 0.07
available_kv_cache_memory_gib: 20.79
gpu_kv_cache_tokens: 1060912
max_concurrency_for_262144_token_requests: 4.05
startup_warnings:
- vLLM command does not pin --revision; the recorded hf_revision is the downloaded snapshot observed in cache.
- Prefix caching in Mamba cache align mode is experimental for this architecture.
- Default MoE config for NVIDIA_GB10 FP8 was not found; vLLM used default MoE config, which may affect performance.
- Model generation_config overrides vLLM defaults unless callers provide sampling params or vLLM is relaunched with --generation-config vllm.
official_4bit_gguf_status:
hf_model: deepreinforce-ai/Ornith-1.0-35B-GGUF
hf_url: https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B-GGUF
hf_revision: c2e1703039380de4ce6820e97afd185682d3c16c
hf_ref: main
hf_cache_snapshot_path: /models/.cache/huggingface/hub/models--deepreinforce-ai--Ornith-1.0-35B-GGUF/snapshots/c2e1703039380de4ce6820e97afd185682d3c16c
file: ornith-1.0-35b-Q4_K_M.gguf
status: not_active
reason: current vLLM GGUF plugin path exposed /v1/models but failed semantic generation smoke; active working runtime uses the official FP8 vLLM model instead.
smoke:
english_final_answer: passed
korean_final_answer: passed
pi_direct_cli: passed
output_throughput:
method: completion_tokens divided by end-to-end request wall time, max_tokens=768
average_completion_tok_s: 29.8
run_completion_tok_s:
- 29.65
- 30.62
- 29.15
note: vLLM logs showed aggregate generation throughput around 50-61 tok/s with two running requests.
ornith_lemonade_direct_profile:
observed_at: "2026-07-12"
provider: ornith-lemonade
status: removed_from_pi_models_json_2026_07_12_runtime_still_active_as_onexplayer_iop_ornith_provider
family: spark_ornith
api: openai-completions
base_url: http://192.168.0.59:13305/v1
model_id: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M
display_name: Ornith 1.0 35B Q5 Direct Lemonade
context_window: 262144
max_tokens: 32768
reasoning: true
config_file: /config/.pi/agent/models.json
default_profile_changed: true
pi_default_provider_after_add: iop
pi_default_model_after_add: ornith:35b
endpoint_policy: Pi no longer connects directly to the OneXPlayer Lemonade OpenAI-compatible endpoint. This runtime remains reachable through IOP Edge under ornith:35b.
api_key_value_tracked: false
runtime_alignment:
lemonade_runtime_model: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M
lemonade_response_model: ornith-1.0-35b-Q5_K_M.gguf
quantization: Q5_K_M
ctx_size: 524288
kv_policy: unified_kv_with_np_omitted
observed_slot_n_ctx: 262144
iop_capacity_guard: 3
long_context_capacity: 2
smoke:
endpoint_direct_chat_completions: passed
request_model: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M
response_model: ornith-1.0-35b-Q5_K_M.gguf
finish_reason: stop
content: "4"
completion_tokens: 2
separation_note: The host Pi default is Laguna S 2.1 through IOP Edge. Keep Laguna, Qwen, Ornith, and dev-corp Gemma parser/template/runtime settings separate.
agy:
status: not_configured
note: No agy host profile is recorded in the dev inventory.
dev-corp:
source_inventory: agent-test/inventory-dev-corp.yaml
agents:
claude:
status: not_configured
codex:
status: not_configured
pi:
status: not_configured
agy:
status: not_configured

428
agent-test/inventory-dev.yaml
View file

@ -2,7 +2,7 @@ inventory_id: inventory-dev
common_inventory: agent-test/inventory.yaml
test_env: dev
profile: dev-runtime-provider-pool
last_updated_at: "2026-07-30"
last_updated_at: "2026-08-02"
source:
remote_runner:
@ -600,427 +600,6 @@ model:
preserve_thinking: true
tool_call_parser_status: Pi read-tool multi-turn smoke passed with reasoning before the tool call and after the tool result
separation_note: Keep Laguna poolside_v1, Qwen qwen/qwen3, and dev-corp Gemma parser/template profiles separate.
pi_agent_profile:
observed_at: "2026-07-24"
config_scope: host_local_pi_agent
config_dir: /config/.pi/agent
current_default_provider: iop
current_default_model: laguna-s:2.1
current_default_thinking_level: high
current_default_api: openai-completions
current_default_base_url: http://toki-labs.com:18083/v1
current_default_auth_header: true
sampling_policy:
explicit_parameters: false
source: dev_laguna_provider_defaults
provider_default:
temperature: 0.7
top_p: 0.95
top_k: 20
omitted_fields:
- temperature
- top_p
- top_k
- repeat_penalty
note: Pi omits these fields and the selected GX10 Laguna provider applies the model baseline.
api_key_value_tracked: false
endpoint_policy: current default Pi route is IOP Edge provider iop with model laguna-s:2.1. Pi does not bypass Edge to the GX10 vLLM endpoint.
local_pi_providers_at_observation:
- iop
direct_providers_removed_at_observation:
- iop-gemma
- ornith-direct
- ornith-lemonade
iop_provider_models_at_observation:
- qwen3.6:35b
- ornith:35b
- ornith-fast
- laguna-s:2.1
pi_model_parameters:
provider: iop
model_id: laguna-s:2.1
display_name: IOP Poolside Laguna S 2.1
api: openai-completions
context_window: 262144
max_tokens: 65536
reasoning: true
thinking_level: high
thinking_format: chat-template
enable_thinking_source: thinking.enabled
preserve_thinking: true
current_laguna_parameter_alignment:
model_alias: laguna-s:2.1
pi_context_window: 262144
edge_context_window: 262144
pi_max_tokens: 65536
edge_default_max_tokens: 65536
pi_output_budget_within_edge_limit: true
edge_model_group_registered: true
edge_group_min_max_tokens: 0
gx10_edge_capacity: 4
gx10_long_context_capacity: 1
reasoning_parser: poolside_v1
tool_call_parser: poolside_v1
quantization: nvfp4
dflash_speculative_tokens: 7
chat_template_container_path: /run/iop/laguna-s-2.1-thinking.jinja
thinking_generation_prefix: "<think>\n"
note: The active GX10 runtime uses the official Laguna S 2.1 NVFP4 checkpoint and quantization-matched DFlash NVFP4 draft. Its local chat-template override adds a newline after the generation-prefix think tag so enabled requests do not immediately emit the closing think tag.
qwen_runtime_parameter_alignment:
alignment_policy: Pi model parameters must be kept in sync with the Edge model catalog and each provider runtime profile. Do not optimize or debug Pi behavior from the endpoint URL alone; verify the alias-to-served-model mapping, context window, max token budget, thinking/reasoning parser, chat template kwargs, and provider capacity/sequence limits together.
common_requirements:
model_alias: qwen3.6:35b
pi_context_window: 262144
edge_context_window: 262144
pi_max_tokens: 32768
edge_default_max_tokens: 32768
edge_min_max_tokens: 32768
pi_reasoning: true
edge_reasoning_policy: model_native_thinking
required_template_policy: Qwen providers keep default_chat_template_kwargs.enable_thinking=true
required_reasoning_parser: qwen3
engine_matrix:
previous_gx10-vllm:
status: historical_replaced_by_laguna_s_2_1
runtime_type: vllm
served_model: nvidia/Qwen3.6-35B-A3B-NVFP4
context_match: pi contextWindow 262144 maps to vLLM max_model_len 262144
output_budget_match: pi maxTokens 32768 maps to provider default_max_tokens 32768
thinking_match: pi reasoning=true and thinking_level=high map to reasoning_parser=qwen3 and default_chat_template_kwargs.enable_thinking=true
concurrency_match: provider capacity 4 maps to runtime max_num_seqs 4
tool_parser_profile: qwen3_xml
mac-mlx-vllm:
runtime_type: vllm-mlx
served_model: mlx-community/Qwen3.6-35B-A3B-4bit
context_match: pi contextWindow 262144 maps to vLLM-MLX max_request_tokens 262144 and max_kv_size 262144
output_budget_match: pi maxTokens 32768 maps to provider default_max_tokens 32768
thinking_match: pi reasoning=true and thinking_level=high map to reasoning_parser=qwen3 and default_chat_template_kwargs.enable_thinking=true
concurrency_match: provider capacity 2 maps to runtime max_num_seqs 2
tool_parser_profile: qwen
onexplayer-lemonade:
runtime_type: lemonade
served_model: Qwen3.6-35B-A3B-MTP-GGUF
current_runtime_override_2026_07_12: OneXPlayer Lemonade was switched to Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M for direct Ornith validation; it is not a valid Qwen provider-pool member until Edge provider config is updated.
context_match: Qwen should use the same Lemonade unified KV profile when re-enabled; keep total ctx_size 524288, omit -np, and pass --kv-unified to avoid the prior fixed 174848-token slot partitioning.
output_budget_match: pi maxTokens 32768 maps to load default_max_tokens 32768
thinking_match: pi reasoning=true and thinking_level=high uses model-native thinking; Lemonade load keeps MTP disabled and tool_choice_auto_verified=true
concurrency_match: provider capacity remains capped at 3 for IOP admission; with unified KV, Lemonade auto slots may report 4 runtime slots, so Edge admission must remain the concurrency guard.
tool_parser_profile: lemonade_qwen_tool_choice_auto_verified
qwen_reenable_profile:
status: applicable_not_reloaded_while_ornith_q5_is_active
ctx_size: 524288
llamacpp_args: "--spec-type none -cb -fa on -b 4096 -ub 1024 --kv-unified"
removed_args: "-np 3"
expected_slot_n_ctx: 262144
expected_capacity: 3
expected_long_context_capacity: 2
validation_basis: Same Lemonade llama.cpp backend and current Ornith Q5 runtime validated --kv-unified with auto slots reporting n_ctx 262144; Qwen still needs smoke validation after it is reloaded.
separation_note: These alignment rules are for the dev Qwen3.6 provider pool only. Gemma4 and DiffusionGemma use separate provider/model profiles and must not inherit Qwen parser/template/context assumptions.
provider_passthrough:
default: pure_passthrough
selector: none; request model/provider capability determines passthrough vs normalized execution
verification: Chat Completions streaming probe returned HTTP 200 through the provider tunnel with provider body preserved and no IOP response envelope injected.
previous_qwen_provider_pool_routing:
status: historical_gx10_and_onexplayer_no_longer_in_qwen_group
observed_at: "2026-07-09"
caller_shape: Pi-compatible OpenAI Chat Completions streaming requests to provider/model iop/qwen3.6:35b
concurrent_requests: 10
peak_total_in_flight: 9
peak_total_queued_snapshot_units: 3
final_recovery: in_flight_0_queued_0
provider_peaks:
gx10-vllm:
runtime_type: vllm
served_model: nvidia/Qwen3.6-35B-A3B-NVFP4
capacity: 4
max_in_flight: 4
max_queued: 1
mac-mlx-vllm:
runtime_type: vllm-mlx
served_model: mlx-community/Qwen3.6-35B-A3B-4bit
capacity: 2
max_in_flight: 2
max_queued: 1
onexplayer-lemonade:
runtime_type: lemonade
served_model: Qwen3.6-35B-A3B-MTP-GGUF
capacity: 3
max_in_flight: 3
max_queued: 1
non_default_local_provider:
provider: iop-gemma
status: removed_from_pi_models_json_2026_07_12
api: openai-completions
base_url: http://192.168.0.91:8002/v1
served_model: nvidia/diffusiongemma-26B-A4B-it-NVFP4
reasoning: false
current_runtime_status: stopped_2026_07_12_endpoint_now_serves_laguna_s_2_1
spark_laguna_direct_profile:
observed_at: "2026-07-24"
provider: gx10-vllm
status: active_through_iop_edge
family: poolside_laguna
api: openai-completions
base_url: http://192.168.0.91:8001/v1
model_id: laguna-s:2.1
display_name: Poolside Laguna S 2.1 NVFP4
context_window: 262144
max_tokens: 65536
reasoning: true
endpoint_policy: Pi connects through IOP Edge under laguna-s:2.1 and does not use the direct GX10 endpoint.
api_key_value_tracked: false
pi_current_default_at_observation:
default_provider: iop
default_model: laguna-s:2.1
default_api: openai-completions
default_base_url: http://toki-labs.com:18083/v1
default_thinking_level: high
thinking_format: chat-template
enable_thinking_source: thinking.enabled
preserve_thinking: true
runtime_summary:
host: toki@192.168.0.91
container_name: iop-vllm-laguna-s21
docker_image: iop-vllm-laguna-s21:v0.25.1-cu130
docker_image_id: sha256:4eb2bb71c0ffc6ceb54dd3381030169f5e24ac1477fb1faa1ba90e57e6996331
vllm_version: 0.25.1
torch_version: 2.11.0+cu130
transformers_version: 5.12.1
compressed_tensors_version: 0.17.0
flashinfer_python_version: 0.6.15.dev20260712
gpu: NVIDIA GB10
endpoint: http://192.168.0.91:8001/v1
hf_model: poolside/Laguna-S-2.1-NVFP4
hf_url: https://huggingface.co/poolside/Laguna-S-2.1-NVFP4
hf_revision: 07614121b31898586430f189d27a25a0be310843
hf_cache_snapshot_path: /models/.cache/huggingface/hub/models--poolside--Laguna-S-2.1-NVFP4/snapshots/07614121b31898586430f189d27a25a0be310843
draft_model: poolside/Laguna-S-2.1-DFlash-NVFP4
draft_url: https://huggingface.co/poolside/Laguna-S-2.1-DFlash-NVFP4
draft_revision: 723794750422b3efbf3a7b3af76dffb4ba035943
served_model_name: laguna-s:2.1
quantization: nvfp4
vllm_quantization_backend: compressed-tensors
dtype: bfloat16
max_model_len: 262144
max_num_seqs: 4
gpu_memory_utilization: 0.70
enable_prefix_caching: true
enable_chunked_prefill: true
enable_auto_tool_choice: true
tool_call_parser: poolside_v1
reasoning_parser: poolside_v1
chat_template_host_path: /home/toki/iop-gx10-vllm/laguna-s-2.1-thinking.jinja
chat_template_container_path: /run/iop/laguna-s-2.1-thinking.jinja
thinking_generation_prefix: "<think>\n"
default_chat_template_kwargs:
enable_thinking: true
generation_config_override:
temperature: 0.7
top_p: 0.95
speculative_config:
method: dflash
num_speculative_tokens: 7
model: poolside/Laguna-S-2.1-DFlash-NVFP4
attention_backend: FLASHINFER
kv_cache_dtype: float8_e4m3fn
available_kv_cache_memory_gib: 13.08
gpu_kv_cache_tokens: 373711
max_concurrency_for_262144_token_requests: 1.43
docker_runtime:
recreated_at: "2026-07-24"
host_port: 8001
container_port: 8000
network_mode: bridge
ipc_mode: host
restart_policy: "no"
bind_mounts:
/home/toki/Data/models: /models
/home/toki/Data/vllm/cache/laguna-s21: /root/.cache
/home/toki/iop-gx10-vllm/laguna-s-2.1-thinking.jinja: /run/iop/laguna-s-2.1-thinking.jinja
smoke:
direct_models: passed
direct_chat_completions: passed
edge_chat_completions: passed
pi_agentic_tool_reasoning_stream: passed
reasoning_field: reasoning
pi_high_result: thinking_start, thinking_delta, and thinking_end observed
pi_off_result: no thinking event and final text observed
simple_no_tool_reasoning_note: The previous stock generation prefix closed the think block immediately. The current local template uses a trailing newline and a simple Pi high request emits a non-empty thinking block.
previous_spark_ornith_direct_profile:
observed_at: "2026-07-12"
provider: ornith-direct
status: historical_runtime_replaced_by_laguna_s_2_1
family: spark_ornith
api: openai-completions
base_url: http://192.168.0.91:8001/v1
model_id: ornith:35b
display_name: Ornith 1.0 35B FP8 Direct vLLM
context_window: 262144
max_tokens: 32768
reasoning: true
endpoint_policy: Historical observation only. The endpoint now serves laguna-s:2.1 through IOP Edge.
api_key_value_tracked: false
pi_current_default_at_observation:
default_provider: iop
default_model: ornith:35b
default_api: openai-completions
default_base_url: http://toki-labs.com:18083/v1
qwen_parameter_alignment:
pi_context_window: 262144
pi_max_tokens: 32768
context_match: matches the dev Qwen Pi contextWindow and vLLM max_model_len value
output_budget_match: matches the dev Qwen Pi maxTokens/default_max_tokens value
runtime_summary:
host: toki@192.168.0.91
container_name: iop-vllm-ornith35b-fp8
docker_image: vllm/vllm-openai:nightly-aarch64
docker_image_id: sha256:a720df3e84a89d7db47a3b7a0511cb5b312e203fc4956f7493df248299267a6f
vllm_image_tag_env: vllm/vllm-openai:nightly-a346d589f5932d4234bf5bf8718f10e26d187021
vllm_version: 0.23.1rc1.dev223+ga346d589f
vllm_build_commit: a346d589f5932d4234bf5bf8718f10e26d187021
vllm_build_url: https://buildkite.com/vllm/release-v2/builds/2923
torch_version: 2.11.0+cu130
transformers_version: 5.12.1
compressed_tensors_version: 0.17.0
flashinfer_python_version: 0.6.12
cuda_version_env: 13.0.2
gpu: NVIDIA GB10
nvidia_driver_version: 580.126.09
endpoint: http://192.168.0.91:8001/v1
hf_model: deepreinforce-ai/Ornith-1.0-35B-FP8
hf_url: https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B-FP8
hf_revision: 1ab57ce0b44950e498a88756f40ad1ed4d0f30ca
hf_ref: main
revision_argument: not_set_uses_repo_default_main_at_container_start
tokenizer_revision_argument: not_set_uses_model_revision
hf_home: /models/.cache/huggingface
hf_cache_snapshot_path: /models/.cache/huggingface/hub/models--deepreinforce-ai--Ornith-1.0-35B-FP8/snapshots/1ab57ce0b44950e498a88756f40ad1ed4d0f30ca
served_model_name: ornith:35b
openai_models_root: deepreinforce-ai/Ornith-1.0-35B-FP8
openai_models_max_model_len: 262144
quantization: fp8
vllm_quantization_backend: compressed-tensors
model_type: qwen3_5_moe
model_architecture: Qwen3_5MoeForConditionalGeneration
quantization_config_source: model config.json compressed-tensors W8A8 FP8, weight channel strategy, input activation token strategy, linear_attn/embed/lm_head/shared gates ignored
dtype: bfloat16
max_model_len: 262144
max_num_seqs: 4
max_num_batched_tokens: vllm_default_not_overridden
gpu_memory_utilization: 0.50
enable_prefix_caching: true
enable_chunked_prefill: true
async_scheduling: true
tensor_parallel_size: 1
pipeline_parallel_size: 1
data_parallel_size: 1
load_format: auto
download_dir: null
kv_cache_dtype: auto
enforce_eager: false
seed: 0
enable_auto_tool_choice: true
tool_call_parser: qwen3_xml
reasoning_parser: qwen3
reasoning_parser_enable_in_reasoning: false
trust_remote_code: true
language_model_only: true
attention_backend: FLASH_ATTN
sampling_backend: FlashInfer top-p/top-k
fp8_linear_kernel: CutlassFP8ScaledMMLinearKernel for CompressedTensorsW8A8Fp8
fp8_moe_backend: TRITON
generation_config_override:
temperature: 0.6
top_k: 20
top_p: 0.95
eos_token_id:
- 248046
- 248044
bos_token_id: 248044
docker_runtime:
created_at: "2026-07-12T08:41:04Z"
entrypoint:
- vllm
- serve
container_port: 8000
host_port: 8001
network_mode: bridge
ipc_mode: private
shm_size_bytes: 67108864
restart_policy: "no"
gpu_device_request: all
bind_mounts:
/home/toki/Data/models: /models
/home/toki/Data/vllm/templates: /templates
model_load_memory_gib: 34.43
model_load_seconds: 196.929359
weight_download_seconds: not_observed_cache_hit_restart
cuda_graph_memory_estimated_gib: 0.13
cuda_graph_pool_memory_actual_gib: 0.07
available_kv_cache_memory_gib: 20.79
gpu_kv_cache_tokens: 1060912
max_concurrency_for_262144_token_requests: 4.05
startup_warnings:
- vLLM command does not pin --revision; the recorded hf_revision is the downloaded snapshot observed in cache.
- Prefix caching in Mamba cache align mode is experimental for this architecture.
- Default MoE config for NVIDIA_GB10 FP8 was not found; vLLM used default MoE config, which may affect performance.
- Model generation_config overrides vLLM defaults unless callers provide sampling params or vLLM is relaunched with --generation-config vllm.
official_4bit_gguf_status:
hf_model: deepreinforce-ai/Ornith-1.0-35B-GGUF
hf_url: https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B-GGUF
hf_revision: c2e1703039380de4ce6820e97afd185682d3c16c
hf_ref: main
hf_cache_snapshot_path: /models/.cache/huggingface/hub/models--deepreinforce-ai--Ornith-1.0-35B-GGUF/snapshots/c2e1703039380de4ce6820e97afd185682d3c16c
file: ornith-1.0-35b-Q4_K_M.gguf
status: not_active
reason: current vLLM GGUF plugin path exposed /v1/models but failed semantic generation smoke; active working runtime uses the official FP8 vLLM model instead.
smoke:
english_final_answer: passed
korean_final_answer: passed
pi_direct_cli: passed
output_throughput:
method: completion_tokens divided by end-to-end request wall time, max_tokens=768
average_completion_tok_s: 29.8
run_completion_tok_s:
- 29.65
- 30.62
- 29.15
note: vLLM logs showed aggregate generation throughput around 50-61 tok/s with two running requests.
ornith_lemonade_direct_profile:
observed_at: "2026-07-12"
provider: ornith-lemonade
status: removed_from_pi_models_json_2026_07_12_runtime_still_active_as_onexplayer_iop_ornith_provider
family: spark_ornith
api: openai-completions
base_url: http://192.168.0.59:13305/v1
model_id: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M
display_name: Ornith 1.0 35B Q5 Direct Lemonade
context_window: 262144
max_tokens: 32768
reasoning: true
config_file: /config/.pi/agent/models.json
default_profile_changed: true
pi_default_provider_after_add: iop
pi_default_model_after_add: ornith:35b
endpoint_policy: Pi no longer connects directly to the OneXPlayer Lemonade OpenAI-compatible endpoint. This runtime remains reachable through IOP Edge under ornith:35b.
api_key_value_tracked: false
runtime_alignment:
lemonade_runtime_model: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M
lemonade_response_model: ornith-1.0-35b-Q5_K_M.gguf
quantization: Q5_K_M
ctx_size: 524288
kv_policy: unified_kv_with_np_omitted
observed_slot_n_ctx: 262144
iop_capacity_guard: 3
long_context_capacity: 2
smoke:
endpoint_direct_chat_completions: passed
request_model: Ornith-1.0-35B-GGUF-llamacpp-tp1-Q5_K_M
response_model: ornith-1.0-35b-Q5_K_M.gguf
finish_reason: stop
content: "4"
completion_tokens: 2
separation_note: The host Pi default is Laguna S 2.1 through IOP Edge. Keep Laguna, Qwen, Ornith, and dev-corp Gemma parser/template/runtime settings separate.
content_completion_policy:
enforce_min_max_tokens: true
reason: prevent reasoning-only completions from exhausting caller max_tokens before final content
@ -1219,6 +798,11 @@ retained_test_runtimes:
- keep_it_out_of_the_iop_provider_pool_unless_explicitly_registered
- verify_model_endpoint_and_slot_profile_after_start
agent_inventory:
inventory_id: inventory-agent
path: agent-test/inventory-agent.yaml
environment: dev
nodes:
- id: mac-codex-node
alias: mac-codex

8
agent-test/inventory.yaml
View file

@ -1,6 +1,6 @@
inventory_id: inventory
schema_version: 1
last_updated_at: "2026-07-24"
last_updated_at: "2026-08-02"
scope:
type: shared_manifest
@ -16,6 +16,11 @@ environments:
path: agent-test/inventory-dev-corp.yaml
selection: explicit_only
agent_inventory:
inventory_id: inventory-agent
path: agent-test/inventory-agent.yaml
scope: shared_agent_host_profiles
ownership:
common:
- inventory document routing
@ -39,5 +44,6 @@ policy:
token_api_key_and_private_credential_allowed: false
source_of_truth:
common: agent-test/inventory.yaml
agents: agent-test/inventory-agent.yaml
dev: agent-test/inventory-dev.yaml
dev-corp: agent-test/inventory-dev-corp.yaml