From 5aece17895e143caf7265a3d7861fb5ef3dd592a Mon Sep 17 00:00:00 2001 From: toki Date: Thu, 9 Jul 2026 09:30:16 +0900 Subject: [PATCH] =?UTF-8?q?docs(dev-corp):=20Pi=20=ED=94=84=EB=A1=9C?= =?UTF-8?q?=ED=95=84=20=EB=B6=84=EB=A6=AC=EB=A5=BC=20=EA=B8=B0=EB=A1=9D?= =?UTF-8?q?=ED=95=9C=EB=8B=A4?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../project/dev-corp-runtime-deploy/SKILL.md | 2 +- agent-test/dev-corp/edge-smoke.md | 1 + agent-test/dev-corp/inventory.yaml | 127 +++++++++++++++--- 3 files changed, 114 insertions(+), 16 deletions(-) diff --git a/agent-ops/skills/project/dev-corp-runtime-deploy/SKILL.md b/agent-ops/skills/project/dev-corp-runtime-deploy/SKILL.md index 846af86..eb0c9cf 100644 --- a/agent-ops/skills/project/dev-corp-runtime-deploy/SKILL.md +++ b/agent-ops/skills/project/dev-corp-runtime-deploy/SKILL.md @@ -52,7 +52,7 @@ dev-corp provider pool을 현재 dev-runtime과 같은 구조로 배포한다. - DGX Spark 02 Docker image `vllm/vllm-openai:v0.24.0`의 entrypoint는 이미 `vllm serve`이다. container command에는 `serve`를 중복으로 넣지 말고 positional model `/models/gemma-4-26B-A4B-it-NVFP4`부터 둔다. 현재 Docker publish는 host `8004` -> container `8004` 기준이다. - Gemma4 tool-call/parser profile은 capacity/KV 튜닝과 별개로 agent 동작 가능성을 결정한다. DGX Spark 01은 `/home/digitalcommerce_dgx_spark_01/tool_chat_template_gemma4.jinja`, DGX Spark 02는 `/vllm-workspace/examples/tool_chat_template_gemma4.jinja`를 `--chat-template`로 사용하고, 둘 다 `--enforce-eager --skip-mm-profiling --limit-mm-per-prompt '{"image": 0, "video": 0}' --mm-processor-cache-gb 0 --enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4 --default-chat-template-kwargs '{"enable_thinking":true}'`를 둔다. Mac Studio vLLM-MLX는 별도 chat template override 없이 `--continuous-batching --disable-prefix-cache --enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4 --default-chat-template-kwargs '{"enable_thinking":true}'`를 둔다. - 모델별 parser/template 값은 섞지 않는다. Gemma 계열은 `tool_call_parser=gemma4`, `reasoning_parser=gemma4`, Gemma4 tool chat template 또는 vLLM-MLX 기본 template kwargs를 사용한다. Qwen 계열은 dev-runtime 문서의 `tool_call_parser=qwen`, `reasoning_parser=qwen3`, Qwen 전용 chat template 검증값을 따르며 Gemma4 parser/template을 재사용하지 않는다. -- Pi agent 기본 경로는 Mac Studio vLLM-MLX provider인 `dev-corp-direct` / `mlx-community/gemma-4-26b-a4b-it-nvfp4` / thinking `high`로 저장한다. Pi OpenAI-compatible compat는 `supportsStrictMode=true`, `thinkingFormat=chat-template`, `chatTemplateKwargs.enable_thinking=thinking.enabled` 기준이며, DGX Spark 01 vLLM은 `dev-corp-spark01` fallback으로 둔다. +- Pi agent dev-corp profile은 `agent-test/dev-corp/inventory.yaml`의 `model.pi_agent_profile`을 기준으로 한다. 2026-07-09 현재 host-local Pi 기본 경로는 Edge OpenAI-compatible alias route인 `dev-corp` / `gemma4:26b` / thinking `high`이고, base URL은 `http://172.24.63.178:18086/v1`이다. Mac Studio direct provider `dev-corp-direct` / `mlx-community/gemma-4-26b-a4b-it-nvfp4`와 DGX Spark 01 direct provider `dev-corp-spark01` / `gemma-4-26B-A4B-it-NVFP4`는 non-default direct/fallback profile로 둔다. - Mac Studio는 `DYLD_LIBRARY_PATH=/opt/homebrew/opt/expat/lib` 없이 Python `pyexpat` import가 실패할 수 있다. `screen` session `vllm_mlx_8004` 안에서 이 환경변수를 export한 뒤 시작한다. - 2026-07-08 재확인 기준 Mac Studio vLLM-MLX는 `/Users/dc_dev/vllm-env-0.4.0/bin/python` 환경의 `vllm-mlx=0.4.0`, `mlx=0.32.0`, `mlx-lm=0.31.3`, `mlx-vlm=0.6.4`, `transformers=5.12.1`, `huggingface-hub=1.22.0`로 동작한다. `--continuous-batching`은 유지하고 `--disable-prefix-cache`를 추가했으며, `/health`, `/v1/models`, 직접 API non-stream/stream auto tool-call, streaming multi-turn tool-result final answer, Pi `-p` tool-call, Pi TUI 초기 프롬프트 tool-call 로그 종료를 확인했다. - Mac Studio vLLM-MLX의 현 값은 "구동 가능한 최종 체크포인트"로 취급한다. 추가 미세 조정은 새 포트/새 venv 또는 명시적 rollback 지점을 둔 별도 실험으로만 수행하고, Spark 01/02 vLLM 설정이나 Pi 기본 provider/model을 함께 변경하지 않는다. 알려진 잔여 이슈는 Gemma4 thought channel delimiter가 최종 content에 누수될 수 있는 점이며, 이 경우 serving 옵션을 계속 흔들기보다 Pi extension/provider adapter에서 vLLM-MLX Gemma 전용 sanitizer로 containment 하는 방향을 우선 검토한다. diff --git a/agent-test/dev-corp/edge-smoke.md b/agent-test/dev-corp/edge-smoke.md index 72114c7..ffcf10b 100644 --- a/agent-test/dev-corp/edge-smoke.md +++ b/agent-test/dev-corp/edge-smoke.md @@ -56,6 +56,7 @@ dev-corp provider pool과 3-node 연결 상태를 점검할 때는 `agent-test/d - Edge-Node TCP transport 기본 후보: `115.21.224.82:18087` - node-only/internal 후보: `172.24.63.178:18085/18086/18087`은 기본 Edge 배포 후보가 아니다. 사용자가 172 Edge 경로를 명시 요청했거나 host별 reverse SSH tunnel이 필요한 경우에만 사용한다. - model alias 후보: `gemma4:26b` +- host Pi dev-corp profile: `agent-test/dev-corp/inventory.yaml`의 `model.pi_agent_profile` 기준. 2026-07-09 현재 이 host의 Pi 기본 provider/model/thinking level은 `dev-corp` / `gemma4:26b` / `high`이며, base URL은 host-local node-internal Edge route `http://172.24.63.178:18086/v1`이다. 기본 public smoke/deploy route는 계속 `http://115.21.224.82:18086/v1`이고, dev `qwen3.6:35b` Pi profile은 `agent-test/dev/inventory.yaml`에서 별도로 관리한다. 노드 후보: diff --git a/agent-test/dev-corp/inventory.yaml b/agent-test/dev-corp/inventory.yaml index 964c03f..80e1346 100644 --- a/agent-test/dev-corp/inventory.yaml +++ b/agent-test/dev-corp/inventory.yaml @@ -1,6 +1,6 @@ test_env: dev-corp profile: dev-corp-provider-pool -last_updated_at: "2026-07-08" +last_updated_at: "2026-07-09" source: remote_runner: @@ -67,20 +67,117 @@ model: strict_output_behavior: provider_pool_catalog_default_overrides_strict_disable adapter_mapping: vllm_and_vllm_mlx_use_chat_template_kwargs_enable_thinking_true pi_agent_profile: - current_default_provider: dev-corp-direct - current_default_served_model: mlx-community/gemma-4-26b-a4b-it-nvfp4 - current_default_runtime: mac_studio_vllm_mlx - thinking_level: high - compat: - supports_strict_mode: true - thinking_format: chat-template - chat_template_kwargs: - enable_thinking: thinking.enabled - fallback_provider: - provider: dev-corp-spark01 - served_model: gemma-4-26B-A4B-it-NVFP4 - runtime: dgx_spark_01_vllm - note: Pi local settings should use the same model-specific parser/template policy as the provider runtime; do not copy Gemma parser/template values to Qwen providers. + observed_at: "2026-07-09" + config_scope: host_local_pi_agent + config_dir: ~/.pi/agent + install: + package: pi-coding-agent + version: 0.80.3 + binary: /opt/homebrew/bin/pi + settings_file: ~/.pi/agent/settings.json + models_file: ~/.pi/agent/models.json + profile_namespace: dev_corp_gemma4_pi + current_default_provider: dev-corp + current_default_model: gemma4:26b + current_default_runtime: edge_openai_compatible_provider_pool + current_default_base_url: http://172.24.63.178:18086/v1 + current_default_api: openai-completions + current_default_auth_header: true + current_default_thinking_level: high + hide_thinking_block: false + http_idle_timeout_ms: 300000 + retry_provider_timeout_ms: 300000 + retry_provider_max_retries: 0 + retry_provider_max_retry_delay_ms: 60000 + enabled_model_patterns: + - dev-corp-direct/** + - dev-corp-spark01/** + - dev-corp/* + api_key_value_tracked: false + route_policy: current host-local Pi default uses the 172.24.63.178 node-internal Edge route because this host can reach the mac-mini directly; this does not replace the default dev-corp public smoke/deploy route 115.21.224.82 + pi_model_parameters: + provider: dev-corp + model_id: gemma4:26b + display_name: dev-corp gemma4:26b + api: openai-completions + context_window: 262144 + max_tokens: 32768 + reasoning: true + thinking_level: high + expected_iop_thinking_token_budget: 1024 + compat: + supports_developer_role: false + supports_usage_in_streaming: false + max_tokens_field: max_tokens + supports_reasoning_effort: false + supports_strict_mode: false + supports_store: false + requires_tool_result_name: true + requires_assistant_after_tool_result: true + direct_provider_profiles: + mac_studio_vllm_mlx: + provider: dev-corp-direct + default_provider: false + base_url: http://172.24.63.178:8004/v1 + served_model: mlx-community/gemma-4-26b-a4b-it-nvfp4 + runtime: mac_studio_vllm_mlx + compat: + supports_strict_mode: true + thinking_format: chat-template + chat_template_kwargs: + enable_thinking: thinking.enabled + dgx_spark_01_vllm: + provider: dev-corp-spark01 + default_provider: false + base_url: http://127.0.0.1:8002/v1 + served_model: gemma-4-26B-A4B-it-NVFP4 + runtime: dgx_spark_01_vllm + route_note: host-local direct or tunnel route only; keep as fallback and do not treat it as the Edge alias route + compat: + supports_strict_mode: true + thinking_format: chat-template + chat_template_kwargs: + enable_thinking: thinking.enabled + runtime_parameter_alignment: + alignment_policy: Pi model parameters must be kept in sync with the dev-corp Edge model alias, provider served_model values, context/output limits, Gemma parser/template policy, and provider capacity/sequence limits. Do not optimize or debug Pi behavior from the endpoint URL alone. + common_requirements: + model_alias: gemma4:26b + pi_context_window: 262144 + edge_context_window: 262144 + pi_max_tokens: 32768 + provider_default_max_tokens: 32768 + pi_reasoning: true + edge_reasoning_policy: bounded_thinking + edge_default_thinking_token_budget: 1024 + required_template_policy: Gemma providers keep default_chat_template_kwargs.enable_thinking=true + required_tool_call_parser: gemma4 + required_reasoning_parser: gemma4 + engine_matrix: + dgx_spark_01_vllm: + runtime_type: vllm + served_model: gemma-4-26B-A4B-it-NVFP4 + context_match: pi contextWindow 262144 maps to vLLM max_model_len 262144 + output_budget_match: pi maxTokens 32768 maps to provider default max tokens 32768 + thinking_match: pi reasoning=true and thinking_level=high map to reasoning_parser=gemma4, default_chat_template_kwargs.enable_thinking=true, and IOP thinking token budget 1024 + concurrency_match: provider capacity 4 maps to runtime max_num_seqs 4 + tool_parser_profile: gemma4 + dgx_spark_02_vllm: + runtime_type: vllm + served_model: gemma-4-26B-A4B-it-NVFP4 + context_match: pi contextWindow 262144 maps to vLLM max_model_len 262144 + output_budget_match: pi maxTokens 32768 maps to provider default max tokens 32768 + thinking_match: pi reasoning=true and thinking_level=high map to reasoning_parser=gemma4, default_chat_template_kwargs.enable_thinking=true, and IOP thinking token budget 1024 + concurrency_match: provider capacity 4 maps to runtime max_num_seqs 4 + tool_parser_profile: gemma4 + mac_studio_vllm_mlx: + runtime_type: vllm-mlx + served_model: mlx-community/gemma-4-26b-a4b-it-nvfp4 + context_match: pi contextWindow 262144 maps to vLLM-MLX max_request_tokens 262144 and max_kv_size 786432 + output_budget_match: pi maxTokens 32768 maps to provider default max tokens 32768 + thinking_match: pi reasoning=true and thinking_level=high map to reasoning_parser=gemma4, default_chat_template_kwargs.enable_thinking=true, and IOP thinking token budget 1024 + concurrency_match: provider catalog capacity 5 maps to runtime headroom max_num_seqs 6 + tool_parser_profile: gemma4 + separation_note: This profile is for dev-corp Gemma4 Pi only. Keep dev Qwen3.6 Pi settings under agent-test/dev/inventory.yaml and do not copy Qwen parser/template/runtime assumptions into this Gemma profile or Gemma parser/template values into Qwen providers. timeout_policy: openai_timeout_sec: 1800 provider_queue_timeout_ms: 0