<!-- task=openai_reasoning_estimated_usage/02+01_dev_corp_rollout plan=1 tag=REVIEW_API -->

# Plan - REVIEW_API

## 이 파일을 읽는 구현 에이전트에게

이 follow-up의 현재 테스트 기준은 `dev` 환경이다. active task directory 이름은 이전 라우팅 이름을 유지하지만, 검증 대상은 `agent-test/dev/**`와 `agent-ops/skills/project/dev-runtime-deploy/SKILL.md`의 dev-runtime 기준이다. `iop.ai.kr`, `digitalplatform.iop.ai.kr`, `fe@172.24.63.178`, dev-corp 포트 `18086/18087/19112`에는 접근하지 않는다.

사용자에게 직접 묻지 말고, dev runner나 dev runtime이 닫혀 있으면 실제 명령과 출력으로 blocker를 기록한 뒤 active `CODE_REVIEW-cloud-G07.md`를 채워 review ready로 멈춘다. token/secret 원문은 stdout/stderr, docs, task artifact에 남기지 않는다.

## 배경

core predecessor는 estimated reasoning token metric을 source/test/docs/spec에 반영하고 PASS로 archive되었다. loop 0은 잘못된 환경 기준으로 작성되었으므로, 현재 follow-up은 dev-runtime Edge에 새 metric을 배포하고 dev Prometheus/Grafana 관측 기준에서 OpenAI-compatible usage accounting을 확인한다.

## Archive Evidence Snapshot

- archived loop:
  - plan: `agent-task/openai_reasoning_estimated_usage/02+01_dev_corp_rollout/plan_cloud_G07_0.log`
  - review: `agent-task/openai_reasoning_estimated_usage/02+01_dev_corp_rollout/code_review_cloud_G07_0.log`
  - verdict: FAIL
- Superseded context:
  - loop 0의 환경/host 기준은 현재 사용자 정정으로 폐기한다.
  - archive의 public-host/runner 명령은 재사용하지 않는다.
- Current dev criteria:
  - environment docs: `agent-test/dev/inventory.yaml`, `agent-test/dev/rules.md`, `agent-test/dev/edge-smoke.md`, `agent-test/dev/platform-common-smoke.md`
  - deploy skill: `agent-ops/skills/project/dev-runtime-deploy/SKILL.md`
  - runner: `ssh toki@toki-labs.com`
  - runner repo root: `/Users/toki/agent-work/iop-dev`
  - Edge config: `build/dev-runtime/edge.yaml`
  - Edge binary: `build/dev-runtime/bin/edge`
  - Edge id: `edge-toki-labs-dev`
  - OpenAI-compatible base URL: runner-local `http://127.0.0.1:18083/v1`, public dev `http://toki-labs.com:18083/v1`
  - Edge metrics: `127.0.0.1:19101`
  - Prometheus: `127.0.0.1:19111`
  - model alias: `ornith:35b`
- Required summary:
  - R1: dev runner checkout must be synced to the source state that contains `iop_openai_reasoning_estimated_tokens_total`, then `go test` and dev Edge rebuild must run on the dev runner.
  - R2: dev-runtime Edge must be restarted from the rebuilt binary and direct dev metrics must expose `iop_openai_reasoning_estimated_tokens_total`.
  - R3: dev OpenAI-compatible smoke and Prometheus queries must show request/accounting evidence without leaking token values.
- Verification evidence from local review:
  - `GOCACHE=/tmp/iop-go-cache go test ./apps/edge/internal/openai` passed locally.
  - `git diff --check -- agent-task/openai_reasoning_estimated_usage` passed with no output.
- Narrow archive reread allowed if needed:
  - `agent-task/openai_reasoning_estimated_usage/02+01_dev_corp_rollout/plan_cloud_G07_0.log`
  - `agent-task/openai_reasoning_estimated_usage/02+01_dev_corp_rollout/code_review_cloud_G07_0.log`
  - `agent-task/archive/2026/07/openai_reasoning_estimated_usage/01_metric_core/complete.log`
- Roadmap carryover: 없음. 이 task는 non-roadmap rollout task이며 `Roadmap Targets`가 없다.

## 사용자 리뷰 요청 흐름

구현 중 직접 사용자에게 묻지 않는다. 이 task는 non-roadmap rollout task이므로 runner, secret, host readiness, Prometheus 접근 문제는 사용자 리뷰가 아니라 검증 blocker 또는 follow-up evidence로 기록한다. 선택된 Milestone lock 결정이 아닌 blocker로 `USER_REVIEW.md`를 만들지 않는다.

## 구현 체크리스트

- [ ] predecessor PASS를 `agent-task/archive/2026/07/openai_reasoning_estimated_usage/01_metric_core/complete.log`로 재확인한다.
- [ ] dev 기준 문서(`agent-test/dev/inventory.yaml`, `agent-test/dev/rules.md`, `agent-test/dev/edge-smoke.md`)의 runner/path/port/model 값을 사용하고, 폐기된 환경 host/port에 접근하지 않는다.
- [ ] dev runner, Prometheus, Edge process/ports, current direct metric exposure preflight를 실행하고 실제 stdout/stderr를 기록한다.
- [ ] dev runner가 reachable이면 `/Users/toki/agent-work/iop-dev` checkout을 배포 기준 source state로 clean sync하고 `GOCACHE=/tmp/iop-go-cache go test -count=1 ./apps/edge/internal/openai`와 dev Edge rebuild를 실행한다.
- [ ] rebuilt `build/dev-runtime/bin/edge`로 dev-runtime Edge를 restart한 뒤 `/healthz`, `/v1/models`, `/metrics`에서 새 metric 노출을 확인한다.
- [ ] token 원문을 출력하지 않고 dev `ornith:35b` OpenAI-compatible smoke를 실행한 뒤 Prometheus에서 estimated reasoning metric과 request counter를 확인한다.
- [ ] 8개(capacity 7 + 1) concurrent `/v1/chat/completions` smoke를 실행하고 request success count, estimated reasoning metric, provider `in_flight/queued` 관측과 완료 후 회복을 기록한다.
- [ ] CODE_REVIEW-*-G??.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채운다. 이 항목이 완료되기 전에는 구현이 완료된 것이 아니다.

### [REVIEW_API-1] dev runner source sync and Edge rebuild

#### 문제

dev-runtime Edge binary가 새 estimated reasoning metric source state로 rebuild되지 않으면 dev Prometheus에서 metric/accounting을 확인할 수 없다.

#### 해결 방법

dev runner와 Prometheus readiness를 확인한다. runner가 열려 있으면 source 기준 ref를 확정해 clean sync하고 targeted package test와 Edge build를 실행한다. runner가 닫혀 있으면 deploy/smoke를 진행하지 말고 blocker evidence를 남긴다.

```bash
test -f agent-task/archive/2026/07/openai_reasoning_estimated_usage/01_metric_core/complete.log
ssh -o BatchMode=yes -o ConnectTimeout=8 toki@toki-labs.com 'cd /Users/toki/agent-work/iop-dev && git status --short --branch && git rev-parse HEAD && (/opt/homebrew/bin/go version || go version)'
ssh -o BatchMode=yes -o ConnectTimeout=8 toki@toki-labs.com 'curl -fsS http://127.0.0.1:19111/-/ready'
ssh toki@toki-labs.com 'cd /Users/toki/agent-work/iop-dev && git fetch origin main && git reset --hard origin/main && git clean -fd'
rsync -a --delete --exclude .git --exclude build/dev-runtime --exclude agent-task/archive ./ toki@toki-labs.com:/Users/toki/agent-work/iop-dev/
ssh toki@toki-labs.com 'cd /Users/toki/agent-work/iop-dev && GOCACHE=/tmp/iop-go-cache go test -count=1 ./apps/edge/internal/openai && GOCACHE=/tmp/iop-go-cache go build -trimpath -o build/dev-runtime/bin/edge ./apps/edge/cmd/edge'
```

If the deploy source is a pushed ref instead of local rsync, record the exact ref and omit the rsync step. Do not deploy from a dirty or stale runner checkout.

#### 중간 검증

```bash
ssh toki@toki-labs.com 'cd /Users/toki/agent-work/iop-dev && test -x build/dev-runtime/bin/edge && build/dev-runtime/bin/edge --help >/tmp/iop-dev-edge-help.out && wc -c /tmp/iop-dev-edge-help.out'
```

Expected: rebuilt Edge binary exists and runs.

### [REVIEW_API-2] dev-runtime Edge restart and metric exposure

#### 문제

Source test success alone does not prove the running dev Edge exposes the new metric.

#### 해결 방법

Use the rebuilt dev binary and `build/dev-runtime/edge.yaml`. Restart the dev-runtime Edge according to the current process manager, then verify ports and direct metrics. If the actual process manager differs from the command below, record the real command and `pgrep -af` evidence.

```bash
ssh toki@toki-labs.com 'cd /Users/toki/agent-work/iop-dev && test -f build/dev-runtime/edge.yaml && pgrep -af "build/dev-runtime/bin/edge.*serve|edge.*serve" || true && lsof -nP -iTCP:18083 -sTCP:LISTEN || true && lsof -nP -iTCP:19101 -sTCP:LISTEN || true'
ssh toki@toki-labs.com 'cd /Users/toki/agent-work/iop-dev && pkill -f "build/dev-runtime/bin/edge.*serve|edge.*serve" || true; nohup build/dev-runtime/bin/edge serve --config build/dev-runtime/edge.yaml > build/dev-runtime/logs/edge-rollout-$(date +%Y%m%d%H%M%S).log 2>&1 &'
ssh toki@toki-labs.com 'curl -fsS http://127.0.0.1:18083/healthz && curl -fsS http://127.0.0.1:18083/v1/models >/tmp/iop-dev-models-after-rollout.json && curl -fsS http://127.0.0.1:19101/metrics | grep -F "iop_openai_reasoning_estimated_tokens_total"'
```

#### 중간 검증

```bash
ssh toki@toki-labs.com 'curl -fsS http://127.0.0.1:19101/metrics | grep -F "iop_openai_reasoning_estimated_tokens_total"'
```

Expected: metric family appears in direct dev Edge metrics.

### [REVIEW_API-3] dev OpenAI-compatible smoke and Prometheus accounting

#### 문제

Metric family exposure alone does not prove request accounting. dev 기준에서는 OpenAI-compatible route와 Prometheus `19111`에서 request/estimated metric evidence를 확인해야 한다.

#### 해결 방법

Use `OPENAI_API_KEY` or `--api-key-file` only from the dev runner environment; never print the value. Run `iop-edge smoke openai` for `/healthz`, `/v1/models`, `/v1/responses`, then run Chat Completions single/concurrent smoke. Query direct metrics and Prometheus after scrape. Capacity smoke should use 8 concurrent requests because the current Ornith provider capacity is `7`.

```bash
ssh toki@toki-labs.com 'cd /Users/toki/agent-work/iop-dev && OPENAI_API_KEY="${OPENAI_API_KEY:-}" build/dev-runtime/bin/edge smoke openai --model ornith:35b --base-url http://127.0.0.1:18083 --prompt "2+2? Answer only the number." --timeout 60s'
ssh toki@toki-labs.com 'cd /Users/toki/agent-work/iop-dev && : "${OPENAI_API_KEY:?OPENAI_API_KEY must be set in the remote environment}"; curl -fsS http://127.0.0.1:18083/v1/chat/completions -H "Authorization: Bearer ${OPENAI_API_KEY}" -H "Content-Type: application/json" -d "{\"model\":\"ornith:35b\",\"messages\":[{\"role\":\"user\",\"content\":\"짧게 생각 과정을 내부적으로 사용해서 2+2 답만 말해.\"}],\"max_tokens\":512,\"include_reasoning\":true}" >/tmp/iop-dev-reasoning-single.json'
ssh toki@toki-labs.com 'rm -f /tmp/iop-dev-reasoning-concurrent-*.json; : "${OPENAI_API_KEY:?OPENAI_API_KEY must be set in the remote environment}"; seq 1 8 | xargs -P8 -I{} sh -c '"'"'curl -fsS http://127.0.0.1:18083/v1/chat/completions -H "Authorization: Bearer '"'"'"${OPENAI_API_KEY}"'"'"'" -H "Content-Type: application/json" -d "{\"model\":\"ornith:35b\",\"messages\":[{\"role\":\"user\",\"content\":\"요청 {}: 700 토큰 안팎의 구조화된 답변을 작성해.\"}],\"max_tokens\":1200,\"include_reasoning\":true}" > /tmp/iop-dev-reasoning-concurrent-{}.json'"'"''
ssh toki@toki-labs.com 'test "$(ls /tmp/iop-dev-reasoning-concurrent-*.json | wc -l | tr -d " ")" -ge 8'
ssh toki@toki-labs.com 'curl -fsG http://127.0.0.1:19111/api/v1/query --data-urlencode "query=sum by (token_ref,status) (iop_openai_requests_total)"'
ssh toki@toki-labs.com 'curl -fsG http://127.0.0.1:19111/api/v1/query --data-urlencode "query=sum by (token_ref,estimation_method) (iop_openai_reasoning_estimated_tokens_total)"'
```

Also capture provider capacity snapshots from the dev Control Plane status if available:

```bash
ssh toki@toki-labs.com 'curl -fsS http://127.0.0.1:18001/edges/edge-toki-labs-dev/status >/tmp/iop-dev-edge-status-after-smoke.json'
```

#### 중간 검증

```bash
ssh toki@toki-labs.com 'test "$(ls /tmp/iop-dev-reasoning-concurrent-*.json | wc -l | tr -d " ")" -ge 8'
```

Expected: at least 8 response files exist and all curl commands succeeded; provider capacity evidence shows total in-flight/queue behavior during the run when status polling is available.

## 수정 파일 요약

| 파일 | 항목 |
|------|------|
| none | REVIEW_API-1, REVIEW_API-2, REVIEW_API-3 |

## 최종 검증

```bash
test -f agent-task/archive/2026/07/openai_reasoning_estimated_usage/01_metric_core/complete.log
ssh toki@toki-labs.com 'cd /Users/toki/agent-work/iop-dev && git status --short --branch && git rev-parse HEAD && (/opt/homebrew/bin/go version || go version)'
ssh toki@toki-labs.com 'curl -fsS http://127.0.0.1:19111/-/ready'
ssh toki@toki-labs.com 'cd /Users/toki/agent-work/iop-dev && test -f build/dev-runtime/edge.yaml && pgrep -af "build/dev-runtime/bin/edge.*serve|edge.*serve" && lsof -nP -iTCP:18083 -sTCP:LISTEN && lsof -nP -iTCP:19101 -sTCP:LISTEN'
ssh toki@toki-labs.com 'curl -fsS http://127.0.0.1:18083/healthz && curl -fsS http://127.0.0.1:18083/v1/models >/tmp/iop-dev-models-final.json && curl -fsS http://127.0.0.1:19101/metrics | grep -F "iop_openai_reasoning_estimated_tokens_total"'
ssh toki@toki-labs.com 'curl -fsG http://127.0.0.1:19111/api/v1/query --data-urlencode "query=sum by (token_ref,estimation_method) (iop_openai_reasoning_estimated_tokens_total)"'
ssh toki@toki-labs.com 'curl -fsG http://127.0.0.1:19111/api/v1/query --data-urlencode "query=sum by (token_ref,status) (iop_openai_requests_total)"'
git diff --check -- agent-task/openai_reasoning_estimated_usage
```

Expected: predecessor PASS evidence exists, dev runner and Prometheus are reachable, dev Edge exposes the new metric, single/concurrent smoke succeeds, Prometheus shows request and estimated reasoning evidence, and task artifact diff has no whitespace errors.

모든 작업 후 반드시 `CODE_REVIEW-*-G??.md`의 구현 에이전트 소유 섹션을 채운다. 이 파일 작성이 구현의 마지막 단계다.
