diff --git a/agent-roadmap/phase/routing-policy-model-orchestration/PHASE.md b/agent-roadmap/phase/routing-policy-model-orchestration/PHASE.md index 411c583..02fc68f 100644 --- a/agent-roadmap/phase/routing-policy-model-orchestration/PHASE.md +++ b/agent-roadmap/phase/routing-policy-model-orchestration/PHASE.md @@ -16,7 +16,7 @@ IOP의 OpenAI-compatible, A2A, IOP native 입력 표면에서 들어온 요청 완료, 검토중, 진행중, 계획, 스케치 순서로 두어 아래로 갈수록 미래 작업에 가까워지게 정렬한다. 스케치 Milestone은 아직 구현 가능한 계획이 아니므로 계획 Milestone보다 아래에 둔다. -- [계획] OpenAI-compatible Raw Tunnel과 Sideband Passthrough +- [진행중] OpenAI-compatible Raw Tunnel과 Sideband Passthrough - 경로: [openai-compatible-raw-tunnel-sideband-passthrough](milestones/openai-compatible-raw-tunnel-sideband-passthrough.md) - 요약: OpenAI-compatible provider 응답을 기존 Edge-Node proto-socket 위 lossless raw tunnel로 전달하고, 기본값은 provider-original `passthrough`로 두며, 요청이 명시한 경우에만 `passthrough+sideband` 또는 `transformed`를 사용한다. diff --git a/agent-roadmap/phase/routing-policy-model-orchestration/milestones/openai-compatible-raw-tunnel-sideband-passthrough.md b/agent-roadmap/phase/routing-policy-model-orchestration/milestones/openai-compatible-raw-tunnel-sideband-passthrough.md index a1636ff..c3fc83c 100644 --- a/agent-roadmap/phase/routing-policy-model-orchestration/milestones/openai-compatible-raw-tunnel-sideband-passthrough.md +++ b/agent-roadmap/phase/routing-policy-model-orchestration/milestones/openai-compatible-raw-tunnel-sideband-passthrough.md @@ -13,7 +13,7 @@ OpenAI-compatible 추론 엔진(vLLM, SGLang, Lemonade, OpenAI-compatible API pr ## 상태 -[계획] +[진행중] ## 승격 조건 @@ -52,9 +52,9 @@ OpenAI-compatible 추론 엔진(vLLM, SGLang, Lemonade, OpenAI-compatible API pr OpenAI-compatible provider의 원본 응답을 정규화/재조립하지 않고 기존 Edge-Node socket으로 전달하는 핵심 경로를 묶는다. -- [ ] [wire-contract] Edge-Node wire에 tunnel request/frame 메시지를 추가하고 status, headers, body bytes, sequence, end, error, usage trailer 후보를 정의한다. 검증: proto 생성과 contract test에서 raw body가 `RunEvent.delta`나 `events.Bus`를 통하지 않음을 확인한다. -- [ ] [node-relay] Node는 OpenAI-compatible provider HTTP/SSE response를 열고 provider가 보낸 chunk를 tunnel frame으로 relay한다. 검증: vLLM Qwen `reasoning_content` fixture와 Gemma content fixture가 byte comparison으로 동일하게 왕복한다. -- [ ] [edge-stream] Edge OpenAI-compatible handler는 tunnel status/headers/body chunks를 caller에게 그대로 write하고 pure passthrough를 기본값으로 적용한다. 검증: streaming/non-streaming Chat Completions fixture가 provider 원본과 byte-identical하게 반환된다. +- [x] [wire-contract] Edge-Node wire에 tunnel request/frame 메시지를 추가하고 status, headers, body bytes, sequence, end, error, usage trailer 후보를 정의한다. 검증: proto 생성과 contract test에서 raw body가 `RunEvent.delta`나 `events.Bus`를 통하지 않음을 확인한다. +- [x] [node-relay] Node는 OpenAI-compatible provider HTTP/SSE response를 열고 provider가 보낸 chunk를 tunnel frame으로 relay한다. 검증: vLLM Qwen `reasoning_content` fixture와 Gemma content fixture가 byte comparison으로 동일하게 왕복한다. +- [x] [edge-stream] Edge OpenAI-compatible handler는 tunnel status/headers/body chunks를 caller에게 그대로 write하고 pure passthrough를 기본값으로 적용한다. 검증: streaming/non-streaming Chat Completions fixture가 provider 원본과 byte-identical하게 반환된다. ### Epic: [response-modes] Request-selected Modes diff --git a/agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics/code_review_cloud_G06_0.log b/agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics/code_review_cloud_G06_0.log new file mode 100644 index 0000000..b053c38 --- /dev/null +++ b/agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics/code_review_cloud_G06_0.log @@ -0,0 +1,197 @@ + + +# Code Review Reference - API + +> **[IMPLEMENTING AGENT — READ FIRST] Filling in this file is the mandatory final step of implementation.** +> The task is NOT complete until every implementation-owned section below is filled in. +> Complete the `구현 체크리스트`; the final checklist item is mandatory before saving. +> Fill implementation-owned sections, then stop with active files in place and report ready for review. +> If implementation is blocked by a selected Milestone `구현 잠금 > 결정 필요` item, fill `사용자 리뷰 요청` with linked evidence and stop with active files in place; code-review decides whether to write `USER_REVIEW.md`. Environment/secret/service blockers, generic scope changes, repeated failures, and evidence gaps that a follow-up agent can close are normal follow-up issues, not user-review blockers by themselves. +> Do not ask the user directly, present choices in chat, or call `request_user_input` during implementation; record only Milestone lock decisions in `사용자 리뷰 요청` and stop for code-review. +> Finalization (`코드리뷰 결과`, log rename, `complete.log`, archive moves, `코드리뷰 전용 체크리스트`) is review-agent-only, even after compaction/resume. +> Follow the ownership table at the bottom of this file for which sections you own. + +## 개요 + +date=2026-07-08 +task=m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics, plan=0, tag=API + +## Roadmap Targets + +- Milestone: `agent-roadmap/phase/routing-policy-model-orchestration/milestones/openai-compatible-raw-tunnel-sideband-passthrough.md` +- Milestone link: [Milestone 문서](agent-roadmap/phase/routing-policy-model-orchestration/milestones/openai-compatible-raw-tunnel-sideband-passthrough.md) +- Task ids: + - `mode-contract`: OpenAI-compatible request metadata/config 계약에 `metadata.iop_response_mode`와 `passthrough`, `passthrough+sideband`, `transformed` 모드를 정의하고 omitted mode는 `passthrough`로 해석한다. + - `sideband-overlay`: `passthrough+sideband`는 provider original stream과 IOP sideband를 명시적 IOP extension stream/schema로 함께 노출하되 pure `passthrough` body에는 IOP field/event를 주입하지 않는다. + - `transform-output`: `transformed`는 provider raw stream을 IOP schema/output으로 수정할 수 있지만 byte-identical provider stream으로 표시하지 않는다. +- Completion mode: check-on-pass + +## Archive Evidence Snapshot + +- `agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/01_node_relay/complete.log`: PASS. Node relay가 OpenAI-compatible provider response를 기존 Edge-Node socket tunnel frame으로 전달하는 선행 작업을 완료했다. +- `agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/02+01_edge_stream/complete.log`: PASS. Edge stream pure passthrough 기본값과 streaming/non-streaming byte identity 경로가 완료되어 이번 subtask의 predecessor `02`를 만족한다. +- 더 자세한 선행 맥락이 필요할 때만 위 두 `complete.log`를 좁게 읽는다. `agent-task/archive/**`를 광범위하게 검색하지 않는다. + +## 이 파일을 읽는 리뷰 에이전트에게 + +> **[REVIEW AGENT ONLY]** 아래 종결 절차는 코드리뷰 에이전트 전용이다. 구현 에이전트는 이 섹션을 실행하지 않는다. + +각 항목의 구현을 실제 소스 파일과 대조하고, `검증 결과` 섹션의 출력이 코드와 일치하는지 확인하세요. +리뷰 완료는 아래 순서까지 끝난 상태를 의미합니다. + +1. 판정을 append한다. +2. `CODE_REVIEW-cloud-G06.md` -> `code_review_cloud_G06_N.log`, `PLAN-cloud-G06.md` -> `plan_cloud_G06_M.log`로 아카이브한다. +3. PASS이면 `complete.log` 작성 후 active task 디렉터리를 `agent-task/archive/YYYY/MM/m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics/`로 이동한다. WARN/FAIL이면 user-review gate를 확인한 뒤 다음 active plan/review 파일 또는 `USER_REVIEW.md`를 작성한다. `USER_REVIEW.md`가 연결된 Milestone 결정으로 완료/PASS 해소되면 code-review가 `USER_REVIEW.md`를 해소 상태로 갱신하고 `complete.log` 작성 후 archive 이동한다. +4. PASS이고 task group이 `m-`이면 완료 이벤트 메타데이터를 보고한다. roadmap 상태 체크와 `update-roadmap` 호출은 런타임 책임이다. +5. 적용 가능한 `코드리뷰 전용 체크리스트` 항목을 최종 `.log` 위치에서 체크한 뒤 보고한다. + +--- + +## 구현 항목별 완료 여부 + +| 항목 | 완료 여부 | +|------|---------| +| [API-1] Mode Contract | [x] | +| [API-2] Sideband Overlay | [x] | +| [API-3] Transform Output | [x] | + +## 구현 체크리스트 + +- [x] `mode-contract`: official mode constants/parser/routing guard are complete, omitted mode stays pure `passthrough`, explicit sideband/transformed choose the intended non-byte-identity paths, and unknown mode returns invalid request. 검증: mode 누락, explicit sideband, explicit transformed routing test가 각각 기대 경로를 타고 unknown mode는 fail-fast error가 된다. +- [x] `sideband-overlay`: explicit `passthrough+sideband` exposes provider-original content plus IOP observation through a contracted extension stream/schema, while pure `passthrough` contains no IOP sideband field/event. 검증: pure passthrough 응답에 IOP sideband가 없고 opt-in sideband mode에서 같은 response stream 또는 계약된 extension surface로 route/provider/assembled observation이 확인된다. +- [x] `transform-output`: explicit `transformed` response is labeled/metadata-marked as IOP transformed output and is excluded from provider byte-identity assertions. 검증: transformed response가 IOP output으로 라벨링되고 original passthrough 검증과 분리된다. +- [x] Final verification command passes: `go test -count=1 ./apps/edge/internal/service ./apps/edge/internal/transport ./apps/edge/internal/openai`. +- [x] `git diff --check` passes. +- [x] CODE_REVIEW-*-G??.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채운다. 이 항목이 완료되기 전에는 구현이 완료된 것이 아니다. + +## 코드리뷰 전용 체크리스트 + +> **[REVIEW AGENT ONLY]** 이 체크리스트는 코드리뷰 에이전트만 사용한다. +> 구현 에이전트는 이 섹션을 수정하거나 체크하지 않는다. + +- [x] `코드리뷰 결과`에 `PASS`, `WARN`, `FAIL` 중 하나의 판정을 append한다. +- [x] 판정과 `차원별 평가`, Required/Suggested/Nit 분류가 서로 일치한다. +- [x] active `CODE_REVIEW-*-G??.md`를 `code_review_cloud_G06_N.log`로 아카이브한다. +- [x] active `PLAN-*-G??.md`를 `plan_cloud_G06_M.log`로 아카이브한다. +- [x] `.gitignore`의 Agent-Ops 관리 block이 `agent-task/**/*.md`와 `agent-task/**/*.log`를 unignore하고 `agent-roadmap/current.md`를 ignore하는지 확인한다. +- [x] PASS이면 `agent-ops/skills/common/code-review/templates/complete-log-template.md` 기준으로 `complete.log`를 작성하고 active `.md` 파일을 남기지 않는다. +- [x] PASS이면 active task 디렉터리 `agent-task/m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics/`를 `agent-task/archive/YYYY/MM/m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics/`로 이동하고 최종 archive 경로에서 이 체크리스트를 갱신한다. +- [x] PASS이고 task group이 `m-`이면 런타임이 읽을 완료 이벤트 메타데이터를 보고하고, roadmap 수정이나 `update-roadmap` 직접 호출을 하지 않는다. +- [x] PASS split 작업이면 이동 후 빈 active parent `agent-task/m-openai-compatible-raw-tunnel-sideband-passthrough/`를 제거하거나, 남은 sibling/file이 있어 유지했다고 확인한다. +- [ ] WARN/FAIL이고 user-review gate가 트리거되지 않았으면 다음 active `PLAN-{build_lane}-GNN.md`와 `CODE_REVIEW-{review_lane}-GNN.md`를 작성하고 `complete.log`를 작성하지 않는다. +- [ ] USER_REVIEW이면 `agent-ops/skills/common/code-review/templates/user-review-template.md` 기준으로 `USER_REVIEW.md`를 작성하고 active `PLAN-*.md`, `CODE_REVIEW-*.md`, `complete.log`를 남기지 않는다. +- [ ] USER_REVIEW가 연결된 Milestone 결정으로 완료/PASS 해소되면 `USER_REVIEW.md`를 해소 상태로 갱신하고 `complete.log`를 작성한 뒤 task directory를 archive로 이동한다. + +## 계획 대비 변경 사항 + +- `apps/edge/internal/service`와 `apps/edge/internal/transport`는 수정하지 않았다. 계획이 조건부로만 허용한 항목이며, 기존 `ProviderTunnelFrame`의 `usage`/`status_code`/`headers`와 `handle.Dispatch()` 메타데이터만으로 sideband route/usage/assembled observation을 구성할 수 있었다. +- 계획의 API-2 테스트 목록에 없던 `TestChatCompletionsPassthroughSidebandNonStreamingWrapsProviderBody`를 추가했다. 계획 본문이 요구한 non-streaming sideband envelope 계약("wrap or annotate only under an explicit IOP extension schema")을 검증하기 위한 것으로, streaming 테스트만으로는 envelope schema가 커버되지 않는다. +- 검증 명령은 계획 그대로 실행했고 대체하지 않았다. 참고: `go vet ./apps/edge/internal/openai`는 이번 변경과 무관한 기존 이슈(`input_estimator.go:95` `WriteByte` signature)를 보고하지만, 이는 계약된 검증 명령이 아니며 이번 diff에 포함되지 않은 파일이라 수정하지 않았다(요청 범위 초과 방지). + +## 주요 설계 결정 + +- **Sideband streaming surface**: explicit `passthrough+sideband` streaming 응답은 provider SSE bytes를 그대로 relay하면서 named SSE event `event: iop.sideband`로 IOP observation을 interleave한다. injection은 provider event 경계(스트림 시작, 누적 스트림이 `\n\n`으로 끝난 지점, 스트림 종료)에서만 일어나며, 경계 판정은 마지막 2바이트 tail 추적(`sseTail`)으로 chunk 분할에 안전하게 처리한다. 경계가 아닐 때 도착한 usage observation은 pending queue에 쌓았다가 다음 경계에서 방출한다. provider stream이 event 중간에서 끝나면 `\n\n`으로 종결한 뒤 trailing IOP event를 붙인다 — sideband mode는 byte identity를 주장하지 않으므로 허용된다. +- **Sideband observation 종류**: `route`(run_id, node_id, adapter, target, model_group, response_mode, provider_status_code) — 헤더 직후 방출, `usage`(input/output tokens) — tunnel USAGE frame에서 파생, `assembled`(content, reasoning, tool_call_names, body_bytes) — 관대한(tolerant) `providerChatAssembler`가 provider SSE `delta` 또는 non-streaming `message`에서 조립하며 파싱 실패 시 빈 요약으로 무해하게 동작한다. +- **Sideband non-streaming envelope**: `{"object":"iop.chat.passthrough_sideband","iop_sideband":{route,usage,assembled},"provider_status_code":...,"provider_response":<원본 JSON verbatim>}` 스키마로 wrap한다. provider body가 JSON이 아니면 `provider_body` string field로 담는다. provider HTTP status는 envelope 응답의 status로 보존한다. +- **IOP extension 라벨 헤더**: `X-IOP-Response-Mode` 응답 헤더를 explicit IOP extension 출력에만 부여한다 — `transformed` 응답과 `passthrough+sideband` 응답에 설정하고, pure `passthrough`는 provider-original 헤더를 유지하며 절대 부여하지 않는다. transformed는 추가로 run metadata `iop_response_mode=transformed`를 정규화해 남긴다. sideband streaming 응답에서는 body가 확장되므로 provider `Content-Length` 헤더를 제거한다. +- **경로 구성**: `submitChatCompletionTunnel` 공용 helper가 response mode를 tunnel metadata에 실어 dispatch하고, pure passthrough byte writer(`writeProviderTunnelResponse`)는 변경 없이 유지했다. sideband는 `writeProviderTunnelSidebandStream`(streaming)과 `writeProviderTunnelSidebandResponse`(non-streaming)로 분리했다. sideband mode가 non-tunnel(CLI/ollama) route에 지정되면 기존과 동일하게 normalized path로 내려간다(제공자 원본 스트림이 없어 sideband overlay 대상이 아님). + +## 사용자 리뷰 요청 + +_기본값은 `없음`이다. 구현 중 새 결정이 필요해 보여도 직접 질문하거나 선택지를 제시하거나 `request_user_input`을 호출하지 않는다. 이 섹션은 선택된 Milestone `구현 잠금 > 결정 필요` 항목이 실구현을 차단할 때만 채운다. 외부 환경/secret/서비스 준비, 검증 증거 공백, 반복 실패, 일반 범위 조정은 사용자 리뷰 요청이 아니며 `검증 결과`, `계획 대비 변경 사항`, 또는 code-review의 일반 follow-up plan으로 처리한다._ + +- 상태: 없음 +- 사유 유형: 없음 +- 연결 대상: 없음 +- 결정 필요: 없음 +- 차단 근거: 없음 +- 실행한 검증/명령: 없음 +- 자동 후속 불가 이유: 없음 +- 재개 조건: 없음 + +## 리뷰어를 위한 체크포인트 + +- Unknown `metadata.iop_response_mode` fails before dispatch and cannot silently fall back to normalized path. +- Pure `passthrough` remains byte-identical and does not expose any IOP sideband marker/event. +- `passthrough+sideband` is explicit opt-in and is not represented as provider-original byte identity. +- `transformed` output has a testable IOP transformed label/metadata and remains separated from passthrough byte-identity tests. +- The review evidence maps exactly to Roadmap Targets `mode-contract`, `sideband-overlay`, and `transform-output`. + +## 검증 결과 + +_구현 에이전트가 각 중간 검증 및 최종 검증 명령 실행 후 출력을 여기에 붙여 넣는다._ + +필수 규칙: +- 검증 명령은 고정된 계약이다. 임의로 대체하지 않는다. +- 대체가 필요하면 `계획 대비 변경 사항`에 이유와 대체 명령을 기록한다. +- `검증 결과`에는 실제 stdout/stderr를 붙여 넣는다. +- 사용자 리뷰 요청으로 명령을 끝까지 실행하지 못했다면 `사용자 리뷰 요청`에 실행한 명령, 실제 출력, 미실행 명령의 사유를 기록한다. +- mobile/UI hang, timeout, 또는 2분 무진행은 blind retry를 중단하고 focused rerun 명령과 screenshot/window/UI-tree evidence path를 남기며, 불가능하면 정확한 사유를 남긴다. + +### API-1 중간 검증 +```bash +$ go test -count=1 ./apps/edge/internal/openai -run 'TestChatCompletionsResponseModeRouting|TestChatCompletionsProviderPoolDispatch|TestChatCompletionsExplicitNonPassthroughModeKeepsNormalizedPath' +ok iop/apps/edge/internal/openai 0.009s +``` + +### API-2 중간 검증 +```bash +$ go test -count=1 ./apps/edge/internal/openai -run 'TestChatCompletionsPassthroughSideband|TestChatCompletionsPassthroughDoesNotExposeSideband|TestChatCompletionsResponseModeRouting|TestChatCompletionsPassthroughStreamingByteIdentity|TestChatCompletionsPassthroughNonStreamingByteIdentity' +ok iop/apps/edge/internal/openai 0.022s +``` + +`-v` 실행에서 개별 테스트 확인: `TestChatCompletionsResponseModeRouting`(4 subtests), `TestChatCompletionsPassthroughSidebandStreamingExposesIOPExtension`, `TestChatCompletionsPassthroughSidebandNonStreamingWrapsProviderBody`, `TestChatCompletionsPassthroughDoesNotExposeSideband`, `TestChatCompletionsPassthroughNonStreamingByteIdentity`, `TestChatCompletionsPassthroughStreamingByteIdentity` 전부 PASS. + +### API-3 중간 검증 +```bash +$ go test -count=1 ./apps/edge/internal/openai -run 'TestChatCompletionsTransformedModeLabelsIOPOutput|TestChatCompletionsPassthroughDoesNotLabelTransformed|TestChatCompletionsExplicitNonPassthroughModeKeepsNormalizedPath|TestChatCompletionsPassthroughNonStreamingByteIdentity|TestChatCompletionsPassthroughStreamingByteIdentity' +ok iop/apps/edge/internal/openai 0.013s +``` + +### 최종 검증 +```bash +$ go test -count=1 ./apps/edge/internal/service ./apps/edge/internal/transport ./apps/edge/internal/openai +ok iop/apps/edge/internal/service 0.710s +ok iop/apps/edge/internal/transport 2.039s +ok iop/apps/edge/internal/openai 1.580s + +$ git diff --check +(출력 없음: clean, exit code 0) +``` + +--- + +> **[IMPLEMENTING AGENT — BEFORE SAVING] Have you filled in every implementation-owned section: completion table, implementation checklist, changes from plan, design decisions, and verification output?** +> If anything is blank, go back and fill it in before saving this file. +> Leave review-agent-only sections unchanged. + +## Ownership + +| Section | Owner | Note | +|---------|-------|------| +| Header comment, 개요, 리뷰 에이전트 지시 | Fixed at stub creation | Implementing agent must not modify or execute these. | +| Roadmap Targets | Fixed at stub creation from plan | Implementing agent must not modify; code-review copies it into `complete.log` as `Roadmap Completion` only on PASS. | +| Archive Evidence Snapshot | Fixed at stub creation from plan | Implementing agent uses it as default prior-loop context; read only the specific archive files cited there when more detail is required. | +| 구현 항목별 완료 여부 | Implementing agent checks boxes only | Item names stay fixed. | +| 구현 체크리스트 | Implementing agent checks boxes only | Text/order stay fixed; final checkbox is mandatory. | +| 코드리뷰 전용 체크리스트 | Review agent only | Implementing agent must not modify or check this section. | +| 계획 대비 변경 사항, 주요 설계 결정 | Implementing agent | Replace placeholder text with actual content. | +| 사용자 리뷰 요청 | Implementing agent | Keep `상태: 없음` unless a selected Milestone lock decision blocks implementation. | +| 리뷰어를 위한 체크포인트 | Fixed at stub creation | Pre-filled from plan. | +| 검증 결과 | Implementing agent | Fill actual command output only; command changes require `계획 대비 변경 사항`. | +| 코드리뷰 결과 | Review agent appends | Not included in stub. | + +## 코드리뷰 결과 + +- 종합 판정: PASS +- 차원별 평가: + - Correctness: Pass + - Completeness: Pass + - Test coverage: Pass + - API contract: Pass + - Code quality: Pass + - Implementation deviation: Pass + - Verification trust: Pass + - Spec conformance: Pass +- 발견된 문제: 없음 +- 다음 단계: PASS. `complete.log`를 작성하고 active task directory를 archive로 이동한다. `m-openai-compatible-raw-tunnel-sideband-passthrough` completion event metadata를 보고하며 roadmap은 code-review에서 직접 수정하지 않는다. diff --git a/agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics/complete.log b/agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics/complete.log new file mode 100644 index 0000000..389ed95 --- /dev/null +++ b/agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics/complete.log @@ -0,0 +1,47 @@ +# Complete - m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics + +## 완료 일시 + +2026-07-08 + +## 요약 + +OpenAI-compatible Chat Completions response mode semantics를 구현한 1회차 루프가 PASS로 종료되었다. + +## 루프 이력 + +| Plan | Review | Verdict | 메모 | +|------|--------|---------|------| +| `plan_cloud_G06_0.log` | `code_review_cloud_G06_0.log` | PASS | `passthrough`, `passthrough+sideband`, `transformed` routing/output semantics와 SDD S05-S07/S11 evidence가 충족되었다. | + +## 구현/정리 내용 + +- `metadata.iop_response_mode` parser와 공식 mode routing guard를 추가하고 unknown mode fail-fast를 검증했다. +- explicit `passthrough+sideband` provider tunnel path를 추가해 streaming SSE sideband event와 non-streaming IOP envelope를 제공했다. +- pure `passthrough` byte identity와 sideband/label 부재를 유지했다. +- explicit `transformed` normalized output에 `X-IOP-Response-Mode: transformed` 라벨과 run metadata를 남겼다. +- handler/service/transport/openai 및 Edge package 테스트로 response mode semantics와 기존 Edge quick check를 검증했다. + +## 최종 검증 + +- `go test -count=1 ./apps/edge/internal/service ./apps/edge/internal/transport ./apps/edge/internal/openai` - PASS; `service`, `transport`, `openai` 패키지가 모두 `ok`로 완료되었다. +- `git diff --check` - PASS; 출력 없음. +- `go test -count=1 ./apps/edge/...` - PASS; `apps/edge` 하위 전체 Go package가 모두 `ok`로 완료되었다. + +## Roadmap Completion + +- Milestone: `agent-roadmap/phase/routing-policy-model-orchestration/milestones/openai-compatible-raw-tunnel-sideband-passthrough.md` +- Milestone link: [Milestone 문서](agent-roadmap/phase/routing-policy-model-orchestration/milestones/openai-compatible-raw-tunnel-sideband-passthrough.md) +- Completed task ids: + - `mode-contract`: PASS; evidence=`plan_cloud_G06_0.log`, `code_review_cloud_G06_0.log`; verification=`go test -count=1 ./apps/edge/internal/openai -run 'TestChatCompletionsResponseModeRouting|TestChatCompletionsProviderPoolDispatch|TestChatCompletionsExplicitNonPassthroughModeKeepsNormalizedPath'` + - `sideband-overlay`: PASS; evidence=`plan_cloud_G06_0.log`, `code_review_cloud_G06_0.log`; verification=`go test -count=1 ./apps/edge/internal/openai -run 'TestChatCompletionsPassthroughSideband|TestChatCompletionsPassthroughDoesNotExposeSideband|TestChatCompletionsResponseModeRouting|TestChatCompletionsPassthroughStreamingByteIdentity|TestChatCompletionsPassthroughNonStreamingByteIdentity'` + - `transform-output`: PASS; evidence=`plan_cloud_G06_0.log`, `code_review_cloud_G06_0.log`; verification=`go test -count=1 ./apps/edge/internal/openai -run 'TestChatCompletionsTransformedModeLabelsIOPOutput|TestChatCompletionsPassthroughDoesNotLabelTransformed|TestChatCompletionsExplicitNonPassthroughModeKeepsNormalizedPath|TestChatCompletionsPassthroughNonStreamingByteIdentity|TestChatCompletionsPassthroughStreamingByteIdentity'` +- Not completed task ids: 없음 + +## 잔여 Nit + +- 없음 + +## 후속 작업 + +- 없음 diff --git a/agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics/plan_cloud_G06_0.log b/agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics/plan_cloud_G06_0.log new file mode 100644 index 0000000..3fe828c --- /dev/null +++ b/agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics/plan_cloud_G06_0.log @@ -0,0 +1,364 @@ + + +# Plan - API Response Modes Semantics + +## 이 파일을 읽는 구현 에이전트에게 + +구현 완료의 마지막 단계는 반드시 `CODE_REVIEW-*-G??.md`의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채우는 것이다. 검증을 실행하고, 실제 stdout/stderr를 붙이고, active 파일을 그대로 둔 채 review ready를 보고한다. finalization, `complete.log` 작성, archive 이동은 code-review-skill 전용이다. + +선택된 Milestone의 `구현 잠금 > 결정 필요` 항목이 실구현을 막을 때만 active review stub의 `사용자 리뷰 요청` 섹션을 정확한 근거와 함께 채우고 멈춘다. 구현 중 직접 사용자에게 질문하거나 선택지를 제시하거나 `request_user_input`을 호출하지 않는다. 환경/secret/service 준비, 일반 범위 조정, 반복 실패, follow-up agent가 닫을 수 있는 evidence gap은 사용자 리뷰 요청이 아니며 `검증 결과`나 `계획 대비 변경 사항`에 기록한다. + +## 배경 + +현재 Milestone의 raw-tunnel epic은 완료되어 기본 omitted mode가 provider-original `passthrough`로 동작한다. response-modes epic은 이 기본 경로를 훼손하지 않으면서 `metadata.iop_response_mode`의 세 공식 모드와 opt-in sideband/transformed semantics를 닫아야 한다. 이번 세션에서 작은 선행 변경으로 mode parser와 unknown mode fail-fast test가 active diff에 들어갔으므로, 구현 에이전트는 이를 보존하고 실제 sideband/transformed 출력 계약까지 이어서 완성한다. + +## 사용자 리뷰 요청 흐름 + +사용자 리뷰 요청은 선택된 Milestone lock decision이 실구현을 차단할 때만 active `CODE_REVIEW-*-G??.md`의 `사용자 리뷰 요청` 섹션에 기록한다. 이 섹션은 `agent-ops/skills/common/_templates/implementation-user-review-request-section.md` 형식을 그대로 따른다. 구현 중 direct user prompt, chat 선택지 제시, `request_user_input`, `USER_REVIEW.md` 직접 생성은 금지이며 code-review가 validation과 실제 `USER_REVIEW.md` 작성을 소유한다. + +## Archive Evidence Snapshot + +- `agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/01_node_relay/complete.log`: PASS. Node relay가 OpenAI-compatible provider response를 기존 Edge-Node socket tunnel frame으로 전달하는 선행 작업을 완료했다. +- `agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/02+01_edge_stream/complete.log`: PASS. Edge stream pure passthrough 기본값과 streaming/non-streaming byte identity 경로가 완료되어 이번 subtask의 predecessor `02`를 만족한다. +- 더 자세한 선행 맥락이 필요할 때만 위 두 `complete.log`를 좁게 읽는다. `agent-task/archive/**`를 광범위하게 검색하지 않는다. + +## Roadmap Targets + +- Milestone: `agent-roadmap/phase/routing-policy-model-orchestration/milestones/openai-compatible-raw-tunnel-sideband-passthrough.md` +- Milestone link: [Milestone 문서](agent-roadmap/phase/routing-policy-model-orchestration/milestones/openai-compatible-raw-tunnel-sideband-passthrough.md) +- Task ids: + - `mode-contract`: OpenAI-compatible request metadata/config 계약에 `metadata.iop_response_mode`와 `passthrough`, `passthrough+sideband`, `transformed` 모드를 정의하고 omitted mode는 `passthrough`로 해석한다. + - `sideband-overlay`: `passthrough+sideband`는 provider original stream과 IOP sideband를 명시적 IOP extension stream/schema로 함께 노출하되 pure `passthrough` body에는 IOP field/event를 주입하지 않는다. + - `transform-output`: `transformed`는 provider raw stream을 IOP schema/output으로 수정할 수 있지만 byte-identical provider stream으로 표시하지 않는다. +- Completion mode: check-on-pass + +## 분석 결과 + +### 읽은 파일 + +- `agent-ops/rules/project/rules.md` +- `agent-ops/rules/common/rules-roadmap.md` +- `agent-ops/skills/common/router.md` +- `agent-ops/skills/common/plan/SKILL.md` +- `agent-ops/skills/common/_templates/implementation-user-review-request-section.md` +- `agent-ops/rules/project/domain/edge/rules.md` +- `agent-test/local/rules.md` +- `agent-test/local/edge-smoke.md` +- `agent-contract/index.md` +- `agent-contract/outer/openai-compatible-api.md` +- `agent-contract/inner/edge-node-runtime-wire.md` +- `agent-ops/rules/common/rules-agent-spec.md` +- `agent-spec/index.md` +- `agent-spec/input/openai-compatible-surface.md` +- `agent-roadmap/phase/routing-policy-model-orchestration/PHASE.md` +- `agent-roadmap/phase/routing-policy-model-orchestration/milestones/openai-compatible-raw-tunnel-sideband-passthrough.md` +- `agent-roadmap/sdd/routing-policy-model-orchestration/openai-compatible-raw-tunnel-sideband-passthrough/SDD.md` +- `apps/edge/internal/openai/chat_handler.go` +- `apps/edge/internal/openai/stream.go` +- `apps/edge/internal/openai/server_test.go` + +### SDD 기준 + +- SDD: `agent-roadmap/sdd/routing-policy-model-orchestration/openai-compatible-raw-tunnel-sideband-passthrough/SDD.md` +- 상태: `[승인됨]`, SDD 잠금 `해제`, 사용자 리뷰 `없음`. +- S05 maps to `mode-contract` and `sideband-overlay`: explicit `passthrough+sideband` must expose provider original body plus IOP sideband observation through an explicit extension stream/schema. This drives the sideband fixture and routing/output checklist. +- S06 maps to `sideband-overlay`: pure `passthrough` must not contain IOP sideband field/event. This drives negative tests on existing pure passthrough streaming/non-streaming fixtures. +- S07 maps to `transform-output`: explicit `transformed` output must be labeled/metadata-marked as IOP transformed and excluded from byte-identity claims. This drives response metadata/label tests on normalized output. +- S11 maps to `mode-contract`: unknown `metadata.iop_response_mode` must fail fast with invalid request. The active diff already adds the parser and regression test; implementation must preserve or refine it. + +### 테스트 환경 규칙 + +- test_env: `local`. +- `agent-test/local/rules.md` was present and read. The OpenAI-compatible Edge boundary maps to `agent-test/local/edge-smoke.md`, which was present and read. +- Applied verification source: deterministic Go handler/service/transport tests in this checkout, because the requested work is contract-level Edge behavior and the package tests exercise `/v1/chat/completions` handler fixtures without needing a live Edge process or external provider. +- Verified command availability/layout before writing this plan: + - `go test -count=1 ./apps/edge/internal/service ./apps/edge/internal/transport ./apps/edge/internal/openai` passes in the current checkout. +- Required verification does not leave the current checkout. No remote runner, secret, external provider, Docker service, emulator, or long-running runtime preflight is required. +- Fresh execution matters for mode routing and stream behavior, so Go test cache output is not acceptable; use `-count=1`. + +### 테스트 커버리지 공백 + +- `mode-contract`: omitted mode pure passthrough and explicit transformed path already have tests; the active diff adds explicit `passthrough`, `passthrough+sideband`, `transformed`, and unknown mode routing coverage. Remaining gap: contract/spec/doc parity is intentionally deferred to `contract-docs`, but code fixtures must use the final mode names. +- `sideband-overlay`: existing pure passthrough byte-identity tests cover no mutation by body equality, and `stream.go:465` ignores usage frames in pure passthrough. Missing: opt-in `passthrough+sideband` extension schema/stream test and explicit negative assertion that pure passthrough contains no IOP sideband marker/event. +- `transform-output`: existing transformed-mode tests cover normalized tool/reasoning/strict output behavior. Missing: response label/metadata evidence that transformed output is not provider-original byte identity and is separated from passthrough assertions. + +### 심볼 참조 + +- Removed/renamed symbol in active diff: `passthroughResponseMode` removed. +- `rg --sort path -n "passthroughResponseMode|parseResponseMode|responseModePassthroughSideband|responseModeTransformed|iop_response_mode" apps/edge/internal/openai` found no remaining `passthroughResponseMode` references. +- Current `parseResponseMode` references: `apps/edge/internal/openai/chat_handler.go:47`, definition at `apps/edge/internal/openai/chat_handler.go:283`. +- `iop_response_mode` test fixtures are concentrated in `apps/edge/internal/openai/server_test.go` and should continue to use the exact SDD mode strings. + +### 분할 판단 + +- Split decision policy was evaluated before choosing this plan. +- Shared task group: `agent-task/m-openai-compatible-raw-tunnel-sideband-passthrough`. +- Existing completed predecessors: + - `01_node_relay`: satisfied by archived `agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/01_node_relay/complete.log`. + - `02+01_edge_stream`: satisfied by archived `agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/02+01_edge_stream/complete.log`. +- This subtask directory is `03+02_response_modes_semantics`; runtime dependency source of truth is predecessor `02`. +- A further split between `mode-contract`, `sideband-overlay`, and `transform-output` was considered. They share the same public response-mode entry point, the same handler tests, and the same SDD contract boundary; splitting would risk temporary contradictory mode semantics. The scope stays within one primary ownership boundary, `apps/edge/internal/openai`, with optional narrow service/transport touch only if sideband observation needs tunnel result metadata. + +### 범위 결정 근거 + +- Do not change Node relay protocol or generated proto unless implementation proves the current `ProviderTunnelFrame` fields cannot carry the required sideband observation. Existing `wire-contract` and `node-relay` tasks are complete. +- Do not implement Responses API raw tunnel parity here; the Milestone context says Chat Completions first and Responses parity hardening is follow-up. +- Do not change routing score, provider qualification, billing/chargeback, raw payload retention policy, or normalized path deprecation. +- Do not inject IOP sideband into pure `passthrough` body; the only allowed mixed response body is explicit `passthrough+sideband`. +- Do not update agent-contract/spec/docs as completion evidence here beyond code comments/test fixture names; `contract-docs` is a separate roadmap Task. + +### 빌드 등급 + +- `cloud-G06`: behavior is promised by SDD/outer contract beyond a single file, and a plausible local-only implementation could pass simple handler tests while mislabeling sideband/transformed semantics. Scope is moderate and bounded to Edge OpenAI response handling. + +## 구현 체크리스트 + +- [ ] `mode-contract`: official mode constants/parser/routing guard are complete, omitted mode stays pure `passthrough`, explicit sideband/transformed choose the intended non-byte-identity paths, and unknown mode returns invalid request. 검증: mode 누락, explicit sideband, explicit transformed routing test가 각각 기대 경로를 타고 unknown mode는 fail-fast error가 된다. +- [ ] `sideband-overlay`: explicit `passthrough+sideband` exposes provider-original content plus IOP observation through a contracted extension stream/schema, while pure `passthrough` contains no IOP sideband field/event. 검증: pure passthrough 응답에 IOP sideband가 없고 opt-in sideband mode에서 같은 response stream 또는 계약된 extension surface로 route/provider/assembled observation이 확인된다. +- [ ] `transform-output`: explicit `transformed` response is labeled/metadata-marked as IOP transformed output and is excluded from provider byte-identity assertions. 검증: transformed response가 IOP output으로 라벨링되고 original passthrough 검증과 분리된다. +- [ ] Final verification command passes: `go test -count=1 ./apps/edge/internal/service ./apps/edge/internal/transport ./apps/edge/internal/openai`. +- [ ] `git diff --check` passes. +- [ ] CODE_REVIEW-*-G??.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채운다. 이 항목이 완료되기 전에는 구현이 완료된 것이 아니다. + +## 의존 관계 및 구현 순서 + +`03+02_response_modes_semantics` depends only on predecessor index `02`, satisfied by `agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/02+01_edge_stream/complete.log`. Implement in this order: `mode-contract` final check, `sideband-overlay`, then `transform-output`, because sideband/transformed routing depends on the finalized parser/dispatch semantics. + +### [API-1] Mode Contract + +#### 문제 + +`apps/edge/internal/openai/chat_handler.go:75` previously treated all unknown non-`passthrough` modes as normalized path. SDD S11 requires unknown `metadata.iop_response_mode` to fail fast, and the Milestone `mode-contract` task requires explicit `passthrough`, `passthrough+sideband`, and `transformed` routing tests. + +Before active diff: + +```go +// apps/edge/internal/openai/chat_handler.go:72 +if passthroughResponseMode(runMeta) && routeUsesProviderTunnel(dispatch) { + estimate := s.estimateChatInputTokens(basePrompt, runMeta, req.Tools, req.ToolChoice) + contextClass := classifyContext(estimate, s.longContextThreshold()) + s.tunnelChatCompletionPassthrough(w, r, req, dispatch, runMeta, rawBody, estimate, contextClass) + return +} +``` + +Current active diff shape: + +```go +// apps/edge/internal/openai/chat_handler.go:47 +responseMode, err := parseResponseMode(runMeta) +if err != nil { + writeError(w, http.StatusBadRequest, "invalid_request_error", err.Error()) + return +} + +// apps/edge/internal/openai/chat_handler.go:75 +if responseMode == responseModePassthrough && routeUsesProviderTunnel(dispatch) { +``` + +#### 해결 방법 + +Keep `parseResponseMode` as the single source of truth. It must trim whitespace, default empty mode to `passthrough`, accept only `passthrough`, `passthrough+sideband`, `transformed`, and return an OpenAI-compatible invalid request error before route dispatch for unknown modes. + +After target: + +```go +// apps/edge/internal/openai/chat_handler.go:276 +const ( + responseModeMetadataKey = "iop_response_mode" + responseModePassthrough = "passthrough" + responseModePassthroughSideband = "passthrough+sideband" + responseModeTransformed = "transformed" +) + +func parseResponseMode(runMeta map[string]string) (string, error) { + mode := strings.TrimSpace(runMeta[responseModeMetadataKey]) + if mode == "" { + return responseModePassthrough, nil + } + switch mode { + case responseModePassthrough, responseModePassthroughSideband, responseModeTransformed: + return mode, nil + default: + return "", fmt.Errorf("metadata.%s must be one of %s, %s, or %s", responseModeMetadataKey, responseModePassthrough, responseModePassthroughSideband, responseModeTransformed) + } +} +``` + +#### 수정 파일 및 체크리스트 + +- [ ] `apps/edge/internal/openai/chat_handler.go`: keep official constants, parser, and route guard. +- [ ] `apps/edge/internal/openai/server_test.go`: keep `TestChatCompletionsResponseModeRouting`; update expected sideband route after API-2 lands. + +#### 테스트 작성 + +Write/update tests in `apps/edge/internal/openai/server_test.go`: + +- `TestChatCompletionsResponseModeRouting`: omitted behavior remains covered by `TestChatCompletionsProviderPoolDispatch`; explicit `passthrough` uses tunnel; explicit `passthrough+sideband` follows API-2 path; explicit `transformed` follows normalized/transformed path; unknown `rawish` returns HTTP 400 with the allowed-mode message. + +#### 중간 검증 + +```bash +go test -count=1 ./apps/edge/internal/openai -run 'TestChatCompletionsResponseModeRouting|TestChatCompletionsProviderPoolDispatch|TestChatCompletionsExplicitNonPassthroughModeKeepsNormalizedPath' +``` + +Expected: `ok`. + +### [API-2] Sideband Overlay + +#### 문제 + +`apps/edge/internal/openai/stream.go:309` implements pure tunnel passthrough and `apps/edge/internal/openai/stream.go:465` ignores usage frames so pure passthrough remains byte-identical. SDD S05/S06 require a separate explicit `passthrough+sideband` mode that can expose IOP observation through a contracted extension surface while never contaminating pure `passthrough`. + +Current pure passthrough: + +```go +// apps/edge/internal/openai/stream.go:309 +// tunnelChatCompletionPassthrough serves a Chat Completions request over the +// raw provider tunnel (SDD S04/S06): provider status, headers, and body bytes +// are written to the caller byte-identically. No IOP sideband fields or events +// are added to the response body. +func (s *Server) tunnelChatCompletionPassthrough(... ) { + ... + metadata[responseModeMetadataKey] = responseModePassthrough + ... + s.writeProviderTunnelResponse(w, r, handle) +} + +// apps/edge/internal/openai/stream.go:465 +case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_USAGE: + // Usage frames are sideband observation candidates; pure passthrough + // never merges them into the response body (SDD D04). + continue +``` + +#### 해결 방법 + +Add an explicit sideband path selected only by `responseModePassthroughSideband`. Keep pure passthrough byte writer unchanged. For streaming Chat Completions, choose one contracted extension surface and test it end-to-end: + +- Prefer SSE extension events only in `passthrough+sideband`, e.g. preserve provider `data:` body frames and append/emit `event: iop.sideband` with JSON observation for route/provider/usage/assembled summary. +- For non-streaming Chat Completions, wrap or annotate only under an explicit IOP extension schema, not as provider-original byte identity. +- Any sideband marker must include enough evidence for route/provider/assembled observation and must not appear in pure `passthrough` tests. + +Example target sketch: + +```go +// apps/edge/internal/openai/chat_handler.go:75 +switch { +case responseMode == responseModePassthrough && routeUsesProviderTunnel(dispatch): + s.tunnelChatCompletionPassthrough(...) + return +case responseMode == responseModePassthroughSideband && routeUsesProviderTunnel(dispatch): + s.tunnelChatCompletionPassthroughSideband(...) + return +} +``` + +```go +// apps/edge/internal/openai/stream.go +func (s *Server) tunnelChatCompletionPassthroughSideband(...) { + // Submit the same provider tunnel request with metadata mode passthrough+sideband. + // Relay provider bytes through the contracted extension writer and emit + // only explicit IOP sideband observations. +} +``` + +#### 수정 파일 및 체크리스트 + +- [ ] `apps/edge/internal/openai/chat_handler.go`: dispatch `responseModePassthroughSideband` to the new sideband path for provider tunnel routes only. +- [ ] `apps/edge/internal/openai/stream.go`: add the explicit sideband writer without changing `writeProviderTunnelResponse` pure byte writer semantics. +- [ ] `apps/edge/internal/openai/server_test.go`: add sideband fixture tests and pure passthrough negative assertions. +- [ ] `apps/edge/internal/service` / `apps/edge/internal/transport`: touch only if the sideband observation requires existing tunnel result metadata not currently exposed to Edge. + +#### 테스트 작성 + +Write/update tests in `apps/edge/internal/openai/server_test.go`: + +- `TestChatCompletionsPassthroughSidebandStreamingExposesIOPExtension`: provider SSE fixture plus usage/route observation emits provider content and explicit IOP sideband event/schema only in `passthrough+sideband`. +- `TestChatCompletionsPassthroughDoesNotExposeSideband`: pure passthrough fixture with usage frame does not contain `iop.sideband`, `iop_response_mode`, route/provider sideband marker, or any chosen extension key. +- Update `TestChatCompletionsResponseModeRouting` sideband case to expect the sideband path instead of the temporary normalized path. + +#### 중간 검증 + +```bash +go test -count=1 ./apps/edge/internal/openai -run 'TestChatCompletionsPassthroughSideband|TestChatCompletionsPassthroughDoesNotExposeSideband|TestChatCompletionsResponseModeRouting|TestChatCompletionsPassthroughStreamingByteIdentity|TestChatCompletionsPassthroughNonStreamingByteIdentity' +``` + +Expected: `ok`. + +### [API-3] Transform Output + +#### 문제 + +Existing transformed-mode fixtures use the normalized RunEvent path, but SDD S07 and the Milestone `transform-output` task require the output to be explicitly identifiable as IOP transformed output and excluded from passthrough byte-identity claims. Without a label/metadata test, future changes can accidentally treat transformed output as provider-original bytes. + +Relevant current behavior: + +```go +// apps/edge/internal/openai/server_test.go:4399 +req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{ + "model":"pool-model", + "messages":[{"role":"user","content":"hello"}], + "metadata":{"iop_response_mode":"transformed"} +}`)) +... +if len(fake.tunnelReqsSnapshot()) != 0 { + t.Error("explicit non-passthrough mode must not use the tunnel") +} +``` + +#### 해결 방법 + +Define a minimal transformed-output label that is visible in testable response metadata/header/body metadata without breaking OpenAI-compatible clients. Prefer a response header or metadata already accepted by local patterns over adding non-standard body fields to core OpenAI objects. The label must be present only for `transformed`, absent in pure passthrough, and not claimed as byte-identical provider output. + +Example target sketch: + +```go +// apps/edge/internal/openai/chat_handler.go +if responseMode == responseModeTransformed { + metadata[responseModeMetadataKey] = responseModeTransformed + // response writer or header marks output as transformed. +} +``` + +#### 수정 파일 및 체크리스트 + +- [ ] `apps/edge/internal/openai/chat_handler.go`: propagate explicit transformed mode into run metadata/output labeling. +- [ ] `apps/edge/internal/openai/stream.go`: if response headers are the chosen label surface, set them consistently for non-streaming and streaming transformed writers. +- [ ] `apps/edge/internal/openai/server_test.go`: assert transformed label/metadata and absence from pure passthrough. + +#### 테스트 작성 + +Write/update tests in `apps/edge/internal/openai/server_test.go`: + +- `TestChatCompletionsTransformedModeLabelsIOPOutput`: explicit transformed response includes the chosen IOP transformed label and uses normalized path. +- `TestChatCompletionsPassthroughDoesNotLabelTransformed`: pure passthrough response does not include the transformed label and remains byte-identical in existing passthrough fixtures. + +#### 중간 검증 + +```bash +go test -count=1 ./apps/edge/internal/openai -run 'TestChatCompletionsTransformedModeLabelsIOPOutput|TestChatCompletionsPassthroughDoesNotLabelTransformed|TestChatCompletionsExplicitNonPassthroughModeKeepsNormalizedPath|TestChatCompletionsPassthroughNonStreamingByteIdentity|TestChatCompletionsPassthroughStreamingByteIdentity' +``` + +Expected: `ok`. + +## 수정 파일 요약 + +| 파일 | 항목 | +|------|------| +| `apps/edge/internal/openai/chat_handler.go` | API-1, API-2, API-3 | +| `apps/edge/internal/openai/stream.go` | API-2, API-3 | +| `apps/edge/internal/openai/server_test.go` | API-1, API-2, API-3 | +| `apps/edge/internal/service/*` | API-2 only if sideband observation needs tunnel metadata access | +| `apps/edge/internal/transport/*` | API-2 only if sideband observation needs tunnel metadata access | + +## 최종 검증 + +```bash +go test -count=1 ./apps/edge/internal/service ./apps/edge/internal/transport ./apps/edge/internal/openai +git diff --check +``` + +Expected: both commands pass. Go test cache output is not acceptable; keep `-count=1`. + +모든 코드 변경 완료 후 반드시 `CODE_REVIEW-*-G??.md`의 구현 에이전트 소유 섹션을 채운다. 이 파일 작성이 구현의 마지막 단계다. diff --git a/apps/edge/internal/openai/chat_handler.go b/apps/edge/internal/openai/chat_handler.go index 2011dbd..8bec433 100644 --- a/apps/edge/internal/openai/chat_handler.go +++ b/apps/edge/internal/openai/chat_handler.go @@ -44,6 +44,11 @@ func (s *Server) handleChatCompletions(w http.ResponseWriter, r *http.Request) { writeError(w, http.StatusBadRequest, "invalid_request_error", err.Error()) return } + responseMode, err := parseResponseMode(runMeta) + if err != nil { + writeError(w, http.StatusBadRequest, "invalid_request_error", err.Error()) + return + } dispatch, ok := s.resolveRouteDispatch(req.Model) if !ok { @@ -66,12 +71,20 @@ func (s *Server) handleChatCompletions(w http.ResponseWriter, r *http.Request) { // OpenAI-compatible provider routes default to pure passthrough (SDD D02): // provider status/headers/body bytes are relayed to the caller unmodified - // over the raw tunnel instead of the normalized RunEvent path. - if passthroughResponseMode(runMeta) && routeUsesProviderTunnel(dispatch) { + // over the raw tunnel instead of the normalized RunEvent path. Explicit + // passthrough+sideband keeps the provider tunnel but exposes IOP + // observations through the sideband extension surface (SDD S05). + switch { + case responseMode == responseModePassthrough && routeUsesProviderTunnel(dispatch): estimate := s.estimateChatInputTokens(basePrompt, runMeta, req.Tools, req.ToolChoice) contextClass := classifyContext(estimate, s.longContextThreshold()) s.tunnelChatCompletionPassthrough(w, r, req, dispatch, runMeta, rawBody, estimate, contextClass) return + case responseMode == responseModePassthroughSideband && routeUsesProviderTunnel(dispatch): + estimate := s.estimateChatInputTokens(basePrompt, runMeta, req.Tools, req.ToolChoice) + contextClass := classifyContext(estimate, s.longContextThreshold()) + s.tunnelChatCompletionPassthroughSideband(w, r, req, dispatch, runMeta, rawBody, estimate, contextClass) + return } messages := req.Messages if instruction := strictOutputContractInstruction(outputPolicy); instruction != "" { @@ -100,6 +113,13 @@ func (s *Server) handleChatCompletions(w http.ResponseWriter, r *http.Request) { metadata := chatRunMetadata(runMeta, req, outputPolicy) metadata["estimated_input_tokens"] = strconv.Itoa(estimate) metadata["context_class"] = contextClass + if responseMode == responseModeTransformed { + // Explicit transformed mode produces IOP-transformed output (SDD S07): + // label the response so it is never mistaken for provider-original + // byte-identical passthrough. + metadata[responseModeMetadataKey] = responseModeTransformed + w.Header().Set(responseModeHeaderName, responseModeTransformed) + } if validation.enabled { metadata = toolValidationAttemptMetadata(metadata, 1, "", "") } @@ -269,17 +289,27 @@ func chatRunMetadata(runMeta map[string]string, req chatCompletionRequest, outpu } const ( - responseModeMetadataKey = "iop_response_mode" - responseModePassthrough = "passthrough" + responseModeMetadataKey = "iop_response_mode" + responseModePassthrough = "passthrough" + responseModePassthroughSideband = "passthrough+sideband" + responseModeTransformed = "transformed" + // responseModeHeaderName labels explicit IOP extension outputs + // (passthrough+sideband, transformed). Pure passthrough responses never + // carry it: their headers stay provider-original. + responseModeHeaderName = "X-IOP-Response-Mode" ) -// passthroughResponseMode reports whether the request should use the pure -// passthrough tunnel: mode omitted (default, SDD D02) or explicitly -// "passthrough". Other explicit modes stay on the legacy normalized path -// until the mode-contract task lands their semantics. -func passthroughResponseMode(runMeta map[string]string) bool { +func parseResponseMode(runMeta map[string]string) (string, error) { mode := strings.TrimSpace(runMeta[responseModeMetadataKey]) - return mode == "" || mode == responseModePassthrough + if mode == "" { + return responseModePassthrough, nil + } + switch mode { + case responseModePassthrough, responseModePassthroughSideband, responseModeTransformed: + return mode, nil + default: + return "", fmt.Errorf("metadata.%s must be one of %s, %s, or %s", responseModeMetadataKey, responseModePassthrough, responseModePassthroughSideband, responseModeTransformed) + } } // routeUsesProviderTunnel reports whether the resolved dispatch targets an diff --git a/apps/edge/internal/openai/server_test.go b/apps/edge/internal/openai/server_test.go index c09a9aa..ecb55ba 100644 --- a/apps/edge/internal/openai/server_test.go +++ b/apps/edge/internal/openai/server_test.go @@ -4070,6 +4070,400 @@ func TestChatCompletionsProviderPoolDispatch(t *testing.T) { } } +func TestChatCompletionsResponseModeRouting(t *testing.T) { + cases := []struct { + name string + metadata string + wantStatus int + wantTunnel bool + wantRun bool + wantMessage string + }{ + { + name: "explicit passthrough uses tunnel", + metadata: `"metadata":{"iop_response_mode":"passthrough"}`, + wantStatus: http.StatusOK, + wantTunnel: true, + }, + { + name: "sideband mode uses tunnel with IOP extension surface", + metadata: `"metadata":{"iop_response_mode":"passthrough+sideband"}`, + wantStatus: http.StatusOK, + wantTunnel: true, + }, + { + name: "transformed mode keeps normalized path", + metadata: `"metadata":{"iop_response_mode":"transformed"}`, + wantStatus: http.StatusOK, + wantRun: true, + }, + { + name: "unknown mode fails fast", + metadata: `"metadata":{"iop_response_mode":"rawish"}`, + wantStatus: http.StatusBadRequest, + wantMessage: "metadata.iop_response_mode must be one of passthrough, passthrough+sideband, or transformed", + }, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + frames := make(chan *iop.ProviderTunnelFrame, 3) + frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_RESPONSE_START, StatusCode: 200} + frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(`{"ok":true}`)} + frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_END, End: true} + close(frames) + fake := &fakeRunService{ + events: bufferedRunEvents( + &iop.RunEvent{Type: "delta", Delta: "ok"}, + &iop.RunEvent{Type: "complete"}, + ), + tunnelFrames: frames, + } + srv := NewServer(config.EdgeOpenAIConf{}, fake, nil) + srv.SetModelCatalog([]config.ModelCatalogEntry{ + {ID: "pool-model", Providers: map[string]string{"prov-1": "served-model"}}, + }) + + body := `{ + "model":"pool-model", + "messages":[{"role":"user","content":"hello"}], + ` + tc.metadata + ` + }` + req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(body)) + w := httptest.NewRecorder() + srv.handleChatCompletions(w, req) + + if w.Code != tc.wantStatus { + t.Fatalf("status: got %d want %d body=%s", w.Code, tc.wantStatus, w.Body.String()) + } + if tc.wantMessage != "" && !strings.Contains(w.Body.String(), tc.wantMessage) { + t.Fatalf("expected %q in body, got %s", tc.wantMessage, w.Body.String()) + } + if got := len(fake.tunnelReqsSnapshot()) > 0; got != tc.wantTunnel { + t.Fatalf("tunnel dispatch: got %v want %v", got, tc.wantTunnel) + } + if got := len(fake.reqsSnapshot()) > 0; got != tc.wantRun { + t.Fatalf("normalized dispatch: got %v want %v", got, tc.wantRun) + } + }) + } +} + +// chatSidebandServer returns an openai.Server backed by the given tunnel +// frames for explicit passthrough+sideband fixtures. +func chatSidebandServer(frames chan *iop.ProviderTunnelFrame) (*Server, *fakeRunService) { + fake := &fakeRunService{tunnelFrames: frames} + srv := NewServer(config.EdgeOpenAIConf{}, fake, nil) + srv.SetModelCatalog([]config.ModelCatalogEntry{ + {ID: "pool-model", Providers: map[string]string{"prov-1": "served-model"}}, + }) + return srv, fake +} + +// TestChatCompletionsPassthroughSidebandStreamingExposesIOPExtension verifies +// SDD S05 for the streaming path: explicit passthrough+sideband preserves the +// provider SSE events contiguously and exposes IOP route/usage/assembled +// observations as explicit `event: iop.sideband` extension events, injected +// only at provider event boundaries. +func TestChatCompletionsPassthroughSidebandStreamingExposesIOPExtension(t *testing.T) { + roleEvent := "data: {\"choices\":[{\"delta\":{\"role\":\"assistant\"}}]}\n\n" + contentEvent := "data: {\"choices\":[{\"delta\":{\"content\":\"hello\"}}]}\n\n" + doneEvent := "data: [DONE]\n\n" + frames := make(chan *iop.ProviderTunnelFrame, 7) + frames <- &iop.ProviderTunnelFrame{ + Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_RESPONSE_START, + StatusCode: 200, + Headers: map[string]string{"Content-Type": "text/event-stream"}, + } + frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(roleEvent)} + // Split the content event so the usage frame arrives while the provider + // event is incomplete; the sideband writer must not inject there. + frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(contentEvent[:17])} + frames <- &iop.ProviderTunnelFrame{ + Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_USAGE, + Usage: &iop.Usage{InputTokens: 3, OutputTokens: 5}, + } + frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(contentEvent[17:])} + frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(doneEvent)} + frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_END, End: true} + close(frames) + srv, fake := chatSidebandServer(frames) + + req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{ + "model":"pool-model", + "stream":true, + "messages":[{"role":"user","content":"hello"}], + "metadata":{"iop_response_mode":"passthrough+sideband"} + }`)) + w := httptest.NewRecorder() + srv.handleChatCompletions(w, req) + + if w.Code != http.StatusOK { + t.Fatalf("status: got %d body=%s", w.Code, w.Body.String()) + } + if got := w.Header().Get("X-IOP-Response-Mode"); got != "passthrough+sideband" { + t.Errorf("sideband response mode header: got %q", got) + } + body := w.Body.String() + // Provider events reach the caller contiguously: sideband injection never + // splits a provider SSE event. + for _, event := range []string{roleEvent, contentEvent, doneEvent} { + if !strings.Contains(body, event) { + t.Fatalf("provider event not preserved contiguously: %q in %q", event, body) + } + } + if !strings.HasPrefix(body, "event: iop.sideband\n") { + t.Fatalf("stream must open with the IOP route sideband event, got %q", body) + } + routeIdx := strings.Index(body, `"kind":"route"`) + if routeIdx < 0 || routeIdx > strings.Index(body, roleEvent) { + t.Fatalf("route observation must precede provider events: %q", body) + } + for _, marker := range []string{ + `"object":"iop.sideband"`, + `"run_id":"run-tunnel"`, + `"node_id":"node-1"`, + `"response_mode":"passthrough+sideband"`, + } { + if !strings.Contains(body, marker) { + t.Errorf("route observation missing %q: %q", marker, body) + } + } + usageIdx := strings.Index(body, `"kind":"usage"`) + contentEnd := strings.Index(body, contentEvent) + len(contentEvent) + if usageIdx < contentEnd { + t.Errorf("usage observation must wait for the provider event boundary: usage at %d, content event ends at %d", usageIdx, contentEnd) + } + if !strings.Contains(body, `"input_tokens":3`) || !strings.Contains(body, `"output_tokens":5`) { + t.Errorf("usage observation missing token counts: %q", body) + } + assembledIdx := strings.Index(body, `"kind":"assembled"`) + if assembledIdx < 0 || assembledIdx < strings.Index(body, doneEvent) { + t.Errorf("assembled observation must close the stream: %q", body) + } + if !strings.Contains(body, `"content":"hello"`) { + t.Errorf("assembled observation missing assembled content: %q", body) + } + tunnelReqs := fake.tunnelReqsSnapshot() + if len(tunnelReqs) != 1 { + t.Fatalf("expected 1 tunnel dispatch, got %d", len(tunnelReqs)) + } + if got := tunnelReqs[0].Metadata["iop_response_mode"]; got != "passthrough+sideband" { + t.Errorf("tunnel metadata response mode: got %q", got) + } + if len(fake.reqsSnapshot()) != 0 { + t.Error("sideband mode must not use the normalized SubmitRun path") + } +} + +// TestChatCompletionsPassthroughSidebandNonStreamingWrapsProviderBody verifies +// SDD S05 for the non-streaming path: the provider JSON body is carried +// verbatim inside the iop.chat.passthrough_sideband envelope together with +// route/usage/assembled observations, and is not presented as byte identity. +func TestChatCompletionsPassthroughSidebandNonStreamingWrapsProviderBody(t *testing.T) { + providerBody := `{"id":"cmpl-1","object":"chat.completion","choices":[{"index":0,"message":{"role":"assistant","content":"hi","reasoning_content":"because"},"finish_reason":"stop"}]}` + frames := make(chan *iop.ProviderTunnelFrame, 5) + frames <- &iop.ProviderTunnelFrame{ + Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_RESPONSE_START, + StatusCode: 200, + Headers: map[string]string{"Content-Type": "application/json"}, + } + frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(providerBody[:23])} + frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(providerBody[23:])} + frames <- &iop.ProviderTunnelFrame{ + Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_USAGE, + Usage: &iop.Usage{InputTokens: 7, OutputTokens: 11}, + } + frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_END, End: true} + close(frames) + srv, fake := chatSidebandServer(frames) + + req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{ + "model":"pool-model", + "messages":[{"role":"user","content":"hello"}], + "metadata":{"iop_response_mode":"passthrough+sideband"} + }`)) + w := httptest.NewRecorder() + srv.handleChatCompletions(w, req) + + if w.Code != http.StatusOK { + t.Fatalf("status: got %d body=%s", w.Code, w.Body.String()) + } + if got := w.Header().Get("X-IOP-Response-Mode"); got != "passthrough+sideband" { + t.Errorf("sideband response mode header: got %q", got) + } + if !strings.Contains(w.Body.String(), providerBody) { + t.Fatalf("provider body must be embedded verbatim: %s", w.Body.String()) + } + var envelope struct { + Object string `json:"object"` + Sideband struct { + Route struct { + RunID string `json:"run_id"` + NodeID string `json:"node_id"` + ResponseMode string `json:"response_mode"` + ProviderStatusCode int `json:"provider_status_code"` + } `json:"route"` + Usage *struct { + InputTokens int `json:"input_tokens"` + OutputTokens int `json:"output_tokens"` + } `json:"usage"` + Assembled *struct { + Content string `json:"content"` + Reasoning string `json:"reasoning"` + BodyBytes int `json:"body_bytes"` + } `json:"assembled"` + } `json:"iop_sideband"` + ProviderStatusCode int `json:"provider_status_code"` + ProviderResponse json.RawMessage `json:"provider_response"` + } + if err := json.Unmarshal(w.Body.Bytes(), &envelope); err != nil { + t.Fatalf("envelope not valid JSON: %v body=%s", err, w.Body.String()) + } + if envelope.Object != "iop.chat.passthrough_sideband" { + t.Errorf("envelope object: got %q", envelope.Object) + } + if envelope.Sideband.Route.RunID != "run-tunnel" || envelope.Sideband.Route.NodeID != "node-1" { + t.Errorf("route observation: %+v", envelope.Sideband.Route) + } + if envelope.Sideband.Route.ResponseMode != "passthrough+sideband" || envelope.Sideband.Route.ProviderStatusCode != http.StatusOK { + t.Errorf("route observation mode/status: %+v", envelope.Sideband.Route) + } + if envelope.Sideband.Usage == nil || envelope.Sideband.Usage.InputTokens != 7 || envelope.Sideband.Usage.OutputTokens != 11 { + t.Errorf("usage observation: %+v", envelope.Sideband.Usage) + } + if envelope.Sideband.Assembled == nil || + envelope.Sideband.Assembled.Content != "hi" || + envelope.Sideband.Assembled.Reasoning != "because" || + envelope.Sideband.Assembled.BodyBytes != len(providerBody) { + t.Errorf("assembled observation: %+v", envelope.Sideband.Assembled) + } + if envelope.ProviderStatusCode != http.StatusOK { + t.Errorf("provider status: got %d", envelope.ProviderStatusCode) + } + if string(envelope.ProviderResponse) != providerBody { + t.Errorf("provider_response not verbatim:\n got: %s\nwant: %s", envelope.ProviderResponse, providerBody) + } + if len(fake.reqsSnapshot()) != 0 { + t.Error("sideband mode must not use the normalized SubmitRun path") + } +} + +// TestChatCompletionsPassthroughDoesNotExposeSideband verifies SDD S06: pure +// passthrough (omitted mode) stays byte-identical even when the tunnel emits +// usage frames, and no IOP sideband marker, event, or label reaches the caller. +func TestChatCompletionsPassthroughDoesNotExposeSideband(t *testing.T) { + providerBody := "data: {\"choices\":[{\"delta\":{\"content\":\"pure\"}}]}\n\ndata: [DONE]\n\n" + frames := make(chan *iop.ProviderTunnelFrame, 4) + frames <- &iop.ProviderTunnelFrame{ + Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_RESPONSE_START, + StatusCode: 200, + Headers: map[string]string{"Content-Type": "text/event-stream"}, + } + frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(providerBody)} + frames <- &iop.ProviderTunnelFrame{ + Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_USAGE, + Usage: &iop.Usage{InputTokens: 3, OutputTokens: 5}, + } + frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_END, End: true} + close(frames) + srv, _ := chatSidebandServer(frames) + + req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{ + "model":"pool-model", + "stream":true, + "messages":[{"role":"user","content":"hello"}] + }`)) + w := httptest.NewRecorder() + srv.handleChatCompletions(w, req) + + if w.Code != http.StatusOK { + t.Fatalf("status: got %d body=%s", w.Code, w.Body.String()) + } + if got := w.Body.String(); got != providerBody { + t.Fatalf("pure passthrough body not byte-identical:\n got: %q\nwant: %q", got, providerBody) + } + for _, marker := range []string{"iop.sideband", "iop_sideband", "iop_response_mode"} { + if strings.Contains(w.Body.String(), marker) { + t.Errorf("pure passthrough body must not contain %q", marker) + } + } + if got := w.Header().Get("X-IOP-Response-Mode"); got != "" { + t.Errorf("pure passthrough must not carry the IOP response mode header, got %q", got) + } +} + +// TestChatCompletionsTransformedModeLabelsIOPOutput verifies SDD S07: explicit +// transformed output uses the normalized path and is labeled as IOP +// transformed output in both the response header and the run metadata. +func TestChatCompletionsTransformedModeLabelsIOPOutput(t *testing.T) { + fake := &fakeRunService{events: bufferedRunEvents( + &iop.RunEvent{Type: "delta", Delta: "ok"}, + &iop.RunEvent{Type: "complete"}, + )} + srv := NewServer(config.EdgeOpenAIConf{}, fake, nil) + srv.SetModelCatalog([]config.ModelCatalogEntry{ + {ID: "pool-model", Providers: map[string]string{"prov-1": "served-model"}}, + }) + + req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{ + "model":"pool-model", + "messages":[{"role":"user","content":"hello"}], + "metadata":{"iop_response_mode":"transformed"} + }`)) + w := httptest.NewRecorder() + srv.handleChatCompletions(w, req) + + if w.Code != http.StatusOK { + t.Fatalf("status: got %d body=%s", w.Code, w.Body.String()) + } + if got := w.Header().Get("X-IOP-Response-Mode"); got != "transformed" { + t.Errorf("transformed label header: got %q", got) + } + if !strings.Contains(w.Body.String(), `"object":"chat.completion"`) { + t.Errorf("transformed output must be the normalized IOP response: %s", w.Body.String()) + } + if len(fake.tunnelReqsSnapshot()) != 0 { + t.Error("transformed mode must not use the tunnel") + } + reqs := fake.reqsSnapshot() + if len(reqs) != 1 { + t.Fatalf("expected 1 normalized SubmitRun dispatch, got %d", len(reqs)) + } + if got := reqs[0].Metadata["iop_response_mode"]; got != "transformed" { + t.Errorf("run metadata response mode: got %q", got) + } +} + +// TestChatCompletionsPassthroughDoesNotLabelTransformed verifies pure +// passthrough responses carry no IOP transformed label and stay byte-identical +// to the provider body. +func TestChatCompletionsPassthroughDoesNotLabelTransformed(t *testing.T) { + providerBody := `{"ok":true}` + frames := make(chan *iop.ProviderTunnelFrame, 3) + frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_RESPONSE_START, StatusCode: 200} + frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(providerBody)} + frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_END, End: true} + close(frames) + srv, _ := chatSidebandServer(frames) + + req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{ + "model":"pool-model", + "messages":[{"role":"user","content":"hello"}] + }`)) + w := httptest.NewRecorder() + srv.handleChatCompletions(w, req) + + if w.Code != http.StatusOK { + t.Fatalf("status: got %d body=%s", w.Code, w.Body.String()) + } + if got := w.Body.String(); got != providerBody { + t.Fatalf("pure passthrough body not byte-identical: %q", got) + } + if got := w.Header().Get("X-IOP-Response-Mode"); got != "" { + t.Errorf("pure passthrough must not carry the IOP response mode header, got %q", got) + } +} + // chatPassthroughServer returns an openai.Server whose fake service relays the // tunnel to the given provider fixture, emulating the Node relay. func chatPassthroughServer(providerURL string) (*Server, *fakeRunService) { @@ -4314,8 +4708,8 @@ func TestChatCompletionsPassthroughLegacyProviderRouteUsesTunnel(t *testing.T) { } // TestChatCompletionsExplicitNonPassthroughModeKeepsNormalizedPath verifies -// that an explicit non-passthrough response mode stays on the legacy -// normalized RunEvent path (mode semantics land with the mode-contract task). +// that an explicit transformed response mode stays on the legacy normalized +// RunEvent path. func TestChatCompletionsExplicitNonPassthroughModeKeepsNormalizedPath(t *testing.T) { fake := &fakeRunService{events: bufferedRunEvents( &iop.RunEvent{Type: "delta", Delta: "ok"}, diff --git a/apps/edge/internal/openai/stream.go b/apps/edge/internal/openai/stream.go index e39347b..4a4c766 100644 --- a/apps/edge/internal/openai/stream.go +++ b/apps/edge/internal/openai/stream.go @@ -1,6 +1,7 @@ package openai import ( + "bytes" "encoding/json" "fmt" "net/http" @@ -311,13 +312,43 @@ func (s *Server) streamChatCompletion(w http.ResponseWriter, r *http.Request, re // are written to the caller byte-identically. No IOP sideband fields or events // are added to the response body. func (s *Server) tunnelChatCompletionPassthrough(w http.ResponseWriter, r *http.Request, req chatCompletionRequest, dispatch routeDispatch, runMeta map[string]string, rawBody []byte, estimate int, contextClass string) { + handle, ok := s.submitChatCompletionTunnel(w, r, req, dispatch, runMeta, rawBody, estimate, contextClass, responseModePassthrough) + if !ok { + return + } + defer handle.Close() + s.writeProviderTunnelResponse(w, r, handle) +} + +// tunnelChatCompletionPassthroughSideband serves a Chat Completions request +// over the raw provider tunnel with the explicit IOP sideband extension +// surface (SDD S05): provider content is preserved, but the response also +// carries IOP route/usage/assembled observations and is never claimed as +// provider-original byte identity. +func (s *Server) tunnelChatCompletionPassthroughSideband(w http.ResponseWriter, r *http.Request, req chatCompletionRequest, dispatch routeDispatch, runMeta map[string]string, rawBody []byte, estimate int, contextClass string) { + handle, ok := s.submitChatCompletionTunnel(w, r, req, dispatch, runMeta, rawBody, estimate, contextClass, responseModePassthroughSideband) + if !ok { + return + } + defer handle.Close() + if req.Stream { + s.writeProviderTunnelSidebandStream(w, r, handle) + return + } + s.writeProviderTunnelSidebandResponse(w, r, handle) +} + +// submitChatCompletionTunnel dispatches a Chat Completions provider tunnel +// request with the given response mode. On failure it writes the dispatch +// error to the caller and returns ok=false. +func (s *Server) submitChatCompletionTunnel(w http.ResponseWriter, r *http.Request, req chatCompletionRequest, dispatch routeDispatch, runMeta map[string]string, rawBody []byte, estimate int, contextClass, responseMode string) (edgeservice.ProviderTunnelResult, bool) { metadata := make(map[string]string, len(runMeta)+5) for k, v := range runMeta { metadata[k] = v } metadata["openai_model"] = req.Model metadata["openai_stream"] = strconv.FormatBool(req.Stream) - metadata[responseModeMetadataKey] = responseModePassthrough + metadata[responseModeMetadataKey] = responseMode metadata["estimated_input_tokens"] = strconv.Itoa(estimate) metadata["context_class"] = contextClass @@ -345,9 +376,8 @@ func (s *Server) tunnelChatCompletionPassthrough(w http.ResponseWriter, r *http. handle, err := s.service.SubmitProviderTunnel(r.Context(), tunnelReq) if err != nil { writeError(w, http.StatusBadGateway, "node_dispatch_error", err.Error()) - return + return nil, false } - defer handle.Close() s.logger.Info("openai chat completion passthrough dispatch", zap.String("run_id", handle.Dispatch().RunID), @@ -356,12 +386,12 @@ func (s *Server) tunnelChatCompletionPassthrough(w http.ResponseWriter, r *http. zap.String("adapter", handle.Dispatch().Adapter), zap.String("target", handle.Dispatch().Target), zap.Bool("stream", req.Stream), + zap.String("response_mode", responseMode), zap.Int("estimated_input_tokens", handle.Dispatch().EstimatedInputTokens), zap.String("context_class", handle.Dispatch().ContextClass), zap.String("queue_reason", handle.Dispatch().QueueReason), ) - - s.writeProviderTunnelResponse(w, r, handle) + return handle, true } // writeProviderTunnelResponse relays ordered tunnel frames to the HTTP caller. @@ -471,6 +501,440 @@ func (s *Server) writeProviderTunnelResponse(w http.ResponseWriter, r *http.Requ } } +// Sideband extension surface (SDD S05): explicit passthrough+sideband +// responses expose IOP observations alongside provider-original content. +// Streaming responses interleave `event: iop.sideband` SSE events at provider +// event boundaries; non-streaming responses wrap the provider body in the +// iop.chat.passthrough_sideband envelope. Both are IOP extension outputs and +// are never claimed as provider-original byte identity. +const ( + sidebandSSEEventName = "iop.sideband" + sidebandEnvelopeObject = "iop.chat.passthrough_sideband" +) + +type sidebandRouteObservation struct { + RunID string `json:"run_id"` + NodeID string `json:"node_id,omitempty"` + Adapter string `json:"adapter,omitempty"` + Target string `json:"target,omitempty"` + ModelGroup string `json:"model_group,omitempty"` + ResponseMode string `json:"response_mode"` + ProviderStatusCode int `json:"provider_status_code,omitempty"` +} + +type sidebandUsageObservation struct { + InputTokens int `json:"input_tokens"` + OutputTokens int `json:"output_tokens"` +} + +type sidebandAssembledObservation struct { + Content string `json:"content,omitempty"` + Reasoning string `json:"reasoning,omitempty"` + ToolCallNames []string `json:"tool_call_names,omitempty"` + BodyBytes int `json:"body_bytes"` +} + +// sidebandObservation is one `event: iop.sideband` SSE payload on the +// streaming sideband surface. +type sidebandObservation struct { + Object string `json:"object"` + Kind string `json:"kind"` + Route *sidebandRouteObservation `json:"route,omitempty"` + Usage *sidebandUsageObservation `json:"usage,omitempty"` + Assembled *sidebandAssembledObservation `json:"assembled,omitempty"` +} + +// sidebandEnvelope is the non-streaming sideband response schema: the provider +// body is carried verbatim next to the IOP observations. +type sidebandEnvelope struct { + Object string `json:"object"` + Sideband sidebandObservations `json:"iop_sideband"` + ProviderStatusCode int `json:"provider_status_code"` + ProviderResponse json.RawMessage `json:"provider_response,omitempty"` + ProviderBody string `json:"provider_body,omitempty"` +} + +type sidebandObservations struct { + Route sidebandRouteObservation `json:"route"` + Usage *sidebandUsageObservation `json:"usage,omitempty"` + Assembled *sidebandAssembledObservation `json:"assembled,omitempty"` +} + +func sidebandRouteFromDispatch(dispatch edgeservice.RunDispatch, providerStatus int) sidebandRouteObservation { + return sidebandRouteObservation{ + RunID: dispatch.RunID, + NodeID: dispatch.NodeID, + Adapter: dispatch.Adapter, + Target: dispatch.Target, + ModelGroup: dispatch.ModelGroupKey, + ResponseMode: responseModePassthroughSideband, + ProviderStatusCode: providerStatus, + } +} + +func sidebandUsageFromFrame(frame *iop.ProviderTunnelFrame) *sidebandUsageObservation { + usage := frame.GetUsage() + if usage == nil { + return nil + } + return &sidebandUsageObservation{ + InputTokens: int(usage.GetInputTokens()), + OutputTokens: int(usage.GetOutputTokens()), + } +} + +// providerChatAssembler accumulates a human-readable view of the provider +// Chat Completions response for the sideband assembled observation. It +// tolerates non-JSON and partial payloads: whatever cannot be parsed simply +// yields an empty summary. +type providerChatAssembler struct { + streaming bool + bodyBytes int + pending []byte + content strings.Builder + reasoning strings.Builder + toolCallNames []string +} + +// providerChatDeltaEnvelope matches the provider fields the assembler +// observes; streaming uses delta, non-streaming uses message. +type providerChatDeltaEnvelope struct { + Content string `json:"content"` + ReasoningContent string `json:"reasoning_content"` + ToolCalls []struct { + Function struct { + Name string `json:"name"` + } `json:"function"` + } `json:"tool_calls"` +} + +func (a *providerChatAssembler) Write(chunk []byte) { + a.bodyBytes += len(chunk) + a.pending = append(a.pending, chunk...) + if !a.streaming { + return + } + for { + idx := bytes.IndexByte(a.pending, '\n') + if idx < 0 { + return + } + line := strings.TrimRight(string(a.pending[:idx]), "\r") + a.pending = a.pending[idx+1:] + a.consumeSSELine(line) + } +} + +func (a *providerChatAssembler) consumeSSELine(line string) { + payload, ok := strings.CutPrefix(line, "data:") + if !ok { + return + } + payload = strings.TrimSpace(payload) + if payload == "" || payload == "[DONE]" { + return + } + var chunk struct { + Choices []struct { + Delta providerChatDeltaEnvelope `json:"delta"` + } `json:"choices"` + } + if err := json.Unmarshal([]byte(payload), &chunk); err != nil { + return + } + for _, choice := range chunk.Choices { + a.consumeDelta(choice.Delta) + } +} + +func (a *providerChatAssembler) consumeDelta(delta providerChatDeltaEnvelope) { + a.content.WriteString(delta.Content) + a.reasoning.WriteString(delta.ReasoningContent) + for _, call := range delta.ToolCalls { + if name := strings.TrimSpace(call.Function.Name); name != "" { + a.toolCallNames = append(a.toolCallNames, name) + } + } +} + +func (a *providerChatAssembler) observation() *sidebandAssembledObservation { + if !a.streaming { + var resp struct { + Choices []struct { + Message providerChatDeltaEnvelope `json:"message"` + } `json:"choices"` + } + if err := json.Unmarshal(a.pending, &resp); err == nil { + for _, choice := range resp.Choices { + a.consumeDelta(choice.Message) + } + } + } + return &sidebandAssembledObservation{ + Content: a.content.String(), + Reasoning: a.reasoning.String(), + ToolCallNames: a.toolCallNames, + BodyBytes: a.bodyBytes, + } +} + +// writeProviderTunnelSidebandStream relays provider SSE bytes unchanged and +// interleaves explicit `event: iop.sideband` extension events. IOP events are +// written only at provider event boundaries (before the first body byte, +// after a "\n\n"-terminated provider event, or at end of stream) so a +// provider event is never split by sideband injection. +func (s *Server) writeProviderTunnelSidebandStream(w http.ResponseWriter, r *http.Request, handle edgeservice.ProviderTunnelResult) { + frames := handle.Stream().Frames + if frames == nil { + writeError(w, http.StatusBadGateway, "provider_tunnel_error", "tunnel stream unavailable") + return + } + flusher, _ := w.(http.Flusher) + timer := time.NewTimer(handle.WaitTimeout()) + defer timer.Stop() + + assembler := &providerChatAssembler{streaming: true} + wroteHeader := false + atEventBoundary := true + tail := "" + var pendingObservations []sidebandObservation + + writeObservation := func(obs sidebandObservation) { + obs.Object = sidebandSSEEventName + payload, err := json.Marshal(obs) + if err != nil { + return + } + fmt.Fprintf(w, "event: %s\ndata: %s\n\n", sidebandSSEEventName, payload) + if flusher != nil { + flusher.Flush() + } + } + flushObservations := func() { + for _, obs := range pendingObservations { + writeObservation(obs) + } + pendingObservations = nil + } + + defer func() { + s.logger.Info("openai chat completion sideband stream closed", + zap.String("run_id", handle.Dispatch().RunID), + zap.Bool("wrote_header", wroteHeader), + zap.Int("body_bytes", assembler.bodyBytes), + ) + }() + + for { + select { + case <-r.Context().Done(): + s.cancelRunOnHTTPGiveUp(handle.Dispatch(), r.Context().Err()) + return + case <-timer.C: + s.cancelRunOnHTTPGiveUp(handle.Dispatch(), errRunTimedOut) + if !wroteHeader { + writeError(w, http.StatusBadGateway, "run_error", errRunTimedOut.Error()) + } + return + case frame, ok := <-frames: + if !ok { + if !wroteHeader { + writeError(w, http.StatusBadGateway, "provider_tunnel_error", "tunnel stream closed before provider response") + } + return + } + switch frame.GetKind() { + case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_RESPONSE_START: + if wroteHeader { + continue + } + copyProviderResponseHeaders(w.Header(), frame.GetHeaders()) + // The sideband stream is an IOP extension surface: label it + // and drop the provider Content-Length, which no longer + // matches the extended body. + w.Header().Del("Content-Length") + w.Header().Set(responseModeHeaderName, responseModePassthroughSideband) + status := int(frame.GetStatusCode()) + if status == 0 { + status = http.StatusOK + } + w.WriteHeader(status) + wroteHeader = true + if flusher != nil { + flusher.Flush() + } + route := sidebandRouteFromDispatch(handle.Dispatch(), status) + writeObservation(sidebandObservation{Kind: "route", Route: &route}) + case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY: + body := frame.GetBody() + if len(body) == 0 { + continue + } + if !wroteHeader { + // Defensive: a body frame before response-start still + // reaches the caller instead of being dropped. + w.Header().Set(responseModeHeaderName, responseModePassthroughSideband) + w.WriteHeader(http.StatusOK) + wroteHeader = true + } + if _, err := w.Write(body); err != nil { + // The caller is gone mid-stream; propagate cancel to the Node. + s.sendCancelRun(handle.Dispatch()) + return + } + assembler.Write(body) + if flusher != nil { + flusher.Flush() + } + tail = sseTail(tail, body) + atEventBoundary = strings.HasSuffix(tail, "\n\n") + if atEventBoundary { + flushObservations() + } + case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_ERROR: + msg := frame.GetError() + if msg == "" { + msg = "provider tunnel failed" + } + if !wroteHeader { + writeError(w, http.StatusBadGateway, "provider_tunnel_error", msg) + return + } + // Status/headers are already committed; the response is + // truncated and the caller observes the broken stream. + s.logger.Warn("openai sideband tunnel error after response start", + zap.String("run_id", handle.Dispatch().RunID), + zap.String("error", msg), + ) + return + case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_USAGE: + usage := sidebandUsageFromFrame(frame) + if usage == nil { + continue + } + pendingObservations = append(pendingObservations, sidebandObservation{Kind: "usage", Usage: usage}) + if wroteHeader && atEventBoundary { + flushObservations() + } + case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_END: + if !wroteHeader { + writeError(w, http.StatusBadGateway, "provider_tunnel_error", "tunnel ended before provider response") + return + } + if !atEventBoundary { + // The provider stream ended mid-event; terminate it so the + // trailing IOP events stay well-formed SSE. + fmt.Fprint(w, "\n\n") + } + flushObservations() + writeObservation(sidebandObservation{Kind: "assembled", Assembled: assembler.observation()}) + return + } + } + } +} + +// sseTail keeps the last two bytes of the relayed provider stream so the +// sideband writer can detect "\n\n" event boundaries across chunk splits. +func sseTail(tail string, chunk []byte) string { + combined := tail + string(chunk) + if len(combined) > 2 { + return combined[len(combined)-2:] + } + return combined +} + +// writeProviderTunnelSidebandResponse buffers the provider response and +// answers with the iop.chat.passthrough_sideband envelope: provider status is +// preserved, the provider body is carried verbatim inside the envelope, and +// IOP route/usage/assembled observations sit alongside it. +func (s *Server) writeProviderTunnelSidebandResponse(w http.ResponseWriter, r *http.Request, handle edgeservice.ProviderTunnelResult) { + frames := handle.Stream().Frames + if frames == nil { + writeError(w, http.StatusBadGateway, "provider_tunnel_error", "tunnel stream unavailable") + return + } + timer := time.NewTimer(handle.WaitTimeout()) + defer timer.Stop() + + assembler := &providerChatAssembler{} + var body bytes.Buffer + providerStatus := 0 + var usage *sidebandUsageObservation + + for { + select { + case <-r.Context().Done(): + s.cancelRunOnHTTPGiveUp(handle.Dispatch(), r.Context().Err()) + return + case <-timer.C: + s.cancelRunOnHTTPGiveUp(handle.Dispatch(), errRunTimedOut) + writeError(w, http.StatusBadGateway, "run_error", errRunTimedOut.Error()) + return + case frame, ok := <-frames: + if !ok { + writeError(w, http.StatusBadGateway, "provider_tunnel_error", "tunnel stream closed before provider response") + return + } + switch frame.GetKind() { + case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_RESPONSE_START: + if providerStatus != 0 { + continue + } + providerStatus = int(frame.GetStatusCode()) + if providerStatus == 0 { + providerStatus = http.StatusOK + } + case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY: + body.Write(frame.GetBody()) + assembler.Write(frame.GetBody()) + case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_ERROR: + msg := frame.GetError() + if msg == "" { + msg = "provider tunnel failed" + } + writeError(w, http.StatusBadGateway, "provider_tunnel_error", msg) + return + case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_USAGE: + if u := sidebandUsageFromFrame(frame); u != nil { + usage = u + } + case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_END: + if providerStatus == 0 && body.Len() == 0 { + writeError(w, http.StatusBadGateway, "provider_tunnel_error", "tunnel ended before provider response") + return + } + if providerStatus == 0 { + providerStatus = http.StatusOK + } + envelope := sidebandEnvelope{ + Object: sidebandEnvelopeObject, + Sideband: sidebandObservations{ + Route: sidebandRouteFromDispatch(handle.Dispatch(), providerStatus), + Usage: usage, + Assembled: assembler.observation(), + }, + ProviderStatusCode: providerStatus, + } + raw := body.Bytes() + if json.Valid(raw) { + envelope.ProviderResponse = json.RawMessage(raw) + } else if len(raw) > 0 { + envelope.ProviderBody = string(raw) + } + w.Header().Set(responseModeHeaderName, responseModePassthroughSideband) + writeJSON(w, providerStatus, envelope) + s.logger.Info("openai chat completion sideband response", + zap.String("run_id", handle.Dispatch().RunID), + zap.Int("provider_status", providerStatus), + zap.Int("body_bytes", body.Len()), + ) + return + } + } + } +} + // hopByHopResponseHeaders are transport-level headers owned by each hop; they // are not copied from the provider response to the caller response. var hopByHopResponseHeaders = map[string]struct{}{