sync: edge openai raw tunnel sideband passthrough implementation, stream and test updates, roadmap phase sync

This commit is contained in:
toki 2026-07-08 07:17:40 +09:00
parent fc16a1e05c
commit 1bf5eec31a
8 changed files with 1518 additions and 22 deletions

View file

@ -16,7 +16,7 @@ IOP의 OpenAI-compatible, A2A, IOP native 입력 표면에서 들어온 요청
완료, 검토중, 진행중, 계획, 스케치 순서로 두어 아래로 갈수록 미래 작업에 가까워지게 정렬한다.
스케치 Milestone은 아직 구현 가능한 계획이 아니므로 계획 Milestone보다 아래에 둔다.
- [계획] OpenAI-compatible Raw Tunnel과 Sideband Passthrough
- [진행중] OpenAI-compatible Raw Tunnel과 Sideband Passthrough
- 경로: [openai-compatible-raw-tunnel-sideband-passthrough](milestones/openai-compatible-raw-tunnel-sideband-passthrough.md)
- 요약: OpenAI-compatible provider 응답을 기존 Edge-Node proto-socket 위 lossless raw tunnel로 전달하고, 기본값은 provider-original `passthrough`로 두며, 요청이 명시한 경우에만 `passthrough+sideband` 또는 `transformed`를 사용한다.

View file

@ -13,7 +13,7 @@ OpenAI-compatible 추론 엔진(vLLM, SGLang, Lemonade, OpenAI-compatible API pr
## 상태
[계획]
[진행중]
## 승격 조건
@ -52,9 +52,9 @@ OpenAI-compatible 추론 엔진(vLLM, SGLang, Lemonade, OpenAI-compatible API pr
OpenAI-compatible provider의 원본 응답을 정규화/재조립하지 않고 기존 Edge-Node socket으로 전달하는 핵심 경로를 묶는다.
- [ ] [wire-contract] Edge-Node wire에 tunnel request/frame 메시지를 추가하고 status, headers, body bytes, sequence, end, error, usage trailer 후보를 정의한다. 검증: proto 생성과 contract test에서 raw body가 `RunEvent.delta``events.Bus`를 통하지 않음을 확인한다.
- [ ] [node-relay] Node는 OpenAI-compatible provider HTTP/SSE response를 열고 provider가 보낸 chunk를 tunnel frame으로 relay한다. 검증: vLLM Qwen `reasoning_content` fixture와 Gemma content fixture가 byte comparison으로 동일하게 왕복한다.
- [ ] [edge-stream] Edge OpenAI-compatible handler는 tunnel status/headers/body chunks를 caller에게 그대로 write하고 pure passthrough를 기본값으로 적용한다. 검증: streaming/non-streaming Chat Completions fixture가 provider 원본과 byte-identical하게 반환된다.
- [x] [wire-contract] Edge-Node wire에 tunnel request/frame 메시지를 추가하고 status, headers, body bytes, sequence, end, error, usage trailer 후보를 정의한다. 검증: proto 생성과 contract test에서 raw body가 `RunEvent.delta``events.Bus`를 통하지 않음을 확인한다.
- [x] [node-relay] Node는 OpenAI-compatible provider HTTP/SSE response를 열고 provider가 보낸 chunk를 tunnel frame으로 relay한다. 검증: vLLM Qwen `reasoning_content` fixture와 Gemma content fixture가 byte comparison으로 동일하게 왕복한다.
- [x] [edge-stream] Edge OpenAI-compatible handler는 tunnel status/headers/body chunks를 caller에게 그대로 write하고 pure passthrough를 기본값으로 적용한다. 검증: streaming/non-streaming Chat Completions fixture가 provider 원본과 byte-identical하게 반환된다.
### Epic: [response-modes] Request-selected Modes

View file

@ -0,0 +1,197 @@
<!-- task=m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics plan=0 tag=API -->
# Code Review Reference - API
> **[IMPLEMENTING AGENT — READ FIRST] Filling in this file is the mandatory final step of implementation.**
> The task is NOT complete until every implementation-owned section below is filled in.
> Complete the `구현 체크리스트`; the final checklist item is mandatory before saving.
> Fill implementation-owned sections, then stop with active files in place and report ready for review.
> If implementation is blocked by a selected Milestone `구현 잠금 > 결정 필요` item, fill `사용자 리뷰 요청` with linked evidence and stop with active files in place; code-review decides whether to write `USER_REVIEW.md`. Environment/secret/service blockers, generic scope changes, repeated failures, and evidence gaps that a follow-up agent can close are normal follow-up issues, not user-review blockers by themselves.
> Do not ask the user directly, present choices in chat, or call `request_user_input` during implementation; record only Milestone lock decisions in `사용자 리뷰 요청` and stop for code-review.
> Finalization (`코드리뷰 결과`, log rename, `complete.log`, archive moves, `코드리뷰 전용 체크리스트`) is review-agent-only, even after compaction/resume.
> Follow the ownership table at the bottom of this file for which sections you own.
## 개요
date=2026-07-08
task=m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics, plan=0, tag=API
## Roadmap Targets
- Milestone: `agent-roadmap/phase/routing-policy-model-orchestration/milestones/openai-compatible-raw-tunnel-sideband-passthrough.md`
- Milestone link: [Milestone 문서](agent-roadmap/phase/routing-policy-model-orchestration/milestones/openai-compatible-raw-tunnel-sideband-passthrough.md)
- Task ids:
- `mode-contract`: OpenAI-compatible request metadata/config 계약에 `metadata.iop_response_mode`와 `passthrough`, `passthrough+sideband`, `transformed` 모드를 정의하고 omitted mode는 `passthrough`로 해석한다.
- `sideband-overlay`: `passthrough+sideband`는 provider original stream과 IOP sideband를 명시적 IOP extension stream/schema로 함께 노출하되 pure `passthrough` body에는 IOP field/event를 주입하지 않는다.
- `transform-output`: `transformed`는 provider raw stream을 IOP schema/output으로 수정할 수 있지만 byte-identical provider stream으로 표시하지 않는다.
- Completion mode: check-on-pass
## Archive Evidence Snapshot
- `agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/01_node_relay/complete.log`: PASS. Node relay가 OpenAI-compatible provider response를 기존 Edge-Node socket tunnel frame으로 전달하는 선행 작업을 완료했다.
- `agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/02+01_edge_stream/complete.log`: PASS. Edge stream pure passthrough 기본값과 streaming/non-streaming byte identity 경로가 완료되어 이번 subtask의 predecessor `02`를 만족한다.
- 더 자세한 선행 맥락이 필요할 때만 위 두 `complete.log`를 좁게 읽는다. `agent-task/archive/**`를 광범위하게 검색하지 않는다.
## 이 파일을 읽는 리뷰 에이전트에게
> **[REVIEW AGENT ONLY]** 아래 종결 절차는 코드리뷰 에이전트 전용이다. 구현 에이전트는 이 섹션을 실행하지 않는다.
각 항목의 구현을 실제 소스 파일과 대조하고, `검증 결과` 섹션의 출력이 코드와 일치하는지 확인하세요.
리뷰 완료는 아래 순서까지 끝난 상태를 의미합니다.
1. 판정을 append한다.
2. `CODE_REVIEW-cloud-G06.md` -> `code_review_cloud_G06_N.log`, `PLAN-cloud-G06.md` -> `plan_cloud_G06_M.log`로 아카이브한다.
3. PASS이면 `complete.log` 작성 후 active task 디렉터리를 `agent-task/archive/YYYY/MM/m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics/`로 이동한다. WARN/FAIL이면 user-review gate를 확인한 뒤 다음 active plan/review 파일 또는 `USER_REVIEW.md`를 작성한다. `USER_REVIEW.md`가 연결된 Milestone 결정으로 완료/PASS 해소되면 code-review가 `USER_REVIEW.md`를 해소 상태로 갱신하고 `complete.log` 작성 후 archive 이동한다.
4. PASS이고 task group이 `m-<milestone-slug>`이면 완료 이벤트 메타데이터를 보고한다. roadmap 상태 체크와 `update-roadmap` 호출은 런타임 책임이다.
5. 적용 가능한 `코드리뷰 전용 체크리스트` 항목을 최종 `.log` 위치에서 체크한 뒤 보고한다.
---
## 구현 항목별 완료 여부
| 항목 | 완료 여부 |
|------|---------|
| [API-1] Mode Contract | [x] |
| [API-2] Sideband Overlay | [x] |
| [API-3] Transform Output | [x] |
## 구현 체크리스트
- [x] `mode-contract`: official mode constants/parser/routing guard are complete, omitted mode stays pure `passthrough`, explicit sideband/transformed choose the intended non-byte-identity paths, and unknown mode returns invalid request. 검증: mode 누락, explicit sideband, explicit transformed routing test가 각각 기대 경로를 타고 unknown mode는 fail-fast error가 된다.
- [x] `sideband-overlay`: explicit `passthrough+sideband` exposes provider-original content plus IOP observation through a contracted extension stream/schema, while pure `passthrough` contains no IOP sideband field/event. 검증: pure passthrough 응답에 IOP sideband가 없고 opt-in sideband mode에서 같은 response stream 또는 계약된 extension surface로 route/provider/assembled observation이 확인된다.
- [x] `transform-output`: explicit `transformed` response is labeled/metadata-marked as IOP transformed output and is excluded from provider byte-identity assertions. 검증: transformed response가 IOP output으로 라벨링되고 original passthrough 검증과 분리된다.
- [x] Final verification command passes: `go test -count=1 ./apps/edge/internal/service ./apps/edge/internal/transport ./apps/edge/internal/openai`.
- [x] `git diff --check` passes.
- [x] CODE_REVIEW-*-G??.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채운다. 이 항목이 완료되기 전에는 구현이 완료된 것이 아니다.
## 코드리뷰 전용 체크리스트
> **[REVIEW AGENT ONLY]** 이 체크리스트는 코드리뷰 에이전트만 사용한다.
> 구현 에이전트는 이 섹션을 수정하거나 체크하지 않는다.
- [x] `코드리뷰 결과`에 `PASS`, `WARN`, `FAIL` 중 하나의 판정을 append한다.
- [x] 판정과 `차원별 평가`, Required/Suggested/Nit 분류가 서로 일치한다.
- [x] active `CODE_REVIEW-*-G??.md`를 `code_review_cloud_G06_N.log`로 아카이브한다.
- [x] active `PLAN-*-G??.md`를 `plan_cloud_G06_M.log`로 아카이브한다.
- [x] `.gitignore`의 Agent-Ops 관리 block이 `agent-task/**/*.md`와 `agent-task/**/*.log`를 unignore하고 `agent-roadmap/current.md`를 ignore하는지 확인한다.
- [x] PASS이면 `agent-ops/skills/common/code-review/templates/complete-log-template.md` 기준으로 `complete.log`를 작성하고 active `.md` 파일을 남기지 않는다.
- [x] PASS이면 active task 디렉터리 `agent-task/m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics/`를 `agent-task/archive/YYYY/MM/m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics/`로 이동하고 최종 archive 경로에서 이 체크리스트를 갱신한다.
- [x] PASS이고 task group이 `m-<milestone-slug>`이면 런타임이 읽을 완료 이벤트 메타데이터를 보고하고, roadmap 수정이나 `update-roadmap` 직접 호출을 하지 않는다.
- [x] PASS split 작업이면 이동 후 빈 active parent `agent-task/m-openai-compatible-raw-tunnel-sideband-passthrough/`를 제거하거나, 남은 sibling/file이 있어 유지했다고 확인한다.
- [ ] WARN/FAIL이고 user-review gate가 트리거되지 않았으면 다음 active `PLAN-{build_lane}-GNN.md`와 `CODE_REVIEW-{review_lane}-GNN.md`를 작성하고 `complete.log`를 작성하지 않는다.
- [ ] USER_REVIEW이면 `agent-ops/skills/common/code-review/templates/user-review-template.md` 기준으로 `USER_REVIEW.md`를 작성하고 active `PLAN-*.md`, `CODE_REVIEW-*.md`, `complete.log`를 남기지 않는다.
- [ ] USER_REVIEW가 연결된 Milestone 결정으로 완료/PASS 해소되면 `USER_REVIEW.md`를 해소 상태로 갱신하고 `complete.log`를 작성한 뒤 task directory를 archive로 이동한다.
## 계획 대비 변경 사항
- `apps/edge/internal/service`와 `apps/edge/internal/transport`는 수정하지 않았다. 계획이 조건부로만 허용한 항목이며, 기존 `ProviderTunnelFrame`의 `usage`/`status_code`/`headers`와 `handle.Dispatch()` 메타데이터만으로 sideband route/usage/assembled observation을 구성할 수 있었다.
- 계획의 API-2 테스트 목록에 없던 `TestChatCompletionsPassthroughSidebandNonStreamingWrapsProviderBody`를 추가했다. 계획 본문이 요구한 non-streaming sideband envelope 계약("wrap or annotate only under an explicit IOP extension schema")을 검증하기 위한 것으로, streaming 테스트만으로는 envelope schema가 커버되지 않는다.
- 검증 명령은 계획 그대로 실행했고 대체하지 않았다. 참고: `go vet ./apps/edge/internal/openai`는 이번 변경과 무관한 기존 이슈(`input_estimator.go:95` `WriteByte` signature)를 보고하지만, 이는 계약된 검증 명령이 아니며 이번 diff에 포함되지 않은 파일이라 수정하지 않았다(요청 범위 초과 방지).
## 주요 설계 결정
- **Sideband streaming surface**: explicit `passthrough+sideband` streaming 응답은 provider SSE bytes를 그대로 relay하면서 named SSE event `event: iop.sideband`로 IOP observation을 interleave한다. injection은 provider event 경계(스트림 시작, 누적 스트림이 `\n\n`으로 끝난 지점, 스트림 종료)에서만 일어나며, 경계 판정은 마지막 2바이트 tail 추적(`sseTail`)으로 chunk 분할에 안전하게 처리한다. 경계가 아닐 때 도착한 usage observation은 pending queue에 쌓았다가 다음 경계에서 방출한다. provider stream이 event 중간에서 끝나면 `\n\n`으로 종결한 뒤 trailing IOP event를 붙인다 — sideband mode는 byte identity를 주장하지 않으므로 허용된다.
- **Sideband observation 종류**: `route`(run_id, node_id, adapter, target, model_group, response_mode, provider_status_code) — 헤더 직후 방출, `usage`(input/output tokens) — tunnel USAGE frame에서 파생, `assembled`(content, reasoning, tool_call_names, body_bytes) — 관대한(tolerant) `providerChatAssembler`가 provider SSE `delta` 또는 non-streaming `message`에서 조립하며 파싱 실패 시 빈 요약으로 무해하게 동작한다.
- **Sideband non-streaming envelope**: `{"object":"iop.chat.passthrough_sideband","iop_sideband":{route,usage,assembled},"provider_status_code":...,"provider_response":<원본 JSON verbatim>}` 스키마로 wrap한다. provider body가 JSON이 아니면 `provider_body` string field로 담는다. provider HTTP status는 envelope 응답의 status로 보존한다.
- **IOP extension 라벨 헤더**: `X-IOP-Response-Mode` 응답 헤더를 explicit IOP extension 출력에만 부여한다 — `transformed` 응답과 `passthrough+sideband` 응답에 설정하고, pure `passthrough`는 provider-original 헤더를 유지하며 절대 부여하지 않는다. transformed는 추가로 run metadata `iop_response_mode=transformed`를 정규화해 남긴다. sideband streaming 응답에서는 body가 확장되므로 provider `Content-Length` 헤더를 제거한다.
- **경로 구성**: `submitChatCompletionTunnel` 공용 helper가 response mode를 tunnel metadata에 실어 dispatch하고, pure passthrough byte writer(`writeProviderTunnelResponse`)는 변경 없이 유지했다. sideband는 `writeProviderTunnelSidebandStream`(streaming)과 `writeProviderTunnelSidebandResponse`(non-streaming)로 분리했다. sideband mode가 non-tunnel(CLI/ollama) route에 지정되면 기존과 동일하게 normalized path로 내려간다(제공자 원본 스트림이 없어 sideband overlay 대상이 아님).
## 사용자 리뷰 요청
_기본값은 `없음`이다. 구현 중 새 결정이 필요해 보여도 직접 질문하거나 선택지를 제시하거나 `request_user_input`을 호출하지 않는다. 이 섹션은 선택된 Milestone `구현 잠금 > 결정 필요` 항목이 실구현을 차단할 때만 채운다. 외부 환경/secret/서비스 준비, 검증 증거 공백, 반복 실패, 일반 범위 조정은 사용자 리뷰 요청이 아니며 `검증 결과`, `계획 대비 변경 사항`, 또는 code-review의 일반 follow-up plan으로 처리한다._
- 상태: 없음
- 사유 유형: 없음
- 연결 대상: 없음
- 결정 필요: 없음
- 차단 근거: 없음
- 실행한 검증/명령: 없음
- 자동 후속 불가 이유: 없음
- 재개 조건: 없음
## 리뷰어를 위한 체크포인트
- Unknown `metadata.iop_response_mode` fails before dispatch and cannot silently fall back to normalized path.
- Pure `passthrough` remains byte-identical and does not expose any IOP sideband marker/event.
- `passthrough+sideband` is explicit opt-in and is not represented as provider-original byte identity.
- `transformed` output has a testable IOP transformed label/metadata and remains separated from passthrough byte-identity tests.
- The review evidence maps exactly to Roadmap Targets `mode-contract`, `sideband-overlay`, and `transform-output`.
## 검증 결과
_구현 에이전트가 각 중간 검증 및 최종 검증 명령 실행 후 출력을 여기에 붙여 넣는다._
필수 규칙:
- 검증 명령은 고정된 계약이다. 임의로 대체하지 않는다.
- 대체가 필요하면 `계획 대비 변경 사항`에 이유와 대체 명령을 기록한다.
- `검증 결과`에는 실제 stdout/stderr를 붙여 넣는다.
- 사용자 리뷰 요청으로 명령을 끝까지 실행하지 못했다면 `사용자 리뷰 요청`에 실행한 명령, 실제 출력, 미실행 명령의 사유를 기록한다.
- mobile/UI hang, timeout, 또는 2분 무진행은 blind retry를 중단하고 focused rerun 명령과 screenshot/window/UI-tree evidence path를 남기며, 불가능하면 정확한 사유를 남긴다.
### API-1 중간 검증
```bash
$ go test -count=1 ./apps/edge/internal/openai -run 'TestChatCompletionsResponseModeRouting|TestChatCompletionsProviderPoolDispatch|TestChatCompletionsExplicitNonPassthroughModeKeepsNormalizedPath'
ok iop/apps/edge/internal/openai 0.009s
```
### API-2 중간 검증
```bash
$ go test -count=1 ./apps/edge/internal/openai -run 'TestChatCompletionsPassthroughSideband|TestChatCompletionsPassthroughDoesNotExposeSideband|TestChatCompletionsResponseModeRouting|TestChatCompletionsPassthroughStreamingByteIdentity|TestChatCompletionsPassthroughNonStreamingByteIdentity'
ok iop/apps/edge/internal/openai 0.022s
```
`-v` 실행에서 개별 테스트 확인: `TestChatCompletionsResponseModeRouting`(4 subtests), `TestChatCompletionsPassthroughSidebandStreamingExposesIOPExtension`, `TestChatCompletionsPassthroughSidebandNonStreamingWrapsProviderBody`, `TestChatCompletionsPassthroughDoesNotExposeSideband`, `TestChatCompletionsPassthroughNonStreamingByteIdentity`, `TestChatCompletionsPassthroughStreamingByteIdentity` 전부 PASS.
### API-3 중간 검증
```bash
$ go test -count=1 ./apps/edge/internal/openai -run 'TestChatCompletionsTransformedModeLabelsIOPOutput|TestChatCompletionsPassthroughDoesNotLabelTransformed|TestChatCompletionsExplicitNonPassthroughModeKeepsNormalizedPath|TestChatCompletionsPassthroughNonStreamingByteIdentity|TestChatCompletionsPassthroughStreamingByteIdentity'
ok iop/apps/edge/internal/openai 0.013s
```
### 최종 검증
```bash
$ go test -count=1 ./apps/edge/internal/service ./apps/edge/internal/transport ./apps/edge/internal/openai
ok iop/apps/edge/internal/service 0.710s
ok iop/apps/edge/internal/transport 2.039s
ok iop/apps/edge/internal/openai 1.580s
$ git diff --check
(출력 없음: clean, exit code 0)
```
---
> **[IMPLEMENTING AGENT — BEFORE SAVING] Have you filled in every implementation-owned section: completion table, implementation checklist, changes from plan, design decisions, and verification output?**
> If anything is blank, go back and fill it in before saving this file.
> Leave review-agent-only sections unchanged.
## Ownership
| Section | Owner | Note |
|---------|-------|------|
| Header comment, 개요, 리뷰 에이전트 지시 | Fixed at stub creation | Implementing agent must not modify or execute these. |
| Roadmap Targets | Fixed at stub creation from plan | Implementing agent must not modify; code-review copies it into `complete.log` as `Roadmap Completion` only on PASS. |
| Archive Evidence Snapshot | Fixed at stub creation from plan | Implementing agent uses it as default prior-loop context; read only the specific archive files cited there when more detail is required. |
| 구현 항목별 완료 여부 | Implementing agent checks boxes only | Item names stay fixed. |
| 구현 체크리스트 | Implementing agent checks boxes only | Text/order stay fixed; final checkbox is mandatory. |
| 코드리뷰 전용 체크리스트 | Review agent only | Implementing agent must not modify or check this section. |
| 계획 대비 변경 사항, 주요 설계 결정 | Implementing agent | Replace placeholder text with actual content. |
| 사용자 리뷰 요청 | Implementing agent | Keep `상태: 없음` unless a selected Milestone lock decision blocks implementation. |
| 리뷰어를 위한 체크포인트 | Fixed at stub creation | Pre-filled from plan. |
| 검증 결과 | Implementing agent | Fill actual command output only; command changes require `계획 대비 변경 사항`. |
| 코드리뷰 결과 | Review agent appends | Not included in stub. |
## 코드리뷰 결과
- 종합 판정: PASS
- 차원별 평가:
- Correctness: Pass
- Completeness: Pass
- Test coverage: Pass
- API contract: Pass
- Code quality: Pass
- Implementation deviation: Pass
- Verification trust: Pass
- Spec conformance: Pass
- 발견된 문제: 없음
- 다음 단계: PASS. `complete.log`를 작성하고 active task directory를 archive로 이동한다. `m-openai-compatible-raw-tunnel-sideband-passthrough` completion event metadata를 보고하며 roadmap은 code-review에서 직접 수정하지 않는다.

View file

@ -0,0 +1,47 @@
# Complete - m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics
## 완료 일시
2026-07-08
## 요약
OpenAI-compatible Chat Completions response mode semantics를 구현한 1회차 루프가 PASS로 종료되었다.
## 루프 이력
| Plan | Review | Verdict | 메모 |
|------|--------|---------|------|
| `plan_cloud_G06_0.log` | `code_review_cloud_G06_0.log` | PASS | `passthrough`, `passthrough+sideband`, `transformed` routing/output semantics와 SDD S05-S07/S11 evidence가 충족되었다. |
## 구현/정리 내용
- `metadata.iop_response_mode` parser와 공식 mode routing guard를 추가하고 unknown mode fail-fast를 검증했다.
- explicit `passthrough+sideband` provider tunnel path를 추가해 streaming SSE sideband event와 non-streaming IOP envelope를 제공했다.
- pure `passthrough` byte identity와 sideband/label 부재를 유지했다.
- explicit `transformed` normalized output에 `X-IOP-Response-Mode: transformed` 라벨과 run metadata를 남겼다.
- handler/service/transport/openai 및 Edge package 테스트로 response mode semantics와 기존 Edge quick check를 검증했다.
## 최종 검증
- `go test -count=1 ./apps/edge/internal/service ./apps/edge/internal/transport ./apps/edge/internal/openai` - PASS; `service`, `transport`, `openai` 패키지가 모두 `ok`로 완료되었다.
- `git diff --check` - PASS; 출력 없음.
- `go test -count=1 ./apps/edge/...` - PASS; `apps/edge` 하위 전체 Go package가 모두 `ok`로 완료되었다.
## Roadmap Completion
- Milestone: `agent-roadmap/phase/routing-policy-model-orchestration/milestones/openai-compatible-raw-tunnel-sideband-passthrough.md`
- Milestone link: [Milestone 문서](agent-roadmap/phase/routing-policy-model-orchestration/milestones/openai-compatible-raw-tunnel-sideband-passthrough.md)
- Completed task ids:
- `mode-contract`: PASS; evidence=`plan_cloud_G06_0.log`, `code_review_cloud_G06_0.log`; verification=`go test -count=1 ./apps/edge/internal/openai -run 'TestChatCompletionsResponseModeRouting|TestChatCompletionsProviderPoolDispatch|TestChatCompletionsExplicitNonPassthroughModeKeepsNormalizedPath'`
- `sideband-overlay`: PASS; evidence=`plan_cloud_G06_0.log`, `code_review_cloud_G06_0.log`; verification=`go test -count=1 ./apps/edge/internal/openai -run 'TestChatCompletionsPassthroughSideband|TestChatCompletionsPassthroughDoesNotExposeSideband|TestChatCompletionsResponseModeRouting|TestChatCompletionsPassthroughStreamingByteIdentity|TestChatCompletionsPassthroughNonStreamingByteIdentity'`
- `transform-output`: PASS; evidence=`plan_cloud_G06_0.log`, `code_review_cloud_G06_0.log`; verification=`go test -count=1 ./apps/edge/internal/openai -run 'TestChatCompletionsTransformedModeLabelsIOPOutput|TestChatCompletionsPassthroughDoesNotLabelTransformed|TestChatCompletionsExplicitNonPassthroughModeKeepsNormalizedPath|TestChatCompletionsPassthroughNonStreamingByteIdentity|TestChatCompletionsPassthroughStreamingByteIdentity'`
- Not completed task ids: 없음
## 잔여 Nit
- 없음
## 후속 작업
- 없음

View file

@ -0,0 +1,364 @@
<!-- task=m-openai-compatible-raw-tunnel-sideband-passthrough/03+02_response_modes_semantics plan=0 tag=API -->
# Plan - API Response Modes Semantics
## 이 파일을 읽는 구현 에이전트에게
구현 완료의 마지막 단계는 반드시 `CODE_REVIEW-*-G??.md`의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채우는 것이다. 검증을 실행하고, 실제 stdout/stderr를 붙이고, active 파일을 그대로 둔 채 review ready를 보고한다. finalization, `complete.log` 작성, archive 이동은 code-review-skill 전용이다.
선택된 Milestone의 `구현 잠금 > 결정 필요` 항목이 실구현을 막을 때만 active review stub의 `사용자 리뷰 요청` 섹션을 정확한 근거와 함께 채우고 멈춘다. 구현 중 직접 사용자에게 질문하거나 선택지를 제시하거나 `request_user_input`을 호출하지 않는다. 환경/secret/service 준비, 일반 범위 조정, 반복 실패, follow-up agent가 닫을 수 있는 evidence gap은 사용자 리뷰 요청이 아니며 `검증 결과`나 `계획 대비 변경 사항`에 기록한다.
## 배경
현재 Milestone의 raw-tunnel epic은 완료되어 기본 omitted mode가 provider-original `passthrough`로 동작한다. response-modes epic은 이 기본 경로를 훼손하지 않으면서 `metadata.iop_response_mode`의 세 공식 모드와 opt-in sideband/transformed semantics를 닫아야 한다. 이번 세션에서 작은 선행 변경으로 mode parser와 unknown mode fail-fast test가 active diff에 들어갔으므로, 구현 에이전트는 이를 보존하고 실제 sideband/transformed 출력 계약까지 이어서 완성한다.
## 사용자 리뷰 요청 흐름
사용자 리뷰 요청은 선택된 Milestone lock decision이 실구현을 차단할 때만 active `CODE_REVIEW-*-G??.md`의 `사용자 리뷰 요청` 섹션에 기록한다. 이 섹션은 `agent-ops/skills/common/_templates/implementation-user-review-request-section.md` 형식을 그대로 따른다. 구현 중 direct user prompt, chat 선택지 제시, `request_user_input`, `USER_REVIEW.md` 직접 생성은 금지이며 code-review가 validation과 실제 `USER_REVIEW.md` 작성을 소유한다.
## Archive Evidence Snapshot
- `agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/01_node_relay/complete.log`: PASS. Node relay가 OpenAI-compatible provider response를 기존 Edge-Node socket tunnel frame으로 전달하는 선행 작업을 완료했다.
- `agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/02+01_edge_stream/complete.log`: PASS. Edge stream pure passthrough 기본값과 streaming/non-streaming byte identity 경로가 완료되어 이번 subtask의 predecessor `02`를 만족한다.
- 더 자세한 선행 맥락이 필요할 때만 위 두 `complete.log`를 좁게 읽는다. `agent-task/archive/**`를 광범위하게 검색하지 않는다.
## Roadmap Targets
- Milestone: `agent-roadmap/phase/routing-policy-model-orchestration/milestones/openai-compatible-raw-tunnel-sideband-passthrough.md`
- Milestone link: [Milestone 문서](agent-roadmap/phase/routing-policy-model-orchestration/milestones/openai-compatible-raw-tunnel-sideband-passthrough.md)
- Task ids:
- `mode-contract`: OpenAI-compatible request metadata/config 계약에 `metadata.iop_response_mode`와 `passthrough`, `passthrough+sideband`, `transformed` 모드를 정의하고 omitted mode는 `passthrough`로 해석한다.
- `sideband-overlay`: `passthrough+sideband`는 provider original stream과 IOP sideband를 명시적 IOP extension stream/schema로 함께 노출하되 pure `passthrough` body에는 IOP field/event를 주입하지 않는다.
- `transform-output`: `transformed`는 provider raw stream을 IOP schema/output으로 수정할 수 있지만 byte-identical provider stream으로 표시하지 않는다.
- Completion mode: check-on-pass
## 분석 결과
### 읽은 파일
- `agent-ops/rules/project/rules.md`
- `agent-ops/rules/common/rules-roadmap.md`
- `agent-ops/skills/common/router.md`
- `agent-ops/skills/common/plan/SKILL.md`
- `agent-ops/skills/common/_templates/implementation-user-review-request-section.md`
- `agent-ops/rules/project/domain/edge/rules.md`
- `agent-test/local/rules.md`
- `agent-test/local/edge-smoke.md`
- `agent-contract/index.md`
- `agent-contract/outer/openai-compatible-api.md`
- `agent-contract/inner/edge-node-runtime-wire.md`
- `agent-ops/rules/common/rules-agent-spec.md`
- `agent-spec/index.md`
- `agent-spec/input/openai-compatible-surface.md`
- `agent-roadmap/phase/routing-policy-model-orchestration/PHASE.md`
- `agent-roadmap/phase/routing-policy-model-orchestration/milestones/openai-compatible-raw-tunnel-sideband-passthrough.md`
- `agent-roadmap/sdd/routing-policy-model-orchestration/openai-compatible-raw-tunnel-sideband-passthrough/SDD.md`
- `apps/edge/internal/openai/chat_handler.go`
- `apps/edge/internal/openai/stream.go`
- `apps/edge/internal/openai/server_test.go`
### SDD 기준
- SDD: `agent-roadmap/sdd/routing-policy-model-orchestration/openai-compatible-raw-tunnel-sideband-passthrough/SDD.md`
- 상태: `[승인됨]`, SDD 잠금 `해제`, 사용자 리뷰 `없음`.
- S05 maps to `mode-contract` and `sideband-overlay`: explicit `passthrough+sideband` must expose provider original body plus IOP sideband observation through an explicit extension stream/schema. This drives the sideband fixture and routing/output checklist.
- S06 maps to `sideband-overlay`: pure `passthrough` must not contain IOP sideband field/event. This drives negative tests on existing pure passthrough streaming/non-streaming fixtures.
- S07 maps to `transform-output`: explicit `transformed` output must be labeled/metadata-marked as IOP transformed and excluded from byte-identity claims. This drives response metadata/label tests on normalized output.
- S11 maps to `mode-contract`: unknown `metadata.iop_response_mode` must fail fast with invalid request. The active diff already adds the parser and regression test; implementation must preserve or refine it.
### 테스트 환경 규칙
- test_env: `local`.
- `agent-test/local/rules.md` was present and read. The OpenAI-compatible Edge boundary maps to `agent-test/local/edge-smoke.md`, which was present and read.
- Applied verification source: deterministic Go handler/service/transport tests in this checkout, because the requested work is contract-level Edge behavior and the package tests exercise `/v1/chat/completions` handler fixtures without needing a live Edge process or external provider.
- Verified command availability/layout before writing this plan:
- `go test -count=1 ./apps/edge/internal/service ./apps/edge/internal/transport ./apps/edge/internal/openai` passes in the current checkout.
- Required verification does not leave the current checkout. No remote runner, secret, external provider, Docker service, emulator, or long-running runtime preflight is required.
- Fresh execution matters for mode routing and stream behavior, so Go test cache output is not acceptable; use `-count=1`.
### 테스트 커버리지 공백
- `mode-contract`: omitted mode pure passthrough and explicit transformed path already have tests; the active diff adds explicit `passthrough`, `passthrough+sideband`, `transformed`, and unknown mode routing coverage. Remaining gap: contract/spec/doc parity is intentionally deferred to `contract-docs`, but code fixtures must use the final mode names.
- `sideband-overlay`: existing pure passthrough byte-identity tests cover no mutation by body equality, and `stream.go:465` ignores usage frames in pure passthrough. Missing: opt-in `passthrough+sideband` extension schema/stream test and explicit negative assertion that pure passthrough contains no IOP sideband marker/event.
- `transform-output`: existing transformed-mode tests cover normalized tool/reasoning/strict output behavior. Missing: response label/metadata evidence that transformed output is not provider-original byte identity and is separated from passthrough assertions.
### 심볼 참조
- Removed/renamed symbol in active diff: `passthroughResponseMode` removed.
- `rg --sort path -n "passthroughResponseMode|parseResponseMode|responseModePassthroughSideband|responseModeTransformed|iop_response_mode" apps/edge/internal/openai` found no remaining `passthroughResponseMode` references.
- Current `parseResponseMode` references: `apps/edge/internal/openai/chat_handler.go:47`, definition at `apps/edge/internal/openai/chat_handler.go:283`.
- `iop_response_mode` test fixtures are concentrated in `apps/edge/internal/openai/server_test.go` and should continue to use the exact SDD mode strings.
### 분할 판단
- Split decision policy was evaluated before choosing this plan.
- Shared task group: `agent-task/m-openai-compatible-raw-tunnel-sideband-passthrough`.
- Existing completed predecessors:
- `01_node_relay`: satisfied by archived `agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/01_node_relay/complete.log`.
- `02+01_edge_stream`: satisfied by archived `agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/02+01_edge_stream/complete.log`.
- This subtask directory is `03+02_response_modes_semantics`; runtime dependency source of truth is predecessor `02`.
- A further split between `mode-contract`, `sideband-overlay`, and `transform-output` was considered. They share the same public response-mode entry point, the same handler tests, and the same SDD contract boundary; splitting would risk temporary contradictory mode semantics. The scope stays within one primary ownership boundary, `apps/edge/internal/openai`, with optional narrow service/transport touch only if sideband observation needs tunnel result metadata.
### 범위 결정 근거
- Do not change Node relay protocol or generated proto unless implementation proves the current `ProviderTunnelFrame` fields cannot carry the required sideband observation. Existing `wire-contract` and `node-relay` tasks are complete.
- Do not implement Responses API raw tunnel parity here; the Milestone context says Chat Completions first and Responses parity hardening is follow-up.
- Do not change routing score, provider qualification, billing/chargeback, raw payload retention policy, or normalized path deprecation.
- Do not inject IOP sideband into pure `passthrough` body; the only allowed mixed response body is explicit `passthrough+sideband`.
- Do not update agent-contract/spec/docs as completion evidence here beyond code comments/test fixture names; `contract-docs` is a separate roadmap Task.
### 빌드 등급
- `cloud-G06`: behavior is promised by SDD/outer contract beyond a single file, and a plausible local-only implementation could pass simple handler tests while mislabeling sideband/transformed semantics. Scope is moderate and bounded to Edge OpenAI response handling.
## 구현 체크리스트
- [ ] `mode-contract`: official mode constants/parser/routing guard are complete, omitted mode stays pure `passthrough`, explicit sideband/transformed choose the intended non-byte-identity paths, and unknown mode returns invalid request. 검증: mode 누락, explicit sideband, explicit transformed routing test가 각각 기대 경로를 타고 unknown mode는 fail-fast error가 된다.
- [ ] `sideband-overlay`: explicit `passthrough+sideband` exposes provider-original content plus IOP observation through a contracted extension stream/schema, while pure `passthrough` contains no IOP sideband field/event. 검증: pure passthrough 응답에 IOP sideband가 없고 opt-in sideband mode에서 같은 response stream 또는 계약된 extension surface로 route/provider/assembled observation이 확인된다.
- [ ] `transform-output`: explicit `transformed` response is labeled/metadata-marked as IOP transformed output and is excluded from provider byte-identity assertions. 검증: transformed response가 IOP output으로 라벨링되고 original passthrough 검증과 분리된다.
- [ ] Final verification command passes: `go test -count=1 ./apps/edge/internal/service ./apps/edge/internal/transport ./apps/edge/internal/openai`.
- [ ] `git diff --check` passes.
- [ ] CODE_REVIEW-*-G??.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채운다. 이 항목이 완료되기 전에는 구현이 완료된 것이 아니다.
## 의존 관계 및 구현 순서
`03+02_response_modes_semantics` depends only on predecessor index `02`, satisfied by `agent-task/archive/2026/07/m-openai-compatible-raw-tunnel-sideband-passthrough/02+01_edge_stream/complete.log`. Implement in this order: `mode-contract` final check, `sideband-overlay`, then `transform-output`, because sideband/transformed routing depends on the finalized parser/dispatch semantics.
### [API-1] Mode Contract
#### 문제
`apps/edge/internal/openai/chat_handler.go:75` previously treated all unknown non-`passthrough` modes as normalized path. SDD S11 requires unknown `metadata.iop_response_mode` to fail fast, and the Milestone `mode-contract` task requires explicit `passthrough`, `passthrough+sideband`, and `transformed` routing tests.
Before active diff:
```go
// apps/edge/internal/openai/chat_handler.go:72
if passthroughResponseMode(runMeta) && routeUsesProviderTunnel(dispatch) {
estimate := s.estimateChatInputTokens(basePrompt, runMeta, req.Tools, req.ToolChoice)
contextClass := classifyContext(estimate, s.longContextThreshold())
s.tunnelChatCompletionPassthrough(w, r, req, dispatch, runMeta, rawBody, estimate, contextClass)
return
}
```
Current active diff shape:
```go
// apps/edge/internal/openai/chat_handler.go:47
responseMode, err := parseResponseMode(runMeta)
if err != nil {
writeError(w, http.StatusBadRequest, "invalid_request_error", err.Error())
return
}
// apps/edge/internal/openai/chat_handler.go:75
if responseMode == responseModePassthrough && routeUsesProviderTunnel(dispatch) {
```
#### 해결 방법
Keep `parseResponseMode` as the single source of truth. It must trim whitespace, default empty mode to `passthrough`, accept only `passthrough`, `passthrough+sideband`, `transformed`, and return an OpenAI-compatible invalid request error before route dispatch for unknown modes.
After target:
```go
// apps/edge/internal/openai/chat_handler.go:276
const (
responseModeMetadataKey = "iop_response_mode"
responseModePassthrough = "passthrough"
responseModePassthroughSideband = "passthrough+sideband"
responseModeTransformed = "transformed"
)
func parseResponseMode(runMeta map[string]string) (string, error) {
mode := strings.TrimSpace(runMeta[responseModeMetadataKey])
if mode == "" {
return responseModePassthrough, nil
}
switch mode {
case responseModePassthrough, responseModePassthroughSideband, responseModeTransformed:
return mode, nil
default:
return "", fmt.Errorf("metadata.%s must be one of %s, %s, or %s", responseModeMetadataKey, responseModePassthrough, responseModePassthroughSideband, responseModeTransformed)
}
}
```
#### 수정 파일 및 체크리스트
- [ ] `apps/edge/internal/openai/chat_handler.go`: keep official constants, parser, and route guard.
- [ ] `apps/edge/internal/openai/server_test.go`: keep `TestChatCompletionsResponseModeRouting`; update expected sideband route after API-2 lands.
#### 테스트 작성
Write/update tests in `apps/edge/internal/openai/server_test.go`:
- `TestChatCompletionsResponseModeRouting`: omitted behavior remains covered by `TestChatCompletionsProviderPoolDispatch`; explicit `passthrough` uses tunnel; explicit `passthrough+sideband` follows API-2 path; explicit `transformed` follows normalized/transformed path; unknown `rawish` returns HTTP 400 with the allowed-mode message.
#### 중간 검증
```bash
go test -count=1 ./apps/edge/internal/openai -run 'TestChatCompletionsResponseModeRouting|TestChatCompletionsProviderPoolDispatch|TestChatCompletionsExplicitNonPassthroughModeKeepsNormalizedPath'
```
Expected: `ok`.
### [API-2] Sideband Overlay
#### 문제
`apps/edge/internal/openai/stream.go:309` implements pure tunnel passthrough and `apps/edge/internal/openai/stream.go:465` ignores usage frames so pure passthrough remains byte-identical. SDD S05/S06 require a separate explicit `passthrough+sideband` mode that can expose IOP observation through a contracted extension surface while never contaminating pure `passthrough`.
Current pure passthrough:
```go
// apps/edge/internal/openai/stream.go:309
// tunnelChatCompletionPassthrough serves a Chat Completions request over the
// raw provider tunnel (SDD S04/S06): provider status, headers, and body bytes
// are written to the caller byte-identically. No IOP sideband fields or events
// are added to the response body.
func (s *Server) tunnelChatCompletionPassthrough(... ) {
...
metadata[responseModeMetadataKey] = responseModePassthrough
...
s.writeProviderTunnelResponse(w, r, handle)
}
// apps/edge/internal/openai/stream.go:465
case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_USAGE:
// Usage frames are sideband observation candidates; pure passthrough
// never merges them into the response body (SDD D04).
continue
```
#### 해결 방법
Add an explicit sideband path selected only by `responseModePassthroughSideband`. Keep pure passthrough byte writer unchanged. For streaming Chat Completions, choose one contracted extension surface and test it end-to-end:
- Prefer SSE extension events only in `passthrough+sideband`, e.g. preserve provider `data:` body frames and append/emit `event: iop.sideband` with JSON observation for route/provider/usage/assembled summary.
- For non-streaming Chat Completions, wrap or annotate only under an explicit IOP extension schema, not as provider-original byte identity.
- Any sideband marker must include enough evidence for route/provider/assembled observation and must not appear in pure `passthrough` tests.
Example target sketch:
```go
// apps/edge/internal/openai/chat_handler.go:75
switch {
case responseMode == responseModePassthrough && routeUsesProviderTunnel(dispatch):
s.tunnelChatCompletionPassthrough(...)
return
case responseMode == responseModePassthroughSideband && routeUsesProviderTunnel(dispatch):
s.tunnelChatCompletionPassthroughSideband(...)
return
}
```
```go
// apps/edge/internal/openai/stream.go
func (s *Server) tunnelChatCompletionPassthroughSideband(...) {
// Submit the same provider tunnel request with metadata mode passthrough+sideband.
// Relay provider bytes through the contracted extension writer and emit
// only explicit IOP sideband observations.
}
```
#### 수정 파일 및 체크리스트
- [ ] `apps/edge/internal/openai/chat_handler.go`: dispatch `responseModePassthroughSideband` to the new sideband path for provider tunnel routes only.
- [ ] `apps/edge/internal/openai/stream.go`: add the explicit sideband writer without changing `writeProviderTunnelResponse` pure byte writer semantics.
- [ ] `apps/edge/internal/openai/server_test.go`: add sideband fixture tests and pure passthrough negative assertions.
- [ ] `apps/edge/internal/service` / `apps/edge/internal/transport`: touch only if the sideband observation requires existing tunnel result metadata not currently exposed to Edge.
#### 테스트 작성
Write/update tests in `apps/edge/internal/openai/server_test.go`:
- `TestChatCompletionsPassthroughSidebandStreamingExposesIOPExtension`: provider SSE fixture plus usage/route observation emits provider content and explicit IOP sideband event/schema only in `passthrough+sideband`.
- `TestChatCompletionsPassthroughDoesNotExposeSideband`: pure passthrough fixture with usage frame does not contain `iop.sideband`, `iop_response_mode`, route/provider sideband marker, or any chosen extension key.
- Update `TestChatCompletionsResponseModeRouting` sideband case to expect the sideband path instead of the temporary normalized path.
#### 중간 검증
```bash
go test -count=1 ./apps/edge/internal/openai -run 'TestChatCompletionsPassthroughSideband|TestChatCompletionsPassthroughDoesNotExposeSideband|TestChatCompletionsResponseModeRouting|TestChatCompletionsPassthroughStreamingByteIdentity|TestChatCompletionsPassthroughNonStreamingByteIdentity'
```
Expected: `ok`.
### [API-3] Transform Output
#### 문제
Existing transformed-mode fixtures use the normalized RunEvent path, but SDD S07 and the Milestone `transform-output` task require the output to be explicitly identifiable as IOP transformed output and excluded from passthrough byte-identity claims. Without a label/metadata test, future changes can accidentally treat transformed output as provider-original bytes.
Relevant current behavior:
```go
// apps/edge/internal/openai/server_test.go:4399
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{
"model":"pool-model",
"messages":[{"role":"user","content":"hello"}],
"metadata":{"iop_response_mode":"transformed"}
}`))
...
if len(fake.tunnelReqsSnapshot()) != 0 {
t.Error("explicit non-passthrough mode must not use the tunnel")
}
```
#### 해결 방법
Define a minimal transformed-output label that is visible in testable response metadata/header/body metadata without breaking OpenAI-compatible clients. Prefer a response header or metadata already accepted by local patterns over adding non-standard body fields to core OpenAI objects. The label must be present only for `transformed`, absent in pure passthrough, and not claimed as byte-identical provider output.
Example target sketch:
```go
// apps/edge/internal/openai/chat_handler.go
if responseMode == responseModeTransformed {
metadata[responseModeMetadataKey] = responseModeTransformed
// response writer or header marks output as transformed.
}
```
#### 수정 파일 및 체크리스트
- [ ] `apps/edge/internal/openai/chat_handler.go`: propagate explicit transformed mode into run metadata/output labeling.
- [ ] `apps/edge/internal/openai/stream.go`: if response headers are the chosen label surface, set them consistently for non-streaming and streaming transformed writers.
- [ ] `apps/edge/internal/openai/server_test.go`: assert transformed label/metadata and absence from pure passthrough.
#### 테스트 작성
Write/update tests in `apps/edge/internal/openai/server_test.go`:
- `TestChatCompletionsTransformedModeLabelsIOPOutput`: explicit transformed response includes the chosen IOP transformed label and uses normalized path.
- `TestChatCompletionsPassthroughDoesNotLabelTransformed`: pure passthrough response does not include the transformed label and remains byte-identical in existing passthrough fixtures.
#### 중간 검증
```bash
go test -count=1 ./apps/edge/internal/openai -run 'TestChatCompletionsTransformedModeLabelsIOPOutput|TestChatCompletionsPassthroughDoesNotLabelTransformed|TestChatCompletionsExplicitNonPassthroughModeKeepsNormalizedPath|TestChatCompletionsPassthroughNonStreamingByteIdentity|TestChatCompletionsPassthroughStreamingByteIdentity'
```
Expected: `ok`.
## 수정 파일 요약
| 파일 | 항목 |
|------|------|
| `apps/edge/internal/openai/chat_handler.go` | API-1, API-2, API-3 |
| `apps/edge/internal/openai/stream.go` | API-2, API-3 |
| `apps/edge/internal/openai/server_test.go` | API-1, API-2, API-3 |
| `apps/edge/internal/service/*` | API-2 only if sideband observation needs tunnel metadata access |
| `apps/edge/internal/transport/*` | API-2 only if sideband observation needs tunnel metadata access |
## 최종 검증
```bash
go test -count=1 ./apps/edge/internal/service ./apps/edge/internal/transport ./apps/edge/internal/openai
git diff --check
```
Expected: both commands pass. Go test cache output is not acceptable; keep `-count=1`.
모든 코드 변경 완료 후 반드시 `CODE_REVIEW-*-G??.md`의 구현 에이전트 소유 섹션을 채운다. 이 파일 작성이 구현의 마지막 단계다.

View file

@ -44,6 +44,11 @@ func (s *Server) handleChatCompletions(w http.ResponseWriter, r *http.Request) {
writeError(w, http.StatusBadRequest, "invalid_request_error", err.Error())
return
}
responseMode, err := parseResponseMode(runMeta)
if err != nil {
writeError(w, http.StatusBadRequest, "invalid_request_error", err.Error())
return
}
dispatch, ok := s.resolveRouteDispatch(req.Model)
if !ok {
@ -66,12 +71,20 @@ func (s *Server) handleChatCompletions(w http.ResponseWriter, r *http.Request) {
// OpenAI-compatible provider routes default to pure passthrough (SDD D02):
// provider status/headers/body bytes are relayed to the caller unmodified
// over the raw tunnel instead of the normalized RunEvent path.
if passthroughResponseMode(runMeta) && routeUsesProviderTunnel(dispatch) {
// over the raw tunnel instead of the normalized RunEvent path. Explicit
// passthrough+sideband keeps the provider tunnel but exposes IOP
// observations through the sideband extension surface (SDD S05).
switch {
case responseMode == responseModePassthrough && routeUsesProviderTunnel(dispatch):
estimate := s.estimateChatInputTokens(basePrompt, runMeta, req.Tools, req.ToolChoice)
contextClass := classifyContext(estimate, s.longContextThreshold())
s.tunnelChatCompletionPassthrough(w, r, req, dispatch, runMeta, rawBody, estimate, contextClass)
return
case responseMode == responseModePassthroughSideband && routeUsesProviderTunnel(dispatch):
estimate := s.estimateChatInputTokens(basePrompt, runMeta, req.Tools, req.ToolChoice)
contextClass := classifyContext(estimate, s.longContextThreshold())
s.tunnelChatCompletionPassthroughSideband(w, r, req, dispatch, runMeta, rawBody, estimate, contextClass)
return
}
messages := req.Messages
if instruction := strictOutputContractInstruction(outputPolicy); instruction != "" {
@ -100,6 +113,13 @@ func (s *Server) handleChatCompletions(w http.ResponseWriter, r *http.Request) {
metadata := chatRunMetadata(runMeta, req, outputPolicy)
metadata["estimated_input_tokens"] = strconv.Itoa(estimate)
metadata["context_class"] = contextClass
if responseMode == responseModeTransformed {
// Explicit transformed mode produces IOP-transformed output (SDD S07):
// label the response so it is never mistaken for provider-original
// byte-identical passthrough.
metadata[responseModeMetadataKey] = responseModeTransformed
w.Header().Set(responseModeHeaderName, responseModeTransformed)
}
if validation.enabled {
metadata = toolValidationAttemptMetadata(metadata, 1, "", "")
}
@ -269,17 +289,27 @@ func chatRunMetadata(runMeta map[string]string, req chatCompletionRequest, outpu
}
const (
responseModeMetadataKey = "iop_response_mode"
responseModePassthrough = "passthrough"
responseModeMetadataKey = "iop_response_mode"
responseModePassthrough = "passthrough"
responseModePassthroughSideband = "passthrough+sideband"
responseModeTransformed = "transformed"
// responseModeHeaderName labels explicit IOP extension outputs
// (passthrough+sideband, transformed). Pure passthrough responses never
// carry it: their headers stay provider-original.
responseModeHeaderName = "X-IOP-Response-Mode"
)
// passthroughResponseMode reports whether the request should use the pure
// passthrough tunnel: mode omitted (default, SDD D02) or explicitly
// "passthrough". Other explicit modes stay on the legacy normalized path
// until the mode-contract task lands their semantics.
func passthroughResponseMode(runMeta map[string]string) bool {
func parseResponseMode(runMeta map[string]string) (string, error) {
mode := strings.TrimSpace(runMeta[responseModeMetadataKey])
return mode == "" || mode == responseModePassthrough
if mode == "" {
return responseModePassthrough, nil
}
switch mode {
case responseModePassthrough, responseModePassthroughSideband, responseModeTransformed:
return mode, nil
default:
return "", fmt.Errorf("metadata.%s must be one of %s, %s, or %s", responseModeMetadataKey, responseModePassthrough, responseModePassthroughSideband, responseModeTransformed)
}
}
// routeUsesProviderTunnel reports whether the resolved dispatch targets an

View file

@ -4070,6 +4070,400 @@ func TestChatCompletionsProviderPoolDispatch(t *testing.T) {
}
}
func TestChatCompletionsResponseModeRouting(t *testing.T) {
cases := []struct {
name string
metadata string
wantStatus int
wantTunnel bool
wantRun bool
wantMessage string
}{
{
name: "explicit passthrough uses tunnel",
metadata: `"metadata":{"iop_response_mode":"passthrough"}`,
wantStatus: http.StatusOK,
wantTunnel: true,
},
{
name: "sideband mode uses tunnel with IOP extension surface",
metadata: `"metadata":{"iop_response_mode":"passthrough+sideband"}`,
wantStatus: http.StatusOK,
wantTunnel: true,
},
{
name: "transformed mode keeps normalized path",
metadata: `"metadata":{"iop_response_mode":"transformed"}`,
wantStatus: http.StatusOK,
wantRun: true,
},
{
name: "unknown mode fails fast",
metadata: `"metadata":{"iop_response_mode":"rawish"}`,
wantStatus: http.StatusBadRequest,
wantMessage: "metadata.iop_response_mode must be one of passthrough, passthrough+sideband, or transformed",
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
frames := make(chan *iop.ProviderTunnelFrame, 3)
frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_RESPONSE_START, StatusCode: 200}
frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(`{"ok":true}`)}
frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_END, End: true}
close(frames)
fake := &fakeRunService{
events: bufferedRunEvents(
&iop.RunEvent{Type: "delta", Delta: "ok"},
&iop.RunEvent{Type: "complete"},
),
tunnelFrames: frames,
}
srv := NewServer(config.EdgeOpenAIConf{}, fake, nil)
srv.SetModelCatalog([]config.ModelCatalogEntry{
{ID: "pool-model", Providers: map[string]string{"prov-1": "served-model"}},
})
body := `{
"model":"pool-model",
"messages":[{"role":"user","content":"hello"}],
` + tc.metadata + `
}`
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(body))
w := httptest.NewRecorder()
srv.handleChatCompletions(w, req)
if w.Code != tc.wantStatus {
t.Fatalf("status: got %d want %d body=%s", w.Code, tc.wantStatus, w.Body.String())
}
if tc.wantMessage != "" && !strings.Contains(w.Body.String(), tc.wantMessage) {
t.Fatalf("expected %q in body, got %s", tc.wantMessage, w.Body.String())
}
if got := len(fake.tunnelReqsSnapshot()) > 0; got != tc.wantTunnel {
t.Fatalf("tunnel dispatch: got %v want %v", got, tc.wantTunnel)
}
if got := len(fake.reqsSnapshot()) > 0; got != tc.wantRun {
t.Fatalf("normalized dispatch: got %v want %v", got, tc.wantRun)
}
})
}
}
// chatSidebandServer returns an openai.Server backed by the given tunnel
// frames for explicit passthrough+sideband fixtures.
func chatSidebandServer(frames chan *iop.ProviderTunnelFrame) (*Server, *fakeRunService) {
fake := &fakeRunService{tunnelFrames: frames}
srv := NewServer(config.EdgeOpenAIConf{}, fake, nil)
srv.SetModelCatalog([]config.ModelCatalogEntry{
{ID: "pool-model", Providers: map[string]string{"prov-1": "served-model"}},
})
return srv, fake
}
// TestChatCompletionsPassthroughSidebandStreamingExposesIOPExtension verifies
// SDD S05 for the streaming path: explicit passthrough+sideband preserves the
// provider SSE events contiguously and exposes IOP route/usage/assembled
// observations as explicit `event: iop.sideband` extension events, injected
// only at provider event boundaries.
func TestChatCompletionsPassthroughSidebandStreamingExposesIOPExtension(t *testing.T) {
roleEvent := "data: {\"choices\":[{\"delta\":{\"role\":\"assistant\"}}]}\n\n"
contentEvent := "data: {\"choices\":[{\"delta\":{\"content\":\"hello\"}}]}\n\n"
doneEvent := "data: [DONE]\n\n"
frames := make(chan *iop.ProviderTunnelFrame, 7)
frames <- &iop.ProviderTunnelFrame{
Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_RESPONSE_START,
StatusCode: 200,
Headers: map[string]string{"Content-Type": "text/event-stream"},
}
frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(roleEvent)}
// Split the content event so the usage frame arrives while the provider
// event is incomplete; the sideband writer must not inject there.
frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(contentEvent[:17])}
frames <- &iop.ProviderTunnelFrame{
Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_USAGE,
Usage: &iop.Usage{InputTokens: 3, OutputTokens: 5},
}
frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(contentEvent[17:])}
frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(doneEvent)}
frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_END, End: true}
close(frames)
srv, fake := chatSidebandServer(frames)
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{
"model":"pool-model",
"stream":true,
"messages":[{"role":"user","content":"hello"}],
"metadata":{"iop_response_mode":"passthrough+sideband"}
}`))
w := httptest.NewRecorder()
srv.handleChatCompletions(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d body=%s", w.Code, w.Body.String())
}
if got := w.Header().Get("X-IOP-Response-Mode"); got != "passthrough+sideband" {
t.Errorf("sideband response mode header: got %q", got)
}
body := w.Body.String()
// Provider events reach the caller contiguously: sideband injection never
// splits a provider SSE event.
for _, event := range []string{roleEvent, contentEvent, doneEvent} {
if !strings.Contains(body, event) {
t.Fatalf("provider event not preserved contiguously: %q in %q", event, body)
}
}
if !strings.HasPrefix(body, "event: iop.sideband\n") {
t.Fatalf("stream must open with the IOP route sideband event, got %q", body)
}
routeIdx := strings.Index(body, `"kind":"route"`)
if routeIdx < 0 || routeIdx > strings.Index(body, roleEvent) {
t.Fatalf("route observation must precede provider events: %q", body)
}
for _, marker := range []string{
`"object":"iop.sideband"`,
`"run_id":"run-tunnel"`,
`"node_id":"node-1"`,
`"response_mode":"passthrough+sideband"`,
} {
if !strings.Contains(body, marker) {
t.Errorf("route observation missing %q: %q", marker, body)
}
}
usageIdx := strings.Index(body, `"kind":"usage"`)
contentEnd := strings.Index(body, contentEvent) + len(contentEvent)
if usageIdx < contentEnd {
t.Errorf("usage observation must wait for the provider event boundary: usage at %d, content event ends at %d", usageIdx, contentEnd)
}
if !strings.Contains(body, `"input_tokens":3`) || !strings.Contains(body, `"output_tokens":5`) {
t.Errorf("usage observation missing token counts: %q", body)
}
assembledIdx := strings.Index(body, `"kind":"assembled"`)
if assembledIdx < 0 || assembledIdx < strings.Index(body, doneEvent) {
t.Errorf("assembled observation must close the stream: %q", body)
}
if !strings.Contains(body, `"content":"hello"`) {
t.Errorf("assembled observation missing assembled content: %q", body)
}
tunnelReqs := fake.tunnelReqsSnapshot()
if len(tunnelReqs) != 1 {
t.Fatalf("expected 1 tunnel dispatch, got %d", len(tunnelReqs))
}
if got := tunnelReqs[0].Metadata["iop_response_mode"]; got != "passthrough+sideband" {
t.Errorf("tunnel metadata response mode: got %q", got)
}
if len(fake.reqsSnapshot()) != 0 {
t.Error("sideband mode must not use the normalized SubmitRun path")
}
}
// TestChatCompletionsPassthroughSidebandNonStreamingWrapsProviderBody verifies
// SDD S05 for the non-streaming path: the provider JSON body is carried
// verbatim inside the iop.chat.passthrough_sideband envelope together with
// route/usage/assembled observations, and is not presented as byte identity.
func TestChatCompletionsPassthroughSidebandNonStreamingWrapsProviderBody(t *testing.T) {
providerBody := `{"id":"cmpl-1","object":"chat.completion","choices":[{"index":0,"message":{"role":"assistant","content":"hi","reasoning_content":"because"},"finish_reason":"stop"}]}`
frames := make(chan *iop.ProviderTunnelFrame, 5)
frames <- &iop.ProviderTunnelFrame{
Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_RESPONSE_START,
StatusCode: 200,
Headers: map[string]string{"Content-Type": "application/json"},
}
frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(providerBody[:23])}
frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(providerBody[23:])}
frames <- &iop.ProviderTunnelFrame{
Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_USAGE,
Usage: &iop.Usage{InputTokens: 7, OutputTokens: 11},
}
frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_END, End: true}
close(frames)
srv, fake := chatSidebandServer(frames)
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{
"model":"pool-model",
"messages":[{"role":"user","content":"hello"}],
"metadata":{"iop_response_mode":"passthrough+sideband"}
}`))
w := httptest.NewRecorder()
srv.handleChatCompletions(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d body=%s", w.Code, w.Body.String())
}
if got := w.Header().Get("X-IOP-Response-Mode"); got != "passthrough+sideband" {
t.Errorf("sideband response mode header: got %q", got)
}
if !strings.Contains(w.Body.String(), providerBody) {
t.Fatalf("provider body must be embedded verbatim: %s", w.Body.String())
}
var envelope struct {
Object string `json:"object"`
Sideband struct {
Route struct {
RunID string `json:"run_id"`
NodeID string `json:"node_id"`
ResponseMode string `json:"response_mode"`
ProviderStatusCode int `json:"provider_status_code"`
} `json:"route"`
Usage *struct {
InputTokens int `json:"input_tokens"`
OutputTokens int `json:"output_tokens"`
} `json:"usage"`
Assembled *struct {
Content string `json:"content"`
Reasoning string `json:"reasoning"`
BodyBytes int `json:"body_bytes"`
} `json:"assembled"`
} `json:"iop_sideband"`
ProviderStatusCode int `json:"provider_status_code"`
ProviderResponse json.RawMessage `json:"provider_response"`
}
if err := json.Unmarshal(w.Body.Bytes(), &envelope); err != nil {
t.Fatalf("envelope not valid JSON: %v body=%s", err, w.Body.String())
}
if envelope.Object != "iop.chat.passthrough_sideband" {
t.Errorf("envelope object: got %q", envelope.Object)
}
if envelope.Sideband.Route.RunID != "run-tunnel" || envelope.Sideband.Route.NodeID != "node-1" {
t.Errorf("route observation: %+v", envelope.Sideband.Route)
}
if envelope.Sideband.Route.ResponseMode != "passthrough+sideband" || envelope.Sideband.Route.ProviderStatusCode != http.StatusOK {
t.Errorf("route observation mode/status: %+v", envelope.Sideband.Route)
}
if envelope.Sideband.Usage == nil || envelope.Sideband.Usage.InputTokens != 7 || envelope.Sideband.Usage.OutputTokens != 11 {
t.Errorf("usage observation: %+v", envelope.Sideband.Usage)
}
if envelope.Sideband.Assembled == nil ||
envelope.Sideband.Assembled.Content != "hi" ||
envelope.Sideband.Assembled.Reasoning != "because" ||
envelope.Sideband.Assembled.BodyBytes != len(providerBody) {
t.Errorf("assembled observation: %+v", envelope.Sideband.Assembled)
}
if envelope.ProviderStatusCode != http.StatusOK {
t.Errorf("provider status: got %d", envelope.ProviderStatusCode)
}
if string(envelope.ProviderResponse) != providerBody {
t.Errorf("provider_response not verbatim:\n got: %s\nwant: %s", envelope.ProviderResponse, providerBody)
}
if len(fake.reqsSnapshot()) != 0 {
t.Error("sideband mode must not use the normalized SubmitRun path")
}
}
// TestChatCompletionsPassthroughDoesNotExposeSideband verifies SDD S06: pure
// passthrough (omitted mode) stays byte-identical even when the tunnel emits
// usage frames, and no IOP sideband marker, event, or label reaches the caller.
func TestChatCompletionsPassthroughDoesNotExposeSideband(t *testing.T) {
providerBody := "data: {\"choices\":[{\"delta\":{\"content\":\"pure\"}}]}\n\ndata: [DONE]\n\n"
frames := make(chan *iop.ProviderTunnelFrame, 4)
frames <- &iop.ProviderTunnelFrame{
Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_RESPONSE_START,
StatusCode: 200,
Headers: map[string]string{"Content-Type": "text/event-stream"},
}
frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(providerBody)}
frames <- &iop.ProviderTunnelFrame{
Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_USAGE,
Usage: &iop.Usage{InputTokens: 3, OutputTokens: 5},
}
frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_END, End: true}
close(frames)
srv, _ := chatSidebandServer(frames)
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{
"model":"pool-model",
"stream":true,
"messages":[{"role":"user","content":"hello"}]
}`))
w := httptest.NewRecorder()
srv.handleChatCompletions(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d body=%s", w.Code, w.Body.String())
}
if got := w.Body.String(); got != providerBody {
t.Fatalf("pure passthrough body not byte-identical:\n got: %q\nwant: %q", got, providerBody)
}
for _, marker := range []string{"iop.sideband", "iop_sideband", "iop_response_mode"} {
if strings.Contains(w.Body.String(), marker) {
t.Errorf("pure passthrough body must not contain %q", marker)
}
}
if got := w.Header().Get("X-IOP-Response-Mode"); got != "" {
t.Errorf("pure passthrough must not carry the IOP response mode header, got %q", got)
}
}
// TestChatCompletionsTransformedModeLabelsIOPOutput verifies SDD S07: explicit
// transformed output uses the normalized path and is labeled as IOP
// transformed output in both the response header and the run metadata.
func TestChatCompletionsTransformedModeLabelsIOPOutput(t *testing.T) {
fake := &fakeRunService{events: bufferedRunEvents(
&iop.RunEvent{Type: "delta", Delta: "ok"},
&iop.RunEvent{Type: "complete"},
)}
srv := NewServer(config.EdgeOpenAIConf{}, fake, nil)
srv.SetModelCatalog([]config.ModelCatalogEntry{
{ID: "pool-model", Providers: map[string]string{"prov-1": "served-model"}},
})
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{
"model":"pool-model",
"messages":[{"role":"user","content":"hello"}],
"metadata":{"iop_response_mode":"transformed"}
}`))
w := httptest.NewRecorder()
srv.handleChatCompletions(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d body=%s", w.Code, w.Body.String())
}
if got := w.Header().Get("X-IOP-Response-Mode"); got != "transformed" {
t.Errorf("transformed label header: got %q", got)
}
if !strings.Contains(w.Body.String(), `"object":"chat.completion"`) {
t.Errorf("transformed output must be the normalized IOP response: %s", w.Body.String())
}
if len(fake.tunnelReqsSnapshot()) != 0 {
t.Error("transformed mode must not use the tunnel")
}
reqs := fake.reqsSnapshot()
if len(reqs) != 1 {
t.Fatalf("expected 1 normalized SubmitRun dispatch, got %d", len(reqs))
}
if got := reqs[0].Metadata["iop_response_mode"]; got != "transformed" {
t.Errorf("run metadata response mode: got %q", got)
}
}
// TestChatCompletionsPassthroughDoesNotLabelTransformed verifies pure
// passthrough responses carry no IOP transformed label and stay byte-identical
// to the provider body.
func TestChatCompletionsPassthroughDoesNotLabelTransformed(t *testing.T) {
providerBody := `{"ok":true}`
frames := make(chan *iop.ProviderTunnelFrame, 3)
frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_RESPONSE_START, StatusCode: 200}
frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY, Body: []byte(providerBody)}
frames <- &iop.ProviderTunnelFrame{Kind: iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_END, End: true}
close(frames)
srv, _ := chatSidebandServer(frames)
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{
"model":"pool-model",
"messages":[{"role":"user","content":"hello"}]
}`))
w := httptest.NewRecorder()
srv.handleChatCompletions(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d body=%s", w.Code, w.Body.String())
}
if got := w.Body.String(); got != providerBody {
t.Fatalf("pure passthrough body not byte-identical: %q", got)
}
if got := w.Header().Get("X-IOP-Response-Mode"); got != "" {
t.Errorf("pure passthrough must not carry the IOP response mode header, got %q", got)
}
}
// chatPassthroughServer returns an openai.Server whose fake service relays the
// tunnel to the given provider fixture, emulating the Node relay.
func chatPassthroughServer(providerURL string) (*Server, *fakeRunService) {
@ -4314,8 +4708,8 @@ func TestChatCompletionsPassthroughLegacyProviderRouteUsesTunnel(t *testing.T) {
}
// TestChatCompletionsExplicitNonPassthroughModeKeepsNormalizedPath verifies
// that an explicit non-passthrough response mode stays on the legacy
// normalized RunEvent path (mode semantics land with the mode-contract task).
// that an explicit transformed response mode stays on the legacy normalized
// RunEvent path.
func TestChatCompletionsExplicitNonPassthroughModeKeepsNormalizedPath(t *testing.T) {
fake := &fakeRunService{events: bufferedRunEvents(
&iop.RunEvent{Type: "delta", Delta: "ok"},

View file

@ -1,6 +1,7 @@
package openai
import (
"bytes"
"encoding/json"
"fmt"
"net/http"
@ -311,13 +312,43 @@ func (s *Server) streamChatCompletion(w http.ResponseWriter, r *http.Request, re
// are written to the caller byte-identically. No IOP sideband fields or events
// are added to the response body.
func (s *Server) tunnelChatCompletionPassthrough(w http.ResponseWriter, r *http.Request, req chatCompletionRequest, dispatch routeDispatch, runMeta map[string]string, rawBody []byte, estimate int, contextClass string) {
handle, ok := s.submitChatCompletionTunnel(w, r, req, dispatch, runMeta, rawBody, estimate, contextClass, responseModePassthrough)
if !ok {
return
}
defer handle.Close()
s.writeProviderTunnelResponse(w, r, handle)
}
// tunnelChatCompletionPassthroughSideband serves a Chat Completions request
// over the raw provider tunnel with the explicit IOP sideband extension
// surface (SDD S05): provider content is preserved, but the response also
// carries IOP route/usage/assembled observations and is never claimed as
// provider-original byte identity.
func (s *Server) tunnelChatCompletionPassthroughSideband(w http.ResponseWriter, r *http.Request, req chatCompletionRequest, dispatch routeDispatch, runMeta map[string]string, rawBody []byte, estimate int, contextClass string) {
handle, ok := s.submitChatCompletionTunnel(w, r, req, dispatch, runMeta, rawBody, estimate, contextClass, responseModePassthroughSideband)
if !ok {
return
}
defer handle.Close()
if req.Stream {
s.writeProviderTunnelSidebandStream(w, r, handle)
return
}
s.writeProviderTunnelSidebandResponse(w, r, handle)
}
// submitChatCompletionTunnel dispatches a Chat Completions provider tunnel
// request with the given response mode. On failure it writes the dispatch
// error to the caller and returns ok=false.
func (s *Server) submitChatCompletionTunnel(w http.ResponseWriter, r *http.Request, req chatCompletionRequest, dispatch routeDispatch, runMeta map[string]string, rawBody []byte, estimate int, contextClass, responseMode string) (edgeservice.ProviderTunnelResult, bool) {
metadata := make(map[string]string, len(runMeta)+5)
for k, v := range runMeta {
metadata[k] = v
}
metadata["openai_model"] = req.Model
metadata["openai_stream"] = strconv.FormatBool(req.Stream)
metadata[responseModeMetadataKey] = responseModePassthrough
metadata[responseModeMetadataKey] = responseMode
metadata["estimated_input_tokens"] = strconv.Itoa(estimate)
metadata["context_class"] = contextClass
@ -345,9 +376,8 @@ func (s *Server) tunnelChatCompletionPassthrough(w http.ResponseWriter, r *http.
handle, err := s.service.SubmitProviderTunnel(r.Context(), tunnelReq)
if err != nil {
writeError(w, http.StatusBadGateway, "node_dispatch_error", err.Error())
return
return nil, false
}
defer handle.Close()
s.logger.Info("openai chat completion passthrough dispatch",
zap.String("run_id", handle.Dispatch().RunID),
@ -356,12 +386,12 @@ func (s *Server) tunnelChatCompletionPassthrough(w http.ResponseWriter, r *http.
zap.String("adapter", handle.Dispatch().Adapter),
zap.String("target", handle.Dispatch().Target),
zap.Bool("stream", req.Stream),
zap.String("response_mode", responseMode),
zap.Int("estimated_input_tokens", handle.Dispatch().EstimatedInputTokens),
zap.String("context_class", handle.Dispatch().ContextClass),
zap.String("queue_reason", handle.Dispatch().QueueReason),
)
s.writeProviderTunnelResponse(w, r, handle)
return handle, true
}
// writeProviderTunnelResponse relays ordered tunnel frames to the HTTP caller.
@ -471,6 +501,440 @@ func (s *Server) writeProviderTunnelResponse(w http.ResponseWriter, r *http.Requ
}
}
// Sideband extension surface (SDD S05): explicit passthrough+sideband
// responses expose IOP observations alongside provider-original content.
// Streaming responses interleave `event: iop.sideband` SSE events at provider
// event boundaries; non-streaming responses wrap the provider body in the
// iop.chat.passthrough_sideband envelope. Both are IOP extension outputs and
// are never claimed as provider-original byte identity.
const (
sidebandSSEEventName = "iop.sideband"
sidebandEnvelopeObject = "iop.chat.passthrough_sideband"
)
type sidebandRouteObservation struct {
RunID string `json:"run_id"`
NodeID string `json:"node_id,omitempty"`
Adapter string `json:"adapter,omitempty"`
Target string `json:"target,omitempty"`
ModelGroup string `json:"model_group,omitempty"`
ResponseMode string `json:"response_mode"`
ProviderStatusCode int `json:"provider_status_code,omitempty"`
}
type sidebandUsageObservation struct {
InputTokens int `json:"input_tokens"`
OutputTokens int `json:"output_tokens"`
}
type sidebandAssembledObservation struct {
Content string `json:"content,omitempty"`
Reasoning string `json:"reasoning,omitempty"`
ToolCallNames []string `json:"tool_call_names,omitempty"`
BodyBytes int `json:"body_bytes"`
}
// sidebandObservation is one `event: iop.sideband` SSE payload on the
// streaming sideband surface.
type sidebandObservation struct {
Object string `json:"object"`
Kind string `json:"kind"`
Route *sidebandRouteObservation `json:"route,omitempty"`
Usage *sidebandUsageObservation `json:"usage,omitempty"`
Assembled *sidebandAssembledObservation `json:"assembled,omitempty"`
}
// sidebandEnvelope is the non-streaming sideband response schema: the provider
// body is carried verbatim next to the IOP observations.
type sidebandEnvelope struct {
Object string `json:"object"`
Sideband sidebandObservations `json:"iop_sideband"`
ProviderStatusCode int `json:"provider_status_code"`
ProviderResponse json.RawMessage `json:"provider_response,omitempty"`
ProviderBody string `json:"provider_body,omitempty"`
}
type sidebandObservations struct {
Route sidebandRouteObservation `json:"route"`
Usage *sidebandUsageObservation `json:"usage,omitempty"`
Assembled *sidebandAssembledObservation `json:"assembled,omitempty"`
}
func sidebandRouteFromDispatch(dispatch edgeservice.RunDispatch, providerStatus int) sidebandRouteObservation {
return sidebandRouteObservation{
RunID: dispatch.RunID,
NodeID: dispatch.NodeID,
Adapter: dispatch.Adapter,
Target: dispatch.Target,
ModelGroup: dispatch.ModelGroupKey,
ResponseMode: responseModePassthroughSideband,
ProviderStatusCode: providerStatus,
}
}
func sidebandUsageFromFrame(frame *iop.ProviderTunnelFrame) *sidebandUsageObservation {
usage := frame.GetUsage()
if usage == nil {
return nil
}
return &sidebandUsageObservation{
InputTokens: int(usage.GetInputTokens()),
OutputTokens: int(usage.GetOutputTokens()),
}
}
// providerChatAssembler accumulates a human-readable view of the provider
// Chat Completions response for the sideband assembled observation. It
// tolerates non-JSON and partial payloads: whatever cannot be parsed simply
// yields an empty summary.
type providerChatAssembler struct {
streaming bool
bodyBytes int
pending []byte
content strings.Builder
reasoning strings.Builder
toolCallNames []string
}
// providerChatDeltaEnvelope matches the provider fields the assembler
// observes; streaming uses delta, non-streaming uses message.
type providerChatDeltaEnvelope struct {
Content string `json:"content"`
ReasoningContent string `json:"reasoning_content"`
ToolCalls []struct {
Function struct {
Name string `json:"name"`
} `json:"function"`
} `json:"tool_calls"`
}
func (a *providerChatAssembler) Write(chunk []byte) {
a.bodyBytes += len(chunk)
a.pending = append(a.pending, chunk...)
if !a.streaming {
return
}
for {
idx := bytes.IndexByte(a.pending, '\n')
if idx < 0 {
return
}
line := strings.TrimRight(string(a.pending[:idx]), "\r")
a.pending = a.pending[idx+1:]
a.consumeSSELine(line)
}
}
func (a *providerChatAssembler) consumeSSELine(line string) {
payload, ok := strings.CutPrefix(line, "data:")
if !ok {
return
}
payload = strings.TrimSpace(payload)
if payload == "" || payload == "[DONE]" {
return
}
var chunk struct {
Choices []struct {
Delta providerChatDeltaEnvelope `json:"delta"`
} `json:"choices"`
}
if err := json.Unmarshal([]byte(payload), &chunk); err != nil {
return
}
for _, choice := range chunk.Choices {
a.consumeDelta(choice.Delta)
}
}
func (a *providerChatAssembler) consumeDelta(delta providerChatDeltaEnvelope) {
a.content.WriteString(delta.Content)
a.reasoning.WriteString(delta.ReasoningContent)
for _, call := range delta.ToolCalls {
if name := strings.TrimSpace(call.Function.Name); name != "" {
a.toolCallNames = append(a.toolCallNames, name)
}
}
}
func (a *providerChatAssembler) observation() *sidebandAssembledObservation {
if !a.streaming {
var resp struct {
Choices []struct {
Message providerChatDeltaEnvelope `json:"message"`
} `json:"choices"`
}
if err := json.Unmarshal(a.pending, &resp); err == nil {
for _, choice := range resp.Choices {
a.consumeDelta(choice.Message)
}
}
}
return &sidebandAssembledObservation{
Content: a.content.String(),
Reasoning: a.reasoning.String(),
ToolCallNames: a.toolCallNames,
BodyBytes: a.bodyBytes,
}
}
// writeProviderTunnelSidebandStream relays provider SSE bytes unchanged and
// interleaves explicit `event: iop.sideband` extension events. IOP events are
// written only at provider event boundaries (before the first body byte,
// after a "\n\n"-terminated provider event, or at end of stream) so a
// provider event is never split by sideband injection.
func (s *Server) writeProviderTunnelSidebandStream(w http.ResponseWriter, r *http.Request, handle edgeservice.ProviderTunnelResult) {
frames := handle.Stream().Frames
if frames == nil {
writeError(w, http.StatusBadGateway, "provider_tunnel_error", "tunnel stream unavailable")
return
}
flusher, _ := w.(http.Flusher)
timer := time.NewTimer(handle.WaitTimeout())
defer timer.Stop()
assembler := &providerChatAssembler{streaming: true}
wroteHeader := false
atEventBoundary := true
tail := ""
var pendingObservations []sidebandObservation
writeObservation := func(obs sidebandObservation) {
obs.Object = sidebandSSEEventName
payload, err := json.Marshal(obs)
if err != nil {
return
}
fmt.Fprintf(w, "event: %s\ndata: %s\n\n", sidebandSSEEventName, payload)
if flusher != nil {
flusher.Flush()
}
}
flushObservations := func() {
for _, obs := range pendingObservations {
writeObservation(obs)
}
pendingObservations = nil
}
defer func() {
s.logger.Info("openai chat completion sideband stream closed",
zap.String("run_id", handle.Dispatch().RunID),
zap.Bool("wrote_header", wroteHeader),
zap.Int("body_bytes", assembler.bodyBytes),
)
}()
for {
select {
case <-r.Context().Done():
s.cancelRunOnHTTPGiveUp(handle.Dispatch(), r.Context().Err())
return
case <-timer.C:
s.cancelRunOnHTTPGiveUp(handle.Dispatch(), errRunTimedOut)
if !wroteHeader {
writeError(w, http.StatusBadGateway, "run_error", errRunTimedOut.Error())
}
return
case frame, ok := <-frames:
if !ok {
if !wroteHeader {
writeError(w, http.StatusBadGateway, "provider_tunnel_error", "tunnel stream closed before provider response")
}
return
}
switch frame.GetKind() {
case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_RESPONSE_START:
if wroteHeader {
continue
}
copyProviderResponseHeaders(w.Header(), frame.GetHeaders())
// The sideband stream is an IOP extension surface: label it
// and drop the provider Content-Length, which no longer
// matches the extended body.
w.Header().Del("Content-Length")
w.Header().Set(responseModeHeaderName, responseModePassthroughSideband)
status := int(frame.GetStatusCode())
if status == 0 {
status = http.StatusOK
}
w.WriteHeader(status)
wroteHeader = true
if flusher != nil {
flusher.Flush()
}
route := sidebandRouteFromDispatch(handle.Dispatch(), status)
writeObservation(sidebandObservation{Kind: "route", Route: &route})
case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY:
body := frame.GetBody()
if len(body) == 0 {
continue
}
if !wroteHeader {
// Defensive: a body frame before response-start still
// reaches the caller instead of being dropped.
w.Header().Set(responseModeHeaderName, responseModePassthroughSideband)
w.WriteHeader(http.StatusOK)
wroteHeader = true
}
if _, err := w.Write(body); err != nil {
// The caller is gone mid-stream; propagate cancel to the Node.
s.sendCancelRun(handle.Dispatch())
return
}
assembler.Write(body)
if flusher != nil {
flusher.Flush()
}
tail = sseTail(tail, body)
atEventBoundary = strings.HasSuffix(tail, "\n\n")
if atEventBoundary {
flushObservations()
}
case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_ERROR:
msg := frame.GetError()
if msg == "" {
msg = "provider tunnel failed"
}
if !wroteHeader {
writeError(w, http.StatusBadGateway, "provider_tunnel_error", msg)
return
}
// Status/headers are already committed; the response is
// truncated and the caller observes the broken stream.
s.logger.Warn("openai sideband tunnel error after response start",
zap.String("run_id", handle.Dispatch().RunID),
zap.String("error", msg),
)
return
case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_USAGE:
usage := sidebandUsageFromFrame(frame)
if usage == nil {
continue
}
pendingObservations = append(pendingObservations, sidebandObservation{Kind: "usage", Usage: usage})
if wroteHeader && atEventBoundary {
flushObservations()
}
case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_END:
if !wroteHeader {
writeError(w, http.StatusBadGateway, "provider_tunnel_error", "tunnel ended before provider response")
return
}
if !atEventBoundary {
// The provider stream ended mid-event; terminate it so the
// trailing IOP events stay well-formed SSE.
fmt.Fprint(w, "\n\n")
}
flushObservations()
writeObservation(sidebandObservation{Kind: "assembled", Assembled: assembler.observation()})
return
}
}
}
}
// sseTail keeps the last two bytes of the relayed provider stream so the
// sideband writer can detect "\n\n" event boundaries across chunk splits.
func sseTail(tail string, chunk []byte) string {
combined := tail + string(chunk)
if len(combined) > 2 {
return combined[len(combined)-2:]
}
return combined
}
// writeProviderTunnelSidebandResponse buffers the provider response and
// answers with the iop.chat.passthrough_sideband envelope: provider status is
// preserved, the provider body is carried verbatim inside the envelope, and
// IOP route/usage/assembled observations sit alongside it.
func (s *Server) writeProviderTunnelSidebandResponse(w http.ResponseWriter, r *http.Request, handle edgeservice.ProviderTunnelResult) {
frames := handle.Stream().Frames
if frames == nil {
writeError(w, http.StatusBadGateway, "provider_tunnel_error", "tunnel stream unavailable")
return
}
timer := time.NewTimer(handle.WaitTimeout())
defer timer.Stop()
assembler := &providerChatAssembler{}
var body bytes.Buffer
providerStatus := 0
var usage *sidebandUsageObservation
for {
select {
case <-r.Context().Done():
s.cancelRunOnHTTPGiveUp(handle.Dispatch(), r.Context().Err())
return
case <-timer.C:
s.cancelRunOnHTTPGiveUp(handle.Dispatch(), errRunTimedOut)
writeError(w, http.StatusBadGateway, "run_error", errRunTimedOut.Error())
return
case frame, ok := <-frames:
if !ok {
writeError(w, http.StatusBadGateway, "provider_tunnel_error", "tunnel stream closed before provider response")
return
}
switch frame.GetKind() {
case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_RESPONSE_START:
if providerStatus != 0 {
continue
}
providerStatus = int(frame.GetStatusCode())
if providerStatus == 0 {
providerStatus = http.StatusOK
}
case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_BODY:
body.Write(frame.GetBody())
assembler.Write(frame.GetBody())
case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_ERROR:
msg := frame.GetError()
if msg == "" {
msg = "provider tunnel failed"
}
writeError(w, http.StatusBadGateway, "provider_tunnel_error", msg)
return
case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_USAGE:
if u := sidebandUsageFromFrame(frame); u != nil {
usage = u
}
case iop.ProviderTunnelFrameKind_PROVIDER_TUNNEL_FRAME_KIND_END:
if providerStatus == 0 && body.Len() == 0 {
writeError(w, http.StatusBadGateway, "provider_tunnel_error", "tunnel ended before provider response")
return
}
if providerStatus == 0 {
providerStatus = http.StatusOK
}
envelope := sidebandEnvelope{
Object: sidebandEnvelopeObject,
Sideband: sidebandObservations{
Route: sidebandRouteFromDispatch(handle.Dispatch(), providerStatus),
Usage: usage,
Assembled: assembler.observation(),
},
ProviderStatusCode: providerStatus,
}
raw := body.Bytes()
if json.Valid(raw) {
envelope.ProviderResponse = json.RawMessage(raw)
} else if len(raw) > 0 {
envelope.ProviderBody = string(raw)
}
w.Header().Set(responseModeHeaderName, responseModePassthroughSideband)
writeJSON(w, providerStatus, envelope)
s.logger.Info("openai chat completion sideband response",
zap.String("run_id", handle.Dispatch().RunID),
zap.Int("provider_status", providerStatus),
zap.Int("body_bytes", body.Len()),
)
return
}
}
}
}
// hopByHopResponseHeaders are transport-level headers owned by each hop; they
// are not copied from the provider response to the caller response.
var hopByHopResponseHeaders = map[string]struct{}{