update operator workflow tests and backtest runtime

This commit is contained in:
toki 2026-06-03 22:28:59 +09:00
parent 473934212c
commit 96e7c85c6f
31 changed files with 1325 additions and 209 deletions

View file

@ -34,44 +34,49 @@ task=m-command-first-operator-workflow/03+02_market_status_scenario, plan=0, tag
| 항목 | 완료 여부 |
|------|---------|
| [CLI-1] Import Before Status Reads | [ ] |
| [CLI-2] Status Evidence Tests | [ ] |
| [CLI-3] Handoff Discoverability | [ ] |
| [CLI-4] Verification | [ ] |
| [CLI-1] Import Before Status Reads | [x] |
| [CLI-2] Status Evidence Tests | [x] |
| [CLI-3] Handoff Discoverability | [x] |
| [CLI-4] Verification | [x] |
## 구현 체크리스트
- [ ] [CLI-1] Update `market_data_status_query.yaml` to run import before instrument/bar status reads.
- [ ] [CLI-2] Update expected JSONL and tests to require import counts, instrument count, bar count, and typed unavailable/error keys. 검증: `alt operator scenario validate/run` 또는 동등한 command가 instrument/bar count와 typed unavailable/error key를 출력한다.
- [ ] [CLI-3] Update handoff matrix and CLI dev hints so the command-first status scenario is discoverable.
- [ ] [CLI-4] Run CLI verification and `git diff --check`.
- [ ] CODE_REVIEW-*-G??.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채운다. 이 항목이 완료되기 전에는 구현이 완료된 것이 아니다.
- [x] [CLI-1] Update `market_data_status_query.yaml` to run import before instrument/bar status reads.
- [x] [CLI-2] Update expected JSONL and tests to require import counts, instrument count, bar count, and typed unavailable/error keys. 검증: `alt operator scenario validate/run` 또는 동등한 command가 instrument/bar count와 typed unavailable/error key를 출력한다.
- [x] [CLI-3] Update handoff matrix and CLI dev hints so the command-first status scenario is discoverable.
- [x] [CLI-4] Run CLI verification and `git diff --check`.
- [x] CODE_REVIEW-*-G??.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채운다. 이 항목이 완료되기 전에는 구현이 완료된 것이 아니다.
## 코드리뷰 전용 체크리스트
> **[REVIEW AGENT ONLY]** 이 체크리스트는 코드리뷰 에이전트만 사용한다.
> 구현 에이전트는 이 섹션을 수정하거나 체크하지 않는다.
- [ ] `코드리뷰 결과`에 `PASS`, `WARN`, `FAIL` 중 하나의 판정을 append한다.
- [ ] 판정과 `차원별 평가`, Required/Suggested/Nit 분류가 서로 일치한다.
- [ ] active `CODE_REVIEW-*-G??.md`를 `code_review_cloud_G07_N.log`로 아카이브한다.
- [ ] active `PLAN-*-G??.md`를 `plan_cloud_G07_M.log`로 아카이브한다.
- [ ] `.gitignore`의 Agent-Ops 관리 block이 `agent-task/**/*.md`와 `agent-task/**/*.log`를 unignore하여 plan/review/archive 산출물이 추적 가능한지 확인한다.
- [ ] PASS이면 `agent-ops/skills/common/code-review/templates/complete-log-template.md` 기준으로 `complete.log`를 작성하고 active `.md` 파일을 남기지 않는다.
- [ ] PASS이면 active task 디렉터리 `agent-task/m-command-first-operator-workflow/03+02_market_status_scenario/`를 `agent-task/archive/YYYY/MM/m-command-first-operator-workflow/03+02_market_status_scenario/`로 이동하고 최종 archive 경로에서 이 체크리스트를 갱신한다.
- [ ] PASS이고 task group이 `m-<milestone-slug>`이면 런타임이 읽을 완료 이벤트 메타데이터를 보고하고, roadmap 수정이나 `update-roadmap` 직접 호출을 하지 않는다.
- [ ] PASS split 작업이면 이동 후 빈 active parent `agent-task/m-command-first-operator-workflow/`를 제거하거나, 남은 sibling/file이 있어 유지했다고 확인한다.
- [x] `코드리뷰 결과`에 `PASS`, `WARN`, `FAIL` 중 하나의 판정을 append한다.
- [x] 판정과 `차원별 평가`, Required/Suggested/Nit 분류가 서로 일치한다.
- [x] active `CODE_REVIEW-*-G??.md`를 `code_review_cloud_G07_N.log`로 아카이브한다.
- [x] active `PLAN-*-G??.md`를 `plan_cloud_G07_M.log`로 아카이브한다.
- [x] `.gitignore`의 Agent-Ops 관리 block이 `agent-task/**/*.md`와 `agent-task/**/*.log`를 unignore하여 plan/review/archive 산출물이 추적 가능한지 확인한다.
- [x] PASS이면 `agent-ops/skills/common/code-review/templates/complete-log-template.md` 기준으로 `complete.log`를 작성하고 active `.md` 파일을 남기지 않는다.
- [x] PASS이면 active task 디렉터리 `agent-task/m-command-first-operator-workflow/03+02_market_status_scenario/`를 `agent-task/archive/YYYY/MM/m-command-first-operator-workflow/03+02_market_status_scenario/`로 이동하고 최종 archive 경로에서 이 체크리스트를 갱신한다.
- [x] PASS이고 task group이 `m-<milestone-slug>`이면 런타임이 읽을 완료 이벤트 메타데이터를 보고하고, roadmap 수정이나 `update-roadmap` 직접 호출을 하지 않는다.
- [x] PASS split 작업이면 이동 후 빈 active parent `agent-task/m-command-first-operator-workflow/`를 제거하거나, 남은 sibling/file이 있어 유지했다고 확인한다.
- [ ] WARN/FAIL이고 user-review gate가 트리거되지 않았으면 다음 active `PLAN-cloud-G07.md`와 `CODE_REVIEW-cloud-G07.md`를 작성하고 `complete.log`를 작성하지 않는다.
- [ ] USER_REVIEW이면 `agent-ops/skills/common/code-review/templates/user-review-template.md` 기준으로 `USER_REVIEW.md`를 작성하고 active `PLAN-*.md`, `CODE_REVIEW-*.md`, `complete.log`를 남기지 않는다.
- [ ] USER_REVIEW가 사용자 결정으로 완료/PASS 해소되면 `USER_REVIEW.md`를 해소 상태로 갱신하고 `complete.log`를 작성한 뒤 task directory를 archive로 이동한다.
## 계획 대비 변경 사항
_구현 에이전트가 계획과 다르게 구현한 부분을 이유와 함께 기록한다._
- import step id를 계획의 `import_kr_daily_bars` 그대로 사용. (계획 본문 일부에서 `import_kr_daily_bars`/`list_kr_instruments` 표기와 일치)
- `client_test.go`의 변경은 계획이 적은 "expose last import request"가 아니라 호출 순서 기록(`recordCall`/`callOrder`) 추가로 대체. 사유: import-before-read 증거로는 마지막 요청 캡처(`lastImportDailyBarsReq`는 이미 존재)보다 import→list_instruments→list_bars 호출 순서 단언이 더 직접적이다. 마지막 import request 캡처는 KIS window 단언으로 함께 사용.
- handoff_test.go에 추가한 테스트 이름은 `TestStatusScenarioFixtureImportsBeforeReads`로, 고정 검증 명령의 `-run ...|Handoff` 필터에는 잡히지 않지만 전체 테스트(`go test ./apps/cli/...`)에서 실행되어 PASS한다. 계획의 중간 검증 명령(고정 계약)은 그대로 유지했다.
- `bin/dev`는 read-only 문구가 없어 (CLI-3 계획의 "update text only if it says read-only") 변경하지 않음.
## 주요 설계 결정
_구현 에이전트가 주요 설계 결정 사항을 기록한다._
- expected JSONL은 기존 `kis_daily_import_smoke.jsonl`과 동일하게 illustrative seed 값(`instrument_count=1`, `bar_count=42`)을 사용. handoff 문서가 명시하듯 line별 `scenario`/`step`/`action`/`status`와 protobuf field 존재 여부가 계약이고 count 값은 seed 의존이다.
- typed unavailable 검증은 import step이 `ErrorInfo{Code:"unavailable"}`를 돌려줄 때 `expect.status: error`/`error_code: unavailable`로 step이 통과(exit 0)하며 출력에 `error.code=unavailable`이 남는 것을 단언. transport_error로 떨어지지 않음을 증거로 남긴다.
- 호출 순서 기록은 fakeAPI에 mutex 보호 `calls` slice로 최소 침습 추가했고, 기존 핸들러 동작은 유지.
## 사용자 리뷰 요청
@ -106,33 +111,52 @@ _구현 에이전트가 각 중간 검증 및 최종 검증 명령 실행 후
### CLI-1 중간 검증
```bash
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/market_data_status_query.yaml
(output)
scenario=market_data_status_query status=valid steps=4
```
### CLI-2 중간 검증
```bash
$ go test ./apps/cli/internal/operator -run 'MarketDataStatus|MarketStatusTypedUnavailable|Handoff' -count=1
(output)
$ go test ./apps/cli/internal/operator -run 'MarketDataStatus|MarketStatusTypedUnavailable|Handoff' -count=1 -v
=== RUN TestHeadlessValidationHandoffCoversRequiredScenarios
--- PASS: TestHeadlessValidationHandoffCoversRequiredScenarios (0.00s)
=== RUN TestHandoffDocumentedScenarioFixturesExist
--- PASS: TestHandoffDocumentedScenarioFixturesExist (0.00s)
=== RUN TestRunScenarioMarketDataStatus
--- PASS: TestRunScenarioMarketDataStatus (0.00s)
=== RUN TestRunScenarioMarketDataStatusImportsBeforeReads
--- PASS: TestRunScenarioMarketDataStatusImportsBeforeReads (0.00s)
=== RUN TestRunScenarioMarketStatusTypedUnavailable
--- PASS: TestRunScenarioMarketStatusTypedUnavailable (0.00s)
PASS
ok git.toki-labs.com/toki/alt/apps/cli/internal/operator 0.016s
```
참고: 새 handoff 테스트 `TestStatusScenarioFixtureImportsBeforeReads`는 위 고정 필터에는 잡히지 않으나 최종 검증의 `go test ./apps/cli/...`에서 실행되어 PASS한다.
### CLI-3 중간 검증
```bash
$ go test ./apps/cli/internal/operator -run Handoff -count=1
(output)
ok git.toki-labs.com/toki/alt/apps/cli/internal/operator 0.042s
```
### CLI-4 중간 검증
```bash
$ git diff --check
(output)
(no output — clean)
```
### 최종 검증
```bash
$ go test ./apps/cli/... -count=1
? git.toki-labs.com/toki/alt/apps/cli/cmd/alt [no test files]
ok git.toki-labs.com/toki/alt/apps/cli/internal/cli 0.013s
ok git.toki-labs.com/toki/alt/apps/cli/internal/operator 0.182s
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/market_data_status_query.yaml
scenario=market_data_status_query status=valid steps=4
$ git diff --check
(output)
(no output — clean)
```
---
@ -140,3 +164,17 @@ $ git diff --check
> **[IMPLEMENTING AGENT — BEFORE SAVING] Have you filled in every implementation-owned section: completion table, implementation checklist, changes from plan, design decisions, and verification output?**
> If anything is blank, go back and fill it in before saving this file.
> Leave review-agent-only sections unchanged.
## 코드리뷰 결과
- 종합 판정: PASS
- 차원별 평가:
- correctness: Pass
- completeness: Pass
- test coverage: Pass
- API contract: Pass
- code quality: Pass
- plan deviation: Pass
- verification trust: Pass
- 발견된 문제: 없음
- 다음 단계: PASS 절차로 `complete.log`를 작성하고 task directory를 archive로 이동한다.

View file

@ -0,0 +1,43 @@
# Complete - m-command-first-operator-workflow/03+02_market_status_scenario
## 완료 일시
2026-06-03
## 요약
Market status scenario import-before-read workflow completed in 1 review loop; final verdict PASS.
## 루프 이력
| Plan | Review | Verdict | 메모 |
|------|--------|---------|------|
| `plan_cloud_G07_0.log` | `code_review_cloud_G07_0.log` | PASS | `market_data_status_query` now imports daily bars before status reads, with fixture/docs/tests/verification updated. |
## 구현/정리 내용
- Added `import_daily_bars` before instrument/bar status reads in `market_data_status_query.yaml`.
- Updated expected JSONL, handoff evidence, and runner tests to cover import counts, read counts, call order, and typed `unavailable` output.
- Added fake API call-order capture for command-first scenario assertions.
## 최종 검증
- `go test ./apps/cli/internal/operator -run 'MarketDataStatus|MarketStatusTypedUnavailable|Handoff' -count=1 -v` - PASS; focused market status, typed unavailable, and handoff tests passed.
- `go test ./apps/cli/... -count=1` - PASS; `cmd/alt` had no test files, `internal/cli` and `internal/operator` passed.
- `go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/market_data_status_query.yaml` - PASS; `scenario=market_data_status_query status=valid steps=4`.
- `git diff --check` - PASS; no output.
## Roadmap Completion
- Milestone: `agent-roadmap/phase/operator-surface/milestones/command-first-operator-workflow.md`
- Completed task ids:
- `market-status-scenario`: PASS; evidence=`agent-task/archive/2026/06/m-command-first-operator-workflow/03+02_market_status_scenario/plan_cloud_G07_0.log`, `agent-task/archive/2026/06/m-command-first-operator-workflow/03+02_market_status_scenario/code_review_cloud_G07_0.log`; verification=`go test ./apps/cli/... -count=1`, `go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/market_data_status_query.yaml`, `git diff --check`
- Not completed task ids: 없음
## 잔여 Nit
- 없음
## 후속 작업
- 없음

View file

@ -0,0 +1,217 @@
<!-- task=m-command-first-operator-workflow/04+02_backtest_runtime_flow plan=0 tag=API -->
# Code Review Reference - API
> **[IMPLEMENTING AGENT — READ FIRST] Filling in this file is the mandatory final step of implementation.**
> The task is NOT complete until every implementation-owned section below is filled in.
> Complete the `구현 체크리스트`; the final checklist item is mandatory before saving.
> Fill implementation-owned sections, then stop with active files in place and report ready for review.
> If implementation is blocked by a user-only decision, user-owned external environment prerequisite, or scope conflict, fill `사용자 리뷰 요청` with evidence and stop with active files in place; code-review decides whether to write `USER_REVIEW.md`. Evidence gaps that a follow-up agent can close by rerunning commands or collecting artifacts are normal follow-up issues, not user-review blockers by themselves.
> Finalization (`코드리뷰 결과`, log rename, `complete.log`, archive moves, `코드리뷰 전용 체크리스트`) is review-agent-only, even after compaction/resume.
> Follow the ownership table at the bottom of this file for which sections you own.
## 개요
date=2026-06-03
task=m-command-first-operator-workflow/04+02_backtest_runtime_flow, plan=0, tag=API
## Roadmap Targets
- Milestone: `agent-roadmap/phase/operator-surface/milestones/command-first-operator-workflow.md`
- Task ids:
- `backtest-command-flow`: imported daily bars를 입력으로 backtest start/poll/result를 command workflow에서 확인할 수 있다.
- Completion mode: check-on-pass
## 이 파일을 읽는 리뷰 에이전트에게
> **[REVIEW AGENT ONLY]** 아래 종결 절차는 코드리뷰 에이전트 전용이다. 구현 에이전트는 이 섹션을 실행하지 않는다.
각 항목의 구현을 실제 소스 파일과 대조하고, `검증 결과` 섹션의 출력이 코드와 일치하는지 확인하세요.
---
## 구현 항목별 완료 여부
| 항목 | 완료 여부 |
|------|---------|
| [API-1] GetBacktestRun Rail | [x] |
| [API-2] Worker Runtime Wiring | [x] |
| [API-3] Execution Evidence Tests | [x] |
| [API-4] Backtest YAML Flow | [x] |
| [API-5] Verification | [x] |
## 구현 체크리스트
- [x] [API-1] Add API and worker `GetBacktestRun` forwarding/handler support for polling.
- [x] [API-2] Wire worker `alt-worker` with `BacktestStarter`, `RegisterRunBacktestHandler`, `StorageBarSource`, result store, and a built-in strategy resolver.
- [x] [API-3] Add/update tests for built-in strategy resolver, worker execution wiring, and typed run polling.
- [x] [API-4] Update backtest YAML scenarios to import daily bars before start/poll/result. 검증: start/poll/result scenario가 run id, terminal status, result summary key를 출력한다.
- [x] [API-5] Run API/worker/CLI verification and `git diff --check`.
- [x] CODE_REVIEW-*-G??.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채운다. 이 항목이 완료되기 전에는 구현이 완료된 것이 아니다.
## 코드리뷰 전용 체크리스트
> **[REVIEW AGENT ONLY]** 이 체크리스트는 코드리뷰 에이전트만 사용한다.
> 구현 에이전트는 이 섹션을 수정하거나 체크하지 않는다.
- [x] `코드리뷰 결과`에 `PASS`, `WARN`, `FAIL` 중 하나의 판정을 append한다.
- [x] 판정과 `차원별 평가`, Required/Suggested/Nit 분류가 서로 일치한다.
- [x] active `CODE_REVIEW-*-G??.md`를 `code_review_cloud_G08_N.log`로 아카이브한다.
- [x] active `PLAN-*-G??.md`를 `plan_cloud_G08_M.log`로 아카이브한다.
- [x] `.gitignore`의 Agent-Ops 관리 block이 `agent-task/**/*.md`와 `agent-task/**/*.log`를 unignore하여 plan/review/archive 산출물이 추적 가능한지 확인한다.
- [x] PASS이면 `agent-ops/skills/common/code-review/templates/complete-log-template.md` 기준으로 `complete.log`를 작성하고 active `.md` 파일을 남기지 않는다.
- [x] PASS이면 active task 디렉터리 `agent-task/m-command-first-operator-workflow/04+02_backtest_runtime_flow/`를 `agent-task/archive/YYYY/MM/m-command-first-operator-workflow/04+02_backtest_runtime_flow/`로 이동하고 최종 archive 경로에서 이 체크리스트를 갱신한다.
- [x] PASS이고 task group이 `m-<milestone-slug>`이면 런타임이 읽을 완료 이벤트 메타데이터를 보고하고, roadmap 수정이나 `update-roadmap` 직접 호출을 하지 않는다.
- [x] PASS split 작업이면 이동 후 빈 active parent `agent-task/m-command-first-operator-workflow/`를 제거하거나, 남은 sibling/file이 있어 유지했다고 확인한다.
- [ ] WARN/FAIL이고 user-review gate가 트리거되지 않았으면 다음 active `PLAN-cloud-G08.md`와 `CODE_REVIEW-cloud-G08.md`를 작성하고 `complete.log`를 작성하지 않는다.
- [ ] USER_REVIEW이면 `agent-ops/skills/common/code-review/templates/user-review-template.md` 기준으로 `USER_REVIEW.md`를 작성하고 active `PLAN-*.md`, `CODE_REVIEW-*.md`, `complete.log`를 남기지 않는다.
- [ ] USER_REVIEW가 사용자 결정으로 완료/PASS 해소되면 `USER_REVIEW.md`를 해소 상태로 갱신하고 `complete.log`를 작성한 뒤 task directory를 archive로 이동한다.
## 계획 대비 변경 사항
- [API-1] 계획은 `services/api/internal/socket/server_test.go`의 capability sync 가능성을 열어 두었으나, API capability set은 핸들러 등록과 무관하게 고정 목록(`capabilitiesForSession`)이고 `GetBacktestRun`은 기존 `backtest-read`/`backtest-start` capability 묶음 안에 들어가므로 capability 변경이 불필요했다. 따라서 `server_test.go`는 수정하지 않았고, 대신 `backtest_test.go`의 등록 커버리지(`TestBacktestHandlersRegisteredInSession`)와 socket validation 케이스에 `GetBacktestRunRequest`를 추가했다.
- [API-2] worker `server_test.go`는 이미 `TestWorkerHelloCapabilitiesReflectDeps`의 `starter deps`/`all deps` 케이스로 Starter가 있을 때 `backtest-start`/`worker-execution` capability가 노출됨을 검증하므로 추가 수정 없이 그대로 둔다.
- [API-2] 계획이 선호한 "store-backed deps wiring helper 추출 + Starter != nil 단위 테스트" 옵션을 채택했다. `cmd/alt-worker/main.go`에서 `storeBackedDeps(runner, store)` 헬퍼를 분리하고 `cmd/alt-worker/main_test.go`(신규)에서 nil pool로 wiring만 검증한다(생성 경로는 DB를 건드리지 않음).
- [API-3] 실행 증거 테스트는 `fixture_test.go` 확장 대신 신규 파일 `services/worker/internal/backtest/flow_test.go`(`package backtest_test`)로 작성했다. import→start→poll→result를 한 흐름에서 동기 실행(`WithStarterLaunch(fn->fn())`)으로 검증하며, 이는 계획이 허용한 "new worker backtest flow test" 경로다. jobs와 backtest 패키지를 함께 import해야 해 import cycle을 피하려 external test package(`backtest_test`)를 사용했다.
- [API-4] `runner_error_test.go` 대신 기존 backtest 러너 테스트 파일 `runner_backtest_test.go`에 `TestRunBacktestImportThenStartFlow`를 추가해 import 선행 흐름(run_id/summary/import count)을 fake API로 검증했다. 기존 구조 우선 원칙에 따라 backtest 러너 테스트가 모여 있는 파일을 택했다.
## 주요 설계 결정
- [API-1] worker `GetBacktestRun` 핸들러는 신규 run store 의존성을 `Deps`에 추가하지 않고 기존 `deps.Analysis.GetRunDetail`을 재사용한다(계획 선호안). 단일 run 조회 polling이 list/detail과 같은 store-backed read 경로를 공유하고, `not_found`/`unavailable`/`timeout`도 동일한 `backendErrorInfo` 매핑으로 일관되게 typed error를 반환한다.
- [API-1] API는 thin control plane 규칙대로 `handleGetBacktestRun`이 run_id shape만 검증하고 worker로 forwarding하며 transport 실패는 `workerErrorInfo`로 typed `ErrorInfo` 변환한다. 새 메서드는 `sendTyped` 공유 경로를 그대로 쓰는 one-line forwarder다.
- [API-2] 번들 전략 resolver `BuiltInStrategyPort`는 `strategy-v1`(buy-and-hold: 종목별 첫 bar에서 1주 매수 후 hold)만 알고 미지 id는 error를 반환한다. command flow가 imported bars로부터 결정적(deterministic) terminal result를 단언할 수 있도록 최소·결정적으로 유지했다.
- [API-2] 실행은 비동기 규칙을 지켜 `BacktestStarter`가 pending run을 기록하고 import job과 동일한 `Runner`로 실행을 위임한다(두 번째 실행 엔진을 만들지 않음). worker entrypoint에서만 store-backed deps를 연결한다.
- [API-4] backtest 시나리오의 import step은 status 시나리오와 동일한 symbol/날짜(`005930`, `20240527`~`20240528`)를 사용하고 `strategy_id: strategy-v1`을 유지해 번들 resolver와 정렬했다.
## 사용자 리뷰 요청
_기본값은 `없음`이다. 구현 중 사용자 결정, 사용자 소유 외부 환경/secret/서비스 준비, 또는 계획 범위 변경 없이는 안전하게 진행할 수 없으면 아래 항목을 실제 내용으로 교체하고, 구현을 중단한 뒤 active 파일을 그대로 둔 채 리뷰를 요청한다. 후속 에이전트가 명령 재실행이나 산출물 수집으로 해소할 수 있는 검증 증거 공백만으로는 사용자 리뷰 요청을 작성하지 않는다._
- 상태: 없음
- 사유 유형: 없음
- 결정 필요: 없음
- 차단 근거: 없음
- 실행한 검증/명령: 없음
- 자동 후속 불가 이유: 없음
- 재개 조건: 없음
## 리뷰어를 위한 체크포인트
- `GetBacktestRunRequest` is registered and forwarded through API to worker.
- Worker process advertises backtest start only when `Starter` is actually wired.
- Backtest execution uses `StorageBarSource` and a bounded built-in strategy resolver, not API internals.
- YAML backtest scenarios import bars before start/poll/result.
- Result output includes run id, terminal status, and summary keys.
## 검증 결과
_구현 에이전트가 각 중간 검증 및 최종 검증 명령 실행 후 출력을 여기에 붙여 넣는다._
필수 규칙:
- 검증 명령은 고정된 계약이다. 임의로 대체하지 않는다.
- 대체가 필요하면 `계획 대비 변경 사항`에 이유와 대체 명령을 기록한다.
- `검증 결과`에는 실제 stdout/stderr를 붙여 넣는다.
- 사용자 리뷰 요청으로 명령을 끝까지 실행하지 못했다면 `사용자 리뷰 요청`에 실행한 명령, 실제 출력, 미실행 명령의 사유를 기록한다.
- mobile/UI hang, timeout, 또는 2분 무진행은 blind retry를 중단하고 focused rerun 명령과 screenshot/window/UI-tree evidence path를 남기며, 불가능하면 정확한 사유를 남긴다.
### API-1 중간 검증
```bash
$ go test ./services/api/internal/workerclient ./services/api/internal/socket ./services/worker/internal/socket -run 'GetBacktestRun|BacktestHandlers|Capabilities' -count=1
ok git.toki-labs.com/toki/alt/services/api/internal/workerclient 0.027s
ok git.toki-labs.com/toki/alt/services/api/internal/socket 0.018s
ok git.toki-labs.com/toki/alt/services/worker/internal/socket 0.017s
```
### API-2 중간 검증
```bash
$ go test ./services/worker/internal/backtest ./services/worker/internal/jobs ./services/worker/internal/socket -count=1
ok git.toki-labs.com/toki/alt/services/worker/internal/backtest 0.004s
ok git.toki-labs.com/toki/alt/services/worker/internal/jobs 0.003s
ok git.toki-labs.com/toki/alt/services/worker/internal/socket 0.077s
```
### API-3 중간 검증
```bash
$ go test ./services/worker/... -run 'Backtest.*Flow|StorageBarSource|RunBacktest|BacktestStarter' -count=1
ok git.toki-labs.com/toki/alt/services/worker/internal/backtest 0.004s
ok git.toki-labs.com/toki/alt/services/worker/internal/jobs 0.008s
ok git.toki-labs.com/toki/alt/services/worker/internal/socket 0.011s
(그 외 패키지는 [no test files]/[no tests to run])
```
### API-4 중간 검증
```bash
$ go test ./apps/cli/internal/operator -run 'Backtest|Handoff' -count=1
ok git.toki-labs.com/toki/alt/apps/cli/internal/operator 0.148s
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_run_polling.yaml
scenario=backtest_run_polling status=valid steps=3
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_result_summary.yaml
scenario=backtest_result_summary status=valid steps=3
```
### API-5 중간 검증
```bash
$ git diff --check
(출력 없음 / no warnings)
```
### 최종 검증
```bash
$ go test ./services/api/... -count=1
? git.toki-labs.com/toki/alt/services/api/cmd/alt-api [no test files]
ok git.toki-labs.com/toki/alt/services/api/internal/config 0.003s
ok git.toki-labs.com/toki/alt/services/api/internal/contracts 0.026s
ok git.toki-labs.com/toki/alt/services/api/internal/socket 0.042s
ok git.toki-labs.com/toki/alt/services/api/internal/workerclient 0.435s
$ go test ./services/worker/... -count=1
ok git.toki-labs.com/toki/alt/services/worker/cmd/alt-worker 0.010s
? git.toki-labs.com/toki/alt/services/worker/cmd/alt-worker-data-check [no test files]
ok git.toki-labs.com/toki/alt/services/worker/cmd/alt-worker-migrate 0.004s
ok git.toki-labs.com/toki/alt/services/worker/internal/backtest 0.004s
ok git.toki-labs.com/toki/alt/services/worker/internal/config 0.005s
ok git.toki-labs.com/toki/alt/services/worker/internal/contracts 0.009s
ok git.toki-labs.com/toki/alt/services/worker/internal/jobs 0.004s
ok git.toki-labs.com/toki/alt/services/worker/internal/marketdata/datacheck 0.006s
ok git.toki-labs.com/toki/alt/services/worker/internal/marketdata/importer 0.004s
ok git.toki-labs.com/toki/alt/services/worker/internal/providers/kis 0.042s
ok git.toki-labs.com/toki/alt/services/worker/internal/rediskeys 0.004s
ok git.toki-labs.com/toki/alt/services/worker/internal/socket 0.070s
? git.toki-labs.com/toki/alt/services/worker/internal/storage [no test files]
ok git.toki-labs.com/toki/alt/services/worker/internal/storage/postgres 0.366s
? git.toki-labs.com/toki/alt/services/worker/internal/storage/postgres/sqlc [no test files]
$ go test ./apps/cli/... -count=1
? git.toki-labs.com/toki/alt/apps/cli/cmd/alt [no test files]
ok git.toki-labs.com/toki/alt/apps/cli/internal/cli 0.016s
ok git.toki-labs.com/toki/alt/apps/cli/internal/operator 0.195s
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_run_polling.yaml
scenario=backtest_run_polling status=valid steps=3
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_result_summary.yaml
scenario=backtest_result_summary status=valid steps=3
$ git diff --check
(출력 없음 / no warnings)
```
---
> **[IMPLEMENTING AGENT — BEFORE SAVING] Have you filled in every implementation-owned section: completion table, implementation checklist, changes from plan, design decisions, and verification output?**
> If anything is blank, go back and fill it in before saving this file.
> Leave review-agent-only sections unchanged.
## 코드리뷰 결과
- 종합 판정: PASS
- 차원별 평가:
- Correctness: Pass
- Completeness: Pass
- Test coverage: Pass
- API contract: Pass
- Code quality: Pass
- Plan deviation: Pass
- Verification trust: Pass
- 발견된 문제: 없음
- 다음 단계: PASS이므로 active plan/review를 아카이브하고 `complete.log` 작성 후 task directory를 archive로 이동한다.

View file

@ -0,0 +1,46 @@
# Complete - m-command-first-operator-workflow/04+02_backtest_runtime_flow
## 완료 일시
2026-06-03
## 요약
Backtest runtime flow task completed on the first review loop; final verdict PASS.
## 루프 이력
| Plan | Review | Verdict | 메모 |
|------|--------|---------|------|
| `plan_cloud_G08_0.log` | `code_review_cloud_G08_0.log` | PASS | Added API/worker polling rail, store-backed worker execution wiring, built-in strategy resolver, command-flow tests, and import-first backtest fixtures. |
## 구현/정리 내용
- Added API and worker `GetBacktestRun` forwarding/handler support for command polling.
- Wired `alt-worker` store-backed deps with `BacktestStarter`, `RegisterRunBacktestHandler`, `StorageBarSource`, result store, and the bundled `strategy-v1` resolver.
- Added focused worker flow, strategy, wiring, API forwarding, and CLI fixture tests for imported bars -> start/poll/result.
- Updated backtest operator YAML fixtures and expected JSONL to import daily bars before run start/result checks.
## 최종 검증
- `go test ./services/api/... -count=1` - PASS; API command package had no tests, config/contracts/socket/workerclient packages passed.
- `go test ./services/worker/... -count=1` - PASS; worker command/internal packages passed, with no-test packages reported for data-check and storage/sqlc surfaces.
- `go test ./apps/cli/... -count=1` - PASS; CLI command package had no tests, internal cli/operator packages passed.
- `go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_run_polling.yaml` - PASS; `scenario=backtest_run_polling status=valid steps=3`.
- `go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_result_summary.yaml` - PASS; `scenario=backtest_result_summary status=valid steps=3`.
- `git diff --check` - PASS; no output.
## Roadmap Completion
- Milestone: `agent-roadmap/phase/operator-surface/milestones/command-first-operator-workflow.md`
- Completed task ids:
- `backtest-command-flow`: PASS; evidence=`agent-task/archive/2026/06/m-command-first-operator-workflow/04+02_backtest_runtime_flow/plan_cloud_G08_0.log`, `agent-task/archive/2026/06/m-command-first-operator-workflow/04+02_backtest_runtime_flow/code_review_cloud_G08_0.log`; verification=`go test ./services/api/... -count=1`, `go test ./services/worker/... -count=1`, `go test ./apps/cli/... -count=1`, `go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_run_polling.yaml`, `go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_result_summary.yaml`, `git diff --check`
- Not completed task ids: 없음
## 잔여 Nit
- 없음
## 후속 작업
- 없음

View file

@ -1,156 +0,0 @@
<!-- task=m-command-first-operator-workflow/04+02_backtest_runtime_flow plan=0 tag=API -->
# Code Review Reference - API
> **[IMPLEMENTING AGENT — READ FIRST] Filling in this file is the mandatory final step of implementation.**
> The task is NOT complete until every implementation-owned section below is filled in.
> Complete the `구현 체크리스트`; the final checklist item is mandatory before saving.
> Fill implementation-owned sections, then stop with active files in place and report ready for review.
> If implementation is blocked by a user-only decision, user-owned external environment prerequisite, or scope conflict, fill `사용자 리뷰 요청` with evidence and stop with active files in place; code-review decides whether to write `USER_REVIEW.md`. Evidence gaps that a follow-up agent can close by rerunning commands or collecting artifacts are normal follow-up issues, not user-review blockers by themselves.
> Finalization (`코드리뷰 결과`, log rename, `complete.log`, archive moves, `코드리뷰 전용 체크리스트`) is review-agent-only, even after compaction/resume.
> Follow the ownership table at the bottom of this file for which sections you own.
## 개요
date=2026-06-03
task=m-command-first-operator-workflow/04+02_backtest_runtime_flow, plan=0, tag=API
## Roadmap Targets
- Milestone: `agent-roadmap/phase/operator-surface/milestones/command-first-operator-workflow.md`
- Task ids:
- `backtest-command-flow`: imported daily bars를 입력으로 backtest start/poll/result를 command workflow에서 확인할 수 있다.
- Completion mode: check-on-pass
## 이 파일을 읽는 리뷰 에이전트에게
> **[REVIEW AGENT ONLY]** 아래 종결 절차는 코드리뷰 에이전트 전용이다. 구현 에이전트는 이 섹션을 실행하지 않는다.
각 항목의 구현을 실제 소스 파일과 대조하고, `검증 결과` 섹션의 출력이 코드와 일치하는지 확인하세요.
---
## 구현 항목별 완료 여부
| 항목 | 완료 여부 |
|------|---------|
| [API-1] GetBacktestRun Rail | [ ] |
| [API-2] Worker Runtime Wiring | [ ] |
| [API-3] Execution Evidence Tests | [ ] |
| [API-4] Backtest YAML Flow | [ ] |
| [API-5] Verification | [ ] |
## 구현 체크리스트
- [ ] [API-1] Add API and worker `GetBacktestRun` forwarding/handler support for polling.
- [ ] [API-2] Wire worker `alt-worker` with `BacktestStarter`, `RegisterRunBacktestHandler`, `StorageBarSource`, result store, and a built-in strategy resolver.
- [ ] [API-3] Add/update tests for built-in strategy resolver, worker execution wiring, and typed run polling.
- [ ] [API-4] Update backtest YAML scenarios to import daily bars before start/poll/result. 검증: start/poll/result scenario가 run id, terminal status, result summary key를 출력한다.
- [ ] [API-5] Run API/worker/CLI verification and `git diff --check`.
- [ ] CODE_REVIEW-*-G??.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채운다. 이 항목이 완료되기 전에는 구현이 완료된 것이 아니다.
## 코드리뷰 전용 체크리스트
> **[REVIEW AGENT ONLY]** 이 체크리스트는 코드리뷰 에이전트만 사용한다.
> 구현 에이전트는 이 섹션을 수정하거나 체크하지 않는다.
- [ ] `코드리뷰 결과``PASS`, `WARN`, `FAIL` 중 하나의 판정을 append한다.
- [ ] 판정과 `차원별 평가`, Required/Suggested/Nit 분류가 서로 일치한다.
- [ ] active `CODE_REVIEW-*-G??.md``code_review_cloud_G08_N.log`로 아카이브한다.
- [ ] active `PLAN-*-G??.md``plan_cloud_G08_M.log`로 아카이브한다.
- [ ] `.gitignore`의 Agent-Ops 관리 block이 `agent-task/**/*.md``agent-task/**/*.log`를 unignore하여 plan/review/archive 산출물이 추적 가능한지 확인한다.
- [ ] PASS이면 `agent-ops/skills/common/code-review/templates/complete-log-template.md` 기준으로 `complete.log`를 작성하고 active `.md` 파일을 남기지 않는다.
- [ ] PASS이면 active task 디렉터리 `agent-task/m-command-first-operator-workflow/04+02_backtest_runtime_flow/``agent-task/archive/YYYY/MM/m-command-first-operator-workflow/04+02_backtest_runtime_flow/`로 이동하고 최종 archive 경로에서 이 체크리스트를 갱신한다.
- [ ] PASS이고 task group이 `m-<milestone-slug>`이면 런타임이 읽을 완료 이벤트 메타데이터를 보고하고, roadmap 수정이나 `update-roadmap` 직접 호출을 하지 않는다.
- [ ] PASS split 작업이면 이동 후 빈 active parent `agent-task/m-command-first-operator-workflow/`를 제거하거나, 남은 sibling/file이 있어 유지했다고 확인한다.
- [ ] WARN/FAIL이고 user-review gate가 트리거되지 않았으면 다음 active `PLAN-cloud-G08.md``CODE_REVIEW-cloud-G08.md`를 작성하고 `complete.log`를 작성하지 않는다.
- [ ] USER_REVIEW이면 `agent-ops/skills/common/code-review/templates/user-review-template.md` 기준으로 `USER_REVIEW.md`를 작성하고 active `PLAN-*.md`, `CODE_REVIEW-*.md`, `complete.log`를 남기지 않는다.
- [ ] USER_REVIEW가 사용자 결정으로 완료/PASS 해소되면 `USER_REVIEW.md`를 해소 상태로 갱신하고 `complete.log`를 작성한 뒤 task directory를 archive로 이동한다.
## 계획 대비 변경 사항
_구현 에이전트가 계획과 다르게 구현한 부분을 이유와 함께 기록한다._
## 주요 설계 결정
_구현 에이전트가 주요 설계 결정 사항을 기록한다._
## 사용자 리뷰 요청
_기본값은 `없음`이다. 구현 중 사용자 결정, 사용자 소유 외부 환경/secret/서비스 준비, 또는 계획 범위 변경 없이는 안전하게 진행할 수 없으면 아래 항목을 실제 내용으로 교체하고, 구현을 중단한 뒤 active 파일을 그대로 둔 채 리뷰를 요청한다. 후속 에이전트가 명령 재실행이나 산출물 수집으로 해소할 수 있는 검증 증거 공백만으로는 사용자 리뷰 요청을 작성하지 않는다._
- 상태: 없음
- 사유 유형: 없음
- 결정 필요: 없음
- 차단 근거: 없음
- 실행한 검증/명령: 없음
- 자동 후속 불가 이유: 없음
- 재개 조건: 없음
## 리뷰어를 위한 체크포인트
- `GetBacktestRunRequest` is registered and forwarded through API to worker.
- Worker process advertises backtest start only when `Starter` is actually wired.
- Backtest execution uses `StorageBarSource` and a bounded built-in strategy resolver, not API internals.
- YAML backtest scenarios import bars before start/poll/result.
- Result output includes run id, terminal status, and summary keys.
## 검증 결과
_구현 에이전트가 각 중간 검증 및 최종 검증 명령 실행 후 출력을 여기에 붙여 넣는다._
필수 규칙:
- 검증 명령은 고정된 계약이다. 임의로 대체하지 않는다.
- 대체가 필요하면 `계획 대비 변경 사항`에 이유와 대체 명령을 기록한다.
- `검증 결과`에는 실제 stdout/stderr를 붙여 넣는다.
- 사용자 리뷰 요청으로 명령을 끝까지 실행하지 못했다면 `사용자 리뷰 요청`에 실행한 명령, 실제 출력, 미실행 명령의 사유를 기록한다.
- mobile/UI hang, timeout, 또는 2분 무진행은 blind retry를 중단하고 focused rerun 명령과 screenshot/window/UI-tree evidence path를 남기며, 불가능하면 정확한 사유를 남긴다.
### API-1 중간 검증
```bash
$ go test ./services/api/internal/workerclient ./services/api/internal/socket ./services/worker/internal/socket -run 'GetBacktestRun|BacktestHandlers|Capabilities' -count=1
(output)
```
### API-2 중간 검증
```bash
$ go test ./services/worker/internal/backtest ./services/worker/internal/jobs ./services/worker/internal/socket -count=1
(output)
```
### API-3 중간 검증
```bash
$ go test ./services/worker/... -run 'Backtest.*Flow|StorageBarSource|RunBacktest|BacktestStarter' -count=1
(output)
```
### API-4 중간 검증
```bash
$ go test ./apps/cli/internal/operator -run 'Backtest|Handoff' -count=1
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_run_polling.yaml
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_result_summary.yaml
(output)
```
### API-5 중간 검증
```bash
$ git diff --check
(output)
```
### 최종 검증
```bash
$ go test ./services/api/... -count=1
$ go test ./services/worker/... -count=1
$ go test ./apps/cli/... -count=1
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_run_polling.yaml
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_result_summary.yaml
$ git diff --check
(output)
```
---
> **[IMPLEMENTING AGENT — BEFORE SAVING] Have you filled in every implementation-owned section: completion table, implementation checklist, changes from plan, design decisions, and verification output?**
> If anything is blank, go back and fill it in before saving this file.
> Leave review-agent-only sections unchanged.

View file

@ -39,6 +39,21 @@ type fakeAPI struct {
startBacktestReq *altv1.StartBacktestRequest
getBacktestRunReq *altv1.GetBacktestRunRequest
importDailyBarsReq *altv1.ImportDailyBarsRequest
calls []string
}
// recordCall appends an action name in receive order so tests can assert that a
// command-first scenario issued its import before any status read.
func (f *fakeAPI) recordCall(action string) {
f.mu.Lock()
defer f.mu.Unlock()
f.calls = append(f.calls, action)
}
func (f *fakeAPI) callOrder() []string {
f.mu.Lock()
defer f.mu.Unlock()
return append([]string(nil), f.calls...)
}
func (f *fakeAPI) setInstReq(req *altv1.ListInstrumentsRequest) {
@ -128,6 +143,7 @@ func startFakeAPIServer(t *testing.T, api *fakeAPI) string {
}, nil
})
protoSocket.AddRequestListenerTyped[*altv1.ListInstrumentsRequest, *altv1.ListInstrumentsResponse](&c.Communicator, func(req *altv1.ListInstrumentsRequest) (*altv1.ListInstrumentsResponse, error) {
api.recordCall("list_instruments")
api.setInstReq(req)
if api.instResp != nil {
return api.instResp, nil
@ -135,6 +151,7 @@ func startFakeAPIServer(t *testing.T, api *fakeAPI) string {
return &altv1.ListInstrumentsResponse{}, nil
})
protoSocket.AddRequestListenerTyped[*altv1.ListBarsRequest, *altv1.ListBarsResponse](&c.Communicator, func(req *altv1.ListBarsRequest) (*altv1.ListBarsResponse, error) {
api.recordCall("list_bars")
api.setBarsReq(req)
if api.barsResp != nil {
return api.barsResp, nil
@ -183,6 +200,7 @@ func startFakeAPIServer(t *testing.T, api *fakeAPI) string {
return &altv1.CompareBacktestRunsResponse{}, nil
})
protoSocket.AddRequestListenerTyped[*altv1.ImportDailyBarsRequest, *altv1.ImportDailyBarsResponse](&c.Communicator, func(req *altv1.ImportDailyBarsRequest) (*altv1.ImportDailyBarsResponse, error) {
api.recordCall("import_daily_bars")
api.setImportDailyBarsReq(req)
if api.importDailyBarsResp != nil {
return api.importDailyBarsResp, nil

View file

@ -110,6 +110,64 @@ func TestExpectedOutputFixturesAreValidJSONLines(t *testing.T) {
}
}
// TestStatusScenarioFixtureImportsBeforeReads guards the command-first status
// scenario: its expected fixture must carry an import_daily_bars step line with
// instrument_count and bar_count keys before the instrument/bar read lines, so
// the handoff evidence shows imported counts rather than a bare status read.
func TestStatusScenarioFixtureImportsBeforeReads(t *testing.T) {
const scenario = "market_data_status_query"
f, err := os.Open(expectedFixturePath(scenario))
if err != nil {
t.Fatalf("open expected fixture: %v", err)
}
defer f.Close()
scanner := bufio.NewScanner(f)
importLine := -1
instLine := -1
barLine := -1
lineNo := 0
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" {
continue
}
lineNo++
var rec map[string]any
if err := json.Unmarshal([]byte(line), &rec); err != nil {
t.Fatalf("line %d not valid JSON: %v", lineNo, err)
}
switch rec["action"] {
case "import_daily_bars":
importLine = lineNo
if _, ok := rec["instrument_count"]; !ok {
t.Errorf("import step line %d missing instrument_count", lineNo)
}
if _, ok := rec["bar_count"]; !ok {
t.Errorf("import step line %d missing bar_count", lineNo)
}
case "list_instruments":
instLine = lineNo
case "list_bars":
barLine = lineNo
}
}
if err := scanner.Err(); err != nil {
t.Fatalf("scan fixture: %v", err)
}
if importLine < 0 {
t.Fatal("status fixture has no import_daily_bars step line")
}
if instLine < 0 || barLine < 0 {
t.Fatal("status fixture missing instrument/bar read step lines")
}
if importLine > instLine || importLine > barLine {
t.Errorf("import line %d must precede read lines (instruments=%d, bars=%d)", importLine, instLine, barLine)
}
}
// TestHandoffDocumentedScenarioFixturesExist guards the handoff matrix against
// referencing fixture paths that do not exist. Each required scenario must have
// an input YAML fixture and an expected output fixture on disk, and the matrix

View file

@ -59,6 +59,68 @@ func TestRunBacktestRequestOutputsRunID(t *testing.T) {
}
}
func TestRunBacktestImportThenStartFlow(t *testing.T) {
api := &fakeAPI{
importDailyBarsResp: &altv1.ImportDailyBarsResponse{Provider: "kis", InstrumentCount: 1, BarCount: 42},
startBacktestResp: &altv1.StartBacktestResponse{
Run: &altv1.BacktestRun{
Id: "run-12345",
Status: altv1.BacktestRunStatus_BACKTEST_RUN_STATUS_PENDING,
},
},
}
url := startFakeAPIServer(t, api)
sc := &Scenario{
Name: "backtest_run_request",
Steps: []Step{
{
ID: "import_kr_daily_bars",
Action: ActionImportDailyBars,
Request: Request{
Provider: "kis",
SelectorKind: "watchlist",
Market: "kr",
Venue: "krx",
Symbols: []string{"005930"},
FromYYYYMMDD: "20240527",
ToYYYYMMDD: "20240528",
},
Expect: Expect{Status: "ok"},
},
{
ID: "start_run",
Action: ActionStartBacktest,
Request: Request{
StrategyID: "strategy-v1",
Market: "kr",
Timeframe: "daily",
FromUnixMs: 1746057600000,
ToUnixMs: 1747267200000,
},
Expect: Expect{Status: "ok"},
},
},
}
out, code := runScenario(t, sc, url)
if code != codeOK {
t.Fatalf("exit code = %d, want 0 (out=%q)", code, out)
}
// The import step runs before the start step and reports imported counts.
if !strings.Contains(out, "instrument_count=1") || !strings.Contains(out, "bar_count=42") {
t.Errorf("output %q missing import counts", out)
}
if !strings.Contains(out, "run.id=run-12345") || !strings.Contains(out, "run.status=pending") {
t.Errorf("output %q missing start run id/status", out)
}
if api.lastImportDailyBarsReq() == nil {
t.Error("server did not receive an import request before start")
}
if api.lastStartBacktestReq() == nil {
t.Error("server did not receive a start backtest request")
}
}
func TestRunBacktestPollingStopsOnSucceeded(t *testing.T) {
var mu sync.Mutex
pollCount := 0

View file

@ -45,9 +45,10 @@ func TestRunScenarioAPISmoke(t *testing.T) {
func TestRunScenarioMarketDataStatus(t *testing.T) {
url := startFakeAPIServer(t, &fakeAPI{
helloCaps: []string{"market-read"},
instResp: &altv1.ListInstrumentsResponse{Instruments: []*altv1.Instrument{{Id: "KRX:005930"}}},
barsResp: &altv1.ListBarsResponse{Bars: []*altv1.Bar{{InstrumentId: "KRX:005930"}}},
helloCaps: []string{"market-read"},
importDailyBarsResp: &altv1.ImportDailyBarsResponse{Provider: "kis", InstrumentCount: 1, BarCount: 42},
instResp: &altv1.ListInstrumentsResponse{Instruments: []*altv1.Instrument{{Id: "KRX:005930"}}},
barsResp: &altv1.ListBarsResponse{Bars: []*altv1.Bar{{InstrumentId: "KRX:005930"}}},
})
sc, err := LoadScenario(fixture(t, "market_data_status_query.yaml"))
if err != nil {
@ -60,8 +61,17 @@ func TestRunScenarioMarketDataStatus(t *testing.T) {
}
for _, want := range []string{
"scenario=market_data_status_query",
// import-before-read evidence: import counts are in the status output.
"step=import_kr_daily_bars",
"action=import_daily_bars",
"provider=kis",
"instrument_count=1",
"bar_count=42",
// list/read status counts still report.
"step=list_kr_instruments",
"action=list_instruments",
"step=list_daily_bars",
"action=list_bars",
"status=ok",
"count=1",
} {
@ -71,6 +81,88 @@ func TestRunScenarioMarketDataStatus(t *testing.T) {
}
}
// TestRunScenarioMarketDataStatusImportsBeforeReads proves the command-first
// status scenario issues its import_daily_bars call before the instrument and
// bar status reads, so the reported counts describe freshly imported data.
func TestRunScenarioMarketDataStatusImportsBeforeReads(t *testing.T) {
api := &fakeAPI{
helloCaps: []string{"market-read"},
importDailyBarsResp: &altv1.ImportDailyBarsResponse{Provider: "kis", InstrumentCount: 1, BarCount: 42},
instResp: &altv1.ListInstrumentsResponse{Instruments: []*altv1.Instrument{{Id: "KRX:005930"}}},
barsResp: &altv1.ListBarsResponse{Bars: []*altv1.Bar{{InstrumentId: "KRX:005930"}}},
}
url := startFakeAPIServer(t, api)
sc, err := LoadScenario(fixture(t, "market_data_status_query.yaml"))
if err != nil {
t.Fatalf("load scenario: %v", err)
}
out, code := runScenario(t, sc, url)
if code != codeOK {
t.Fatalf("exit code = %d, want 0 (out=%q)", code, out)
}
// The import actually carried the fixture's KIS window, not a default read.
imp := api.lastImportDailyBarsReq()
if imp == nil {
t.Fatal("server did not receive an import_daily_bars request")
}
if imp.GetProvider() != "kis" || imp.GetFromYyyymmdd() != "20240527" || imp.GetToYyyymmdd() != "20240528" {
t.Errorf("import request = %+v, want kis 20240527..20240528", imp)
}
// Call order: import precedes both status reads.
wantOrder := []string{"import_daily_bars", "list_instruments", "list_bars"}
if got := api.callOrder(); !equalStrings(got, wantOrder) {
t.Errorf("call order = %v, want %v (out=%q)", got, wantOrder, out)
}
}
// TestRunScenarioMarketStatusTypedUnavailable proves the status scenario surfaces
// a typed unavailable ErrorInfo in its machine-readable output when the import
// step fails, instead of degrading into a transport error.
func TestRunScenarioMarketStatusTypedUnavailable(t *testing.T) {
url := startFakeAPIServer(t, &fakeAPI{
importDailyBarsResp: &altv1.ImportDailyBarsResponse{
Error: &altv1.ErrorInfo{Code: "unavailable", Message: "market worker offline"},
},
})
sc := &Scenario{
Name: "market_status_typed_unavailable",
Steps: []Step{{
ID: "import_kr_daily_bars",
Action: ActionImportDailyBars,
Request: Request{
Provider: "kis",
SelectorKind: "watchlist",
Symbols: []string{"005930"},
},
Expect: Expect{Status: "error", ErrorCode: "unavailable"},
}},
}
out, code := runScenario(t, sc, url)
if code != codeOK {
t.Fatalf("exit code = %d, want 0 (out=%q)", code, out)
}
if !strings.Contains(out, "status=ok") || !strings.Contains(out, "error.code=unavailable") {
t.Errorf("output %q missing expected typed unavailable markers", out)
}
}
// equalStrings reports whether two string slices have the same elements in order.
func equalStrings(a, b []string) bool {
if len(a) != len(b) {
return false
}
for i := range a {
if a[i] != b[i] {
return false
}
}
return true
}
func TestRunScenarioMarketEmptyResult(t *testing.T) {
url := startFakeAPIServer(t, &fakeAPI{instResp: &altv1.ListInstrumentsResponse{}})
sc := &Scenario{

View file

@ -1,6 +1,18 @@
name: backtest_result_summary
timeout: 5s
steps:
- id: import_kr_daily_bars
action: import_daily_bars
request:
provider: kis
selector_kind: watchlist
market: kr
venue: krx
symbols: ["005930"]
from_yyyymmdd: "20240527"
to_yyyymmdd: "20240528"
expect:
status: ok
- id: start
action: start_backtest
request:

View file

@ -1,6 +1,18 @@
name: backtest_run_polling
timeout: 5s
steps:
- id: import_kr_daily_bars
action: import_daily_bars
request:
provider: kis
selector_kind: watchlist
market: kr
venue: krx
symbols: ["005930"]
from_yyyymmdd: "20240527"
to_yyyymmdd: "20240528"
expect:
status: ok
- id: start
action: start_backtest
request:

View file

@ -1,6 +1,18 @@
name: backtest_run_request
timeout: 5s
steps:
- id: import_kr_daily_bars
action: import_daily_bars
request:
provider: kis
selector_kind: watchlist
market: kr
venue: krx
symbols: ["005930"]
from_yyyymmdd: "20240527"
to_yyyymmdd: "20240528"
expect:
status: ok
- id: start_run
action: start_backtest
request:

View file

@ -1,3 +1,4 @@
{"action":"import_daily_bars","bar_count":42,"instrument_count":1,"provider":"kis","scenario":"backtest_result_summary","status":"ok","step":"import_kr_daily_bars","type":"step"}
{"action":"start_backtest","run_id":"run-0001","run_status":"pending","scenario":"backtest_result_summary","status":"ok","step":"start","type":"step"}
{"action":"get_backtest_result","ending_equity":"1125000","run_id":"run-0001","scenario":"backtest_result_summary","starting_cash":"1000000","status":"ok","step":"result","total_return":"0.125","trade_count":12,"type":"step"}
{"exit_code":0,"passed":2,"scenario":"backtest_result_summary","status":"ok","steps":2,"type":"summary"}
{"exit_code":0,"passed":3,"scenario":"backtest_result_summary","status":"ok","steps":3,"type":"summary"}

View file

@ -1,3 +1,4 @@
{"action":"import_daily_bars","bar_count":42,"instrument_count":1,"provider":"kis","scenario":"backtest_run_polling","status":"ok","step":"import_kr_daily_bars","type":"step"}
{"action":"start_backtest","run_id":"run-0001","run_status":"pending","scenario":"backtest_run_polling","status":"ok","step":"start","type":"step"}
{"action":"poll_backtest_run","run_id":"run-0001","run_status":"succeeded","scenario":"backtest_run_polling","status":"ok","step":"poll","type":"step"}
{"exit_code":0,"passed":2,"scenario":"backtest_run_polling","status":"ok","steps":2,"type":"summary"}
{"exit_code":0,"passed":3,"scenario":"backtest_run_polling","status":"ok","steps":3,"type":"summary"}

View file

@ -1,2 +1,3 @@
{"action":"import_daily_bars","bar_count":42,"instrument_count":1,"provider":"kis","scenario":"backtest_run_request","status":"ok","step":"import_kr_daily_bars","type":"step"}
{"action":"start_backtest","run_id":"run-0001","run_status":"pending","scenario":"backtest_run_request","status":"ok","step":"start_run","type":"step"}
{"exit_code":0,"passed":1,"scenario":"backtest_run_request","status":"ok","steps":1,"type":"summary"}
{"exit_code":0,"passed":2,"scenario":"backtest_run_request","status":"ok","steps":2,"type":"summary"}

View file

@ -1,4 +1,5 @@
{"action":"hello","scenario":"market_data_status_query","status":"ok","step":"hello","type":"step"}
{"action":"import_daily_bars","bar_count":42,"instrument_count":1,"provider":"kis","scenario":"market_data_status_query","status":"ok","step":"import_kr_daily_bars","type":"step"}
{"action":"list_instruments","count":1,"scenario":"market_data_status_query","status":"ok","step":"list_kr_instruments","type":"step"}
{"action":"list_bars","count":10,"scenario":"market_data_status_query","status":"ok","step":"list_daily_bars","type":"step"}
{"exit_code":0,"passed":3,"scenario":"market_data_status_query","status":"ok","steps":3,"type":"summary"}
{"exit_code":0,"passed":4,"scenario":"market_data_status_query","status":"ok","steps":4,"type":"summary"}

View file

@ -26,7 +26,7 @@ ALT 운영 UI 게이트(`agent-ops/rules/project/rules.md`의 "운영 UI 구현
| Scenario | Command | Input fixture | Expected output fixture | Expected exit code | Checked protobuf/view-model field | Remaining Flutter wireframe dependency |
|----------|---------|---------------|-------------------------|--------------------|-----------------------------------|----------------------------------------|
| `api_connection_smoke` | `alt operator scenario run --file testdata/operator/api_connection_smoke.yaml --api-url ws://127.0.0.1:8080/socket --output jsonl` | `testdata/operator/api_connection_smoke.yaml` | `testdata/operator/expected/api_connection_smoke.jsonl` | `0` (성공), `3` (transport 실패 시) | `HelloResponse.capabilities` (`alt.v1`); summary `status`/`exit_code` | connection status / capability 표시 화면 layout 미정 |
| `market_data_status_query` | `alt operator scenario run --file testdata/operator/market_data_status_query.yaml --api-url ws://127.0.0.1:8080/socket --output jsonl` | `testdata/operator/market_data_status_query.yaml` | `testdata/operator/expected/market_data_status_query.jsonl` | `0` (성공/empty), `1` (mismatch) | `ListInstrumentsResponse.instruments`, `ListBarsResponse.bars` (step `count`), `ErrorInfo.code` | market data status card / 빈 상태·unavailable 시각화 미정 |
| `market_data_status_query` | `alt operator scenario run --file testdata/operator/market_data_status_query.yaml --api-url ws://127.0.0.1:8080/socket --output jsonl` | `testdata/operator/market_data_status_query.yaml` | `testdata/operator/expected/market_data_status_query.jsonl` | `0` (성공/empty), `1` (mismatch) | command-first: `import_daily_bars`를 먼저 실행해 `ImportDailyBarsResponse.instrument_count`/`ImportDailyBarsResponse.bar_count`를 확인한 뒤 `ListInstrumentsResponse.instruments`, `ListBarsResponse.bars` (step `count`) 조회; typed 실패 시 `ErrorInfo.code` (`unavailable`) | market data status card / 수집 결과(종목수·바개수)·빈 상태·unavailable 시각화 미정 |
| `backtest_run_request` | `alt operator scenario run --file testdata/operator/backtest_run_request.yaml --api-url ws://127.0.0.1:8080/socket --output jsonl` | `testdata/operator/backtest_run_request.yaml` | `testdata/operator/expected/backtest_run_request.jsonl` | `0` (성공), `1` (typed error/mismatch) | `StartBacktestResponse.run.id`, `BacktestRun.status` (step `run_id`/`run_status`), `ErrorInfo.code` | backtest 생성 form 입력/검증 화면 미정 |
| `backtest_run_polling` | `alt operator scenario run --file testdata/operator/backtest_run_polling.yaml --api-url ws://127.0.0.1:8080/socket --output jsonl` | `testdata/operator/backtest_run_polling.yaml` | `testdata/operator/expected/backtest_run_polling.jsonl` | `0` (terminal 도달), `3` (polling timeout) | `GetBacktestRunResponse.run.status` 전이(pending→running→succeeded/failed/canceled) | run 진행 상태/전이 표시(진행바, 상태 칩) 화면 미정 |
| `backtest_result_summary` | `alt operator scenario run --file testdata/operator/backtest_result_summary.yaml --api-url ws://127.0.0.1:8080/socket --output jsonl` | `testdata/operator/backtest_result_summary.yaml` | `testdata/operator/expected/backtest_result_summary.jsonl` | `0` (성공), `1` (missing run/typed error) | `GetBacktestResultResponse.result.summary` (`starting_cash`, `ending_equity`, `total_return`, `trade_count`) | result summary metrics / chart 시각화 화면 미정 |

View file

@ -1,7 +1,10 @@
# Market data status scenario. It greets the API, then reads instruments and
# bars so an operator can confirm market data status from the terminal without a
# screen. `alt operator scenario run --file <this> --api-url <ws-url>` executes
# it; `alt operator scenario validate --file <this>` checks it offline.
# Market data status scenario. It greets the API, imports KIS daily bars, then
# reads instruments and bars so an operator can confirm imported market data
# status from the terminal without a screen. Running the import first makes this
# a command-first status workflow rather than a generic read of whatever data
# already happens to be stored. `alt operator scenario run --file <this> --api-url
# <ws-url>` executes it; `alt operator scenario validate --file <this>` checks it
# offline.
name: market_data_status_query
timeout: 5s
steps:
@ -11,6 +14,18 @@ steps:
status: ok
capabilities:
- market-read
- id: import_kr_daily_bars
action: import_daily_bars
request:
provider: kis
selector_kind: watchlist
market: kr
venue: krx
symbols: ["005930"]
from_yyyymmdd: "20240527"
to_yyyymmdd: "20240528"
expect:
status: ok
- id: list_kr_instruments
action: list_instruments
request:

View file

@ -36,6 +36,14 @@ func backtestHandlers(worker workerclient.WorkerClient) []sessionHandler {
})
},
},
{
requestType: protoSocket.TypeNameOf(&altv1.GetBacktestRunRequest{}),
register: func(client *protoSocket.WsClient) {
protoSocket.AddRequestListenerTyped[*altv1.GetBacktestRunRequest, *altv1.GetBacktestRunResponse](&client.Communicator, func(req *altv1.GetBacktestRunRequest) (*altv1.GetBacktestRunResponse, error) {
return handleGetBacktestRun(worker, req)
})
},
},
{
requestType: protoSocket.TypeNameOf(&altv1.ListBacktestRunsRequest{}),
register: func(client *protoSocket.WsClient) {
@ -93,6 +101,28 @@ func handleStartBacktest(worker workerclient.WorkerClient, req *altv1.StartBackt
return res, nil
}
func handleGetBacktestRun(worker workerclient.WorkerClient, req *altv1.GetBacktestRunRequest) (*altv1.GetBacktestRunResponse, error) {
if req.GetRunId() == "" {
return &altv1.GetBacktestRunResponse{Error: invalidRequest("run_id is required")}, nil
}
if worker == nil {
return &altv1.GetBacktestRunResponse{Error: workerUnavailable()}, nil
}
ctx, cancel := context.WithTimeout(context.Background(), workerRequestTimeout)
defer cancel()
if err := worker.Connect(ctx); err != nil {
return &altv1.GetBacktestRunResponse{Error: workerErrorInfo(err)}, nil
}
res, err := worker.GetBacktestRun(ctx, req)
if err != nil {
return &altv1.GetBacktestRunResponse{Error: workerErrorInfo(err)}, nil
}
if res == nil {
return &altv1.GetBacktestRunResponse{Error: internalError("worker returned no run response")}, nil
}
return res, nil
}
func handleListBacktestRuns(worker workerclient.WorkerClient, req *altv1.ListBacktestRunsRequest) (*altv1.ListBacktestRunsResponse, error) {
if worker == nil {
return &altv1.ListBacktestRunsResponse{Error: workerUnavailable()}, nil

View file

@ -16,6 +16,7 @@ import (
// the last forwarded request and returns canned responses/errors.
type fakeWorkerClient struct {
startReq *altv1.StartBacktestRequest
getRunReq *altv1.GetBacktestRunRequest
listReq *altv1.ListBacktestRunsRequest
detailReq *altv1.GetBacktestRunDetailRequest
resultReq *altv1.GetBacktestResultRequest
@ -25,6 +26,7 @@ type fakeWorkerClient struct {
importReq *altv1.ImportDailyBarsRequest
startRes *altv1.StartBacktestResponse
getRunRes *altv1.GetBacktestRunResponse
listRes *altv1.ListBacktestRunsResponse
detailRes *altv1.GetBacktestRunDetailResponse
resultRes *altv1.GetBacktestResultResponse
@ -59,6 +61,10 @@ func (f *fakeWorkerClient) StartBacktest(ctx context.Context, req *altv1.StartBa
f.startReq = req
return f.startRes, f.err
}
func (f *fakeWorkerClient) GetBacktestRun(ctx context.Context, req *altv1.GetBacktestRunRequest) (*altv1.GetBacktestRunResponse, error) {
f.getRunReq = req
return f.getRunRes, f.err
}
func (f *fakeWorkerClient) ListBacktestRuns(ctx context.Context, req *altv1.ListBacktestRunsRequest) (*altv1.ListBacktestRunsResponse, error) {
f.listReq = req
return f.listRes, f.err
@ -199,6 +205,47 @@ func TestHandleListBacktestRunsForwards(t *testing.T) {
}
}
func TestHandleGetBacktestRunForwards(t *testing.T) {
fake := &fakeWorkerClient{getRunRes: &altv1.GetBacktestRunResponse{Run: &altv1.BacktestRun{Id: "run-1", Status: altv1.BacktestRunStatus_BACKTEST_RUN_STATUS_RUNNING}}}
resp, err := handleGetBacktestRun(fake, &altv1.GetBacktestRunRequest{RunId: "run-1"})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if fake.getRunReq.GetRunId() != "run-1" {
t.Errorf("run id was not forwarded unchanged, got %q", fake.getRunReq.GetRunId())
}
if resp.GetRun().GetId() != "run-1" {
t.Errorf("unexpected run id: %q", resp.GetRun().GetId())
}
}
func TestHandleGetBacktestRunRequiresRunID(t *testing.T) {
fake := &fakeWorkerClient{}
resp, err := handleGetBacktestRun(fake, &altv1.GetBacktestRunRequest{})
if err != nil {
t.Fatalf("expected typed validation response, got error: %v", err)
}
requireBacktestError(t, resp.GetError(), backtestErrorInvalidRequest)
if fake.getRunReq != nil {
t.Error("worker must not be called when validation fails")
}
}
func TestHandleGetBacktestRunNilWorkerResponse(t *testing.T) {
fake := &fakeWorkerClient{}
resp, err := handleGetBacktestRun(fake, &altv1.GetBacktestRunRequest{RunId: "run-1"})
if err != nil {
t.Fatalf("expected typed internal response, got error: %v", err)
}
requireBacktestError(t, resp.GetError(), backtestErrorInternal)
if fake.getRunReq == nil {
t.Error("expected request to be forwarded before nil response was detected")
}
}
func TestHandleGetBacktestRunDetailRequiresRunID(t *testing.T) {
fake := &fakeWorkerClient{}
@ -316,6 +363,21 @@ func TestBacktestSocketValidationReturnsTypedErrors(t *testing.T) {
return resp.GetError()
},
},
{
name: "get run missing run id",
run: func(t *testing.T) *altv1.ErrorInfo {
t.Helper()
resp, err := protoSocket.SendRequestTyped[*altv1.GetBacktestRunRequest, *altv1.GetBacktestRunResponse](
&client.Communicator,
&altv1.GetBacktestRunRequest{},
500*time.Millisecond,
)
if err != nil {
t.Fatalf("request should return typed error response without timeout: %v", err)
}
return resp.GetError()
},
},
{
name: "detail missing run id",
run: func(t *testing.T) *altv1.ErrorInfo {
@ -368,7 +430,7 @@ func TestBacktestSocketValidationReturnsTypedErrors(t *testing.T) {
requireBacktestError(t, tt.run(t), backtestErrorInvalidRequest)
})
}
if fake.startReq != nil || fake.detailReq != nil || fake.resultReq != nil || fake.compareReq != nil {
if fake.startReq != nil || fake.getRunReq != nil || fake.detailReq != nil || fake.resultReq != nil || fake.compareReq != nil {
t.Fatal("worker must not be called for socket-level validation failures")
}
}
@ -439,6 +501,7 @@ func TestBacktestHandlersRegisteredInSession(t *testing.T) {
}
required := []string{
protoSocket.TypeNameOf(&altv1.StartBacktestRequest{}),
protoSocket.TypeNameOf(&altv1.GetBacktestRunRequest{}),
protoSocket.TypeNameOf(&altv1.ListBacktestRunsRequest{}),
protoSocket.TypeNameOf(&altv1.GetBacktestRunDetailRequest{}),
protoSocket.TypeNameOf(&altv1.GetBacktestResultRequest{}),

View file

@ -31,6 +31,7 @@ type WorkerClient interface {
// Backtest command/query surface. The API forwards client requests onto the
// worker unchanged, so each method mirrors a contract request/response pair.
StartBacktest(ctx context.Context, req *altv1.StartBacktestRequest) (*altv1.StartBacktestResponse, error)
GetBacktestRun(ctx context.Context, req *altv1.GetBacktestRunRequest) (*altv1.GetBacktestRunResponse, error)
ListBacktestRuns(ctx context.Context, req *altv1.ListBacktestRunsRequest) (*altv1.ListBacktestRunsResponse, error)
GetBacktestRunDetail(ctx context.Context, req *altv1.GetBacktestRunDetailRequest) (*altv1.GetBacktestRunDetailResponse, error)
GetBacktestResult(ctx context.Context, req *altv1.GetBacktestResultRequest) (*altv1.GetBacktestResultResponse, error)
@ -132,6 +133,10 @@ func (c *socketClient) StartBacktest(ctx context.Context, req *altv1.StartBackte
return sendTyped[*altv1.StartBacktestRequest, *altv1.StartBacktestResponse](c, ctx, req)
}
func (c *socketClient) GetBacktestRun(ctx context.Context, req *altv1.GetBacktestRunRequest) (*altv1.GetBacktestRunResponse, error) {
return sendTyped[*altv1.GetBacktestRunRequest, *altv1.GetBacktestRunResponse](c, ctx, req)
}
func (c *socketClient) ListBacktestRuns(ctx context.Context, req *altv1.ListBacktestRunsRequest) (*altv1.ListBacktestRunsResponse, error) {
return sendTyped[*altv1.ListBacktestRunsRequest, *altv1.ListBacktestRunsResponse](c, ctx, req)
}

View file

@ -122,6 +122,47 @@ func TestWorkerClient_StartBacktest_Success(t *testing.T) {
}
}
func TestWorkerClient_GetBacktestRun_Success(t *testing.T) {
port, cleanup := startFakeWorker(t, func(client *protoSocket.WsClient) {
protoSocket.AddRequestListenerTyped[*altv1.GetBacktestRunRequest, *altv1.GetBacktestRunResponse](&client.Communicator, func(req *altv1.GetBacktestRunRequest) (*altv1.GetBacktestRunResponse, error) {
return &altv1.GetBacktestRunResponse{
Run: &altv1.BacktestRun{
Id: req.GetRunId(),
Status: altv1.BacktestRunStatus_BACKTEST_RUN_STATUS_SUCCEEDED,
},
}, nil
})
})
defer cleanup()
client := New(fmt.Sprintf("ws://127.0.0.1:%d/socket", port))
ctx := context.Background()
if err := client.Connect(ctx); err != nil {
t.Fatalf("failed to connect: %v", err)
}
defer client.Close()
res, err := client.GetBacktestRun(ctx, &altv1.GetBacktestRunRequest{RunId: "run-1"})
if err != nil {
t.Fatalf("GetBacktestRun failed: %v", err)
}
if res.GetRun().GetId() != "run-1" {
t.Errorf("run id did not round-trip, got %q", res.GetRun().GetId())
}
if res.GetRun().GetStatus() != altv1.BacktestRunStatus_BACKTEST_RUN_STATUS_SUCCEEDED {
t.Errorf("status did not round-trip, got %v", res.GetRun().GetStatus())
}
}
func TestWorkerClient_GetBacktestRun_Unavailable(t *testing.T) {
// A never-connected client must report ErrUnavailable instead of blocking.
client := New("ws://127.0.0.1:54321/socket")
_, err := client.GetBacktestRun(context.Background(), &altv1.GetBacktestRunRequest{RunId: "run-1"})
if !errors.Is(err, ErrUnavailable) {
t.Errorf("expected ErrUnavailable, got %v", err)
}
}
func TestWorkerClient_ListInstruments_Success(t *testing.T) {
port, cleanup := startFakeWorker(t, func(client *protoSocket.WsClient) {
protoSocket.AddRequestListenerTyped[*altv1.ListInstrumentsRequest, *altv1.ListInstrumentsResponse](&client.Communicator, func(req *altv1.ListInstrumentsRequest) (*altv1.ListInstrumentsResponse, error) {

View file

@ -8,9 +8,12 @@ import (
"os/signal"
"syscall"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"git.toki-labs.com/toki/alt/packages/domain/market"
workerbacktest "git.toki-labs.com/toki/alt/services/worker/internal/backtest"
"git.toki-labs.com/toki/alt/services/worker/internal/config"
"git.toki-labs.com/toki/alt/services/worker/internal/jobs"
"git.toki-labs.com/toki/alt/services/worker/internal/marketdata/importer"
@ -36,20 +39,7 @@ func main() {
slog.Error("failed to create database pool; store-backed socket surfaces disabled", "error", err)
} else {
defer pool.Close()
store := postgres.NewStore(pool)
kisProvider := kis.NewLiveProvider(kis.NewClient(kis.ConfigFromEnv()))
// One importer instance backs both the async job handler and the socket
// import command so the operator-facing command and scheduled jobs share
// the same provider/store wiring.
kisImporter := importer.New(kisProvider, store)
jobs.RegisterDailyBarImportHandler(runner, market.ProviderKIS, kisImporter)
deps = socket.Deps{
Analysis: store,
Results: store,
Instruments: store,
Bars: store,
DailyBarImporter: kisImporter,
}
deps = storeBackedDeps(runner, postgres.NewStore(pool))
}
server := socket.NewServer(cfg, deps)
@ -75,3 +65,34 @@ func main() {
os.Exit(1)
}
}
// storeBackedDeps wires every store-backed socket dependency around a single
// store, including the backtest execution rail. It is factored out of main so a
// unit test can assert the backtest start/read surfaces are wired without a live
// database. Backtest execution reads imported daily bars from storage and runs
// the bundled strategy resolver; the starter records the pending run and hands
// execution to the same runner the import jobs use, keeping a single async job
// rail rather than a second execution engine.
func storeBackedDeps(runner *jobs.Runner, store *postgres.Store) socket.Deps {
kisProvider := kis.NewLiveProvider(kis.NewClient(kis.ConfigFromEnv()))
// One importer instance backs both the async job handler and the socket
// import command so the operator-facing command and scheduled jobs share the
// same provider/store wiring.
kisImporter := importer.New(kisProvider, store)
jobs.RegisterDailyBarImportHandler(runner, market.ProviderKIS, kisImporter)
barSource := workerbacktest.NewStorageBarSource(store, store)
strategyPort := workerbacktest.NewBuiltInStrategyPort()
engine := workerbacktest.NewEngine(barSource, strategyPort, store)
jobs.RegisterRunBacktestHandler(runner, store, engine, time.Now)
starter := jobs.NewBacktestStarter(runner, store)
return socket.Deps{
Starter: starter,
Analysis: store,
Results: store,
Instruments: store,
Bars: store,
DailyBarImporter: kisImporter,
}
}

View file

@ -0,0 +1,37 @@
package main
import (
"testing"
"git.toki-labs.com/toki/alt/services/worker/internal/jobs"
"git.toki-labs.com/toki/alt/services/worker/internal/storage/postgres"
)
// TestStoreBackedDepsWiresBacktestSurfaces guards against a regression where the
// worker process serves backtest reads but leaves the start surface (Starter)
// unwired, which would advertise no backtest-start capability and reject every
// start command. The store is built with a nil pool because the wiring path only
// constructs adapters; it never touches the database at startup.
func TestStoreBackedDepsWiresBacktestSurfaces(t *testing.T) {
runner := jobs.NewRunner()
deps := storeBackedDeps(runner, postgres.NewStore(nil))
if deps.Starter == nil {
t.Error("expected backtest Starter to be wired")
}
if deps.Analysis == nil || deps.Results == nil {
t.Error("expected backtest read stores to be wired")
}
if deps.Instruments == nil || deps.Bars == nil {
t.Error("expected market stores to be wired")
}
if deps.DailyBarImporter == nil {
t.Error("expected daily bar importer to be wired")
}
// Wiring registers the import and run-backtest job handlers on the runner, so
// it must hold at least those two handlers after wiring.
if runner.Len() < 2 {
t.Errorf("expected wiring to register job handlers, runner holds %d", runner.Len())
}
}

View file

@ -0,0 +1,233 @@
package backtest_test
import (
"context"
"sync"
"testing"
"time"
domainbacktest "git.toki-labs.com/toki/alt/packages/domain/backtest"
"git.toki-labs.com/toki/alt/packages/domain/market"
workerbacktest "git.toki-labs.com/toki/alt/services/worker/internal/backtest"
"git.toki-labs.com/toki/alt/services/worker/internal/jobs"
"git.toki-labs.com/toki/alt/services/worker/internal/storage"
)
// flowStore is an in-memory store that satisfies the instrument, bar, run, and
// result ports the backtest command flow touches. It records every run status
// upsert so the test can assert the pending -> running -> succeeded transition.
type flowStore struct {
mu sync.Mutex
instruments map[market.InstrumentID]market.Instrument
bars map[market.InstrumentID][]market.Bar
runs map[domainbacktest.RunID]domainbacktest.Run
results map[domainbacktest.RunID]domainbacktest.Result
runStatusLog []domainbacktest.RunStatus
}
func newFlowStore() *flowStore {
return &flowStore{
instruments: make(map[market.InstrumentID]market.Instrument),
bars: make(map[market.InstrumentID][]market.Bar),
runs: make(map[domainbacktest.RunID]domainbacktest.Run),
results: make(map[domainbacktest.RunID]domainbacktest.Result),
}
}
func (s *flowStore) UpsertInstrument(ctx context.Context, inst market.Instrument) error {
s.mu.Lock()
defer s.mu.Unlock()
s.instruments[inst.ID] = inst
return nil
}
func (s *flowStore) GetInstrument(ctx context.Context, id market.InstrumentID) (market.Instrument, error) {
s.mu.Lock()
defer s.mu.Unlock()
inst, ok := s.instruments[id]
if !ok {
return market.Instrument{}, storage.ErrInstrumentNotFound
}
return inst, nil
}
func (s *flowStore) ListInstruments(ctx context.Context) ([]market.Instrument, error) {
s.mu.Lock()
defer s.mu.Unlock()
out := make([]market.Instrument, 0, len(s.instruments))
for _, inst := range s.instruments {
out = append(out, inst)
}
return out, nil
}
func (s *flowStore) UpsertBar(ctx context.Context, bar market.Bar) error {
s.mu.Lock()
defer s.mu.Unlock()
s.bars[bar.InstrumentID] = append(s.bars[bar.InstrumentID], bar)
return nil
}
func (s *flowStore) GetBars(ctx context.Context, id market.InstrumentID, timeframe market.Timeframe, from, to time.Time) ([]market.Bar, error) {
s.mu.Lock()
defer s.mu.Unlock()
var out []market.Bar
for _, bar := range s.bars[id] {
if bar.Timeframe != timeframe {
continue
}
if bar.Timestamp.Before(from) || bar.Timestamp.After(to) {
continue
}
out = append(out, bar)
}
return out, nil
}
func (s *flowStore) UpsertRun(ctx context.Context, run domainbacktest.Run) error {
s.mu.Lock()
defer s.mu.Unlock()
s.runs[run.ID] = run
s.runStatusLog = append(s.runStatusLog, run.Status)
return nil
}
func (s *flowStore) GetRun(ctx context.Context, id domainbacktest.RunID) (domainbacktest.Run, error) {
s.mu.Lock()
defer s.mu.Unlock()
run, ok := s.runs[id]
if !ok {
return domainbacktest.Run{}, storage.ErrRunNotFound
}
return run, nil
}
func (s *flowStore) UpsertResult(ctx context.Context, result domainbacktest.Result) error {
s.mu.Lock()
defer s.mu.Unlock()
s.results[result.RunID] = result
return nil
}
func (s *flowStore) GetResult(ctx context.Context, id domainbacktest.RunID) (domainbacktest.Result, error) {
s.mu.Lock()
defer s.mu.Unlock()
result, ok := s.results[id]
if !ok {
return domainbacktest.Result{}, storage.ErrResultNotFound
}
return result, nil
}
func (s *flowStore) statuses() []domainbacktest.RunStatus {
s.mu.Lock()
defer s.mu.Unlock()
out := make([]domainbacktest.RunStatus, len(s.runStatusLog))
copy(out, s.runStatusLog)
return out
}
// TestBacktestCommandFlowFromImportedBars proves the command-first rail end to
// end with storage-backed bars: a started run records a pending run, the runner
// executes it through the bundled strategy resolver against imported daily bars,
// the run reaches terminal succeeded, and the result carries summary fields the
// CLI result step reads. It runs the launch synchronously so no Docker or
// goroutine race is involved.
func TestBacktestCommandFlowFromImportedBars(t *testing.T) {
ctx := context.Background()
store := newFlowStore()
instID := market.InstrumentID("KRX:005930")
if err := store.UpsertInstrument(ctx, market.Instrument{ID: instID, Market: market.MarketKR}); err != nil {
t.Fatalf("upsert instrument: %v", err)
}
// Imported daily bars: a 3-day window the run spec covers.
for _, b := range []market.Bar{
{InstrumentID: instID, Timeframe: market.TimeframeDaily, Timestamp: time.Date(2026, 5, 1, 0, 0, 0, 0, time.UTC), Close: market.Price{Currency: market.CurrencyKRW, Amount: market.Decimal{Value: "1000"}}},
{InstrumentID: instID, Timeframe: market.TimeframeDaily, Timestamp: time.Date(2026, 5, 2, 0, 0, 0, 0, time.UTC), Close: market.Price{Currency: market.CurrencyKRW, Amount: market.Decimal{Value: "1100"}}},
{InstrumentID: instID, Timeframe: market.TimeframeDaily, Timestamp: time.Date(2026, 5, 3, 0, 0, 0, 0, time.UTC), Close: market.Price{Currency: market.CurrencyKRW, Amount: market.Decimal{Value: "1200"}}},
} {
if err := store.UpsertBar(ctx, b); err != nil {
t.Fatalf("upsert bar: %v", err)
}
}
engine := workerbacktest.NewEngine(
workerbacktest.NewStorageBarSource(store, store),
workerbacktest.NewBuiltInStrategyPort(),
store,
)
runner := jobs.NewRunner()
jobs.RegisterRunBacktestHandler(runner, store, engine, time.Now)
starter := jobs.NewBacktestStarter(runner, store,
jobs.WithStarterIDGenerator(func() string { return "run-flow" }),
jobs.WithStarterLaunch(func(fn func()) { fn() }), // synchronous for determinism
)
spec := domainbacktest.RunSpec{
StrategyID: workerbacktest.BuiltInStrategyID,
Market: market.MarketKR,
Timeframe: market.TimeframeDaily,
From: time.Date(2026, 5, 1, 0, 0, 0, 0, time.UTC),
To: time.Date(2026, 5, 3, 0, 0, 0, 0, time.UTC),
}
run, err := starter.StartBacktest(ctx, spec)
if err != nil {
t.Fatalf("start backtest: %v", err)
}
if run.ID != "run-flow" {
t.Fatalf("unexpected run id %q", run.ID)
}
// Poll: the run reached terminal succeeded after synchronous execution.
got, err := store.GetRun(ctx, run.ID)
if err != nil {
t.Fatalf("get run: %v", err)
}
if got.Status != domainbacktest.RunStatusSucceeded {
t.Fatalf("expected terminal status succeeded, got %q (statuses=%v)", got.Status, store.statuses())
}
// The pending -> running -> succeeded transition is recorded.
statuses := store.statuses()
wantTransition := []domainbacktest.RunStatus{
domainbacktest.RunStatusPending,
domainbacktest.RunStatusRunning,
domainbacktest.RunStatusSucceeded,
}
for _, want := range wantTransition {
if !containsStatus(statuses, want) {
t.Errorf("expected status %q in transition log %v", want, statuses)
}
}
// Result: summary fields the CLI result step reads are present and reflect the
// buy-and-hold run over the imported bars.
result, err := store.GetResult(ctx, run.ID)
if err != nil {
t.Fatalf("get result: %v", err)
}
if result.RunID != run.ID {
t.Errorf("result run id = %q, want %q", result.RunID, run.ID)
}
if result.Summary.StartingCash.Amount.Value != "10000000" {
t.Errorf("summary starting cash = %q, want 10000000", result.Summary.StartingCash.Amount.Value)
}
// Buy 1 unit at 1000 on day 1, hold; ending equity = 9,999,000 cash + 1*1200.
if result.Summary.EndingEquity.Amount.Value != "10000200" {
t.Errorf("summary ending equity = %q, want 10000200", result.Summary.EndingEquity.Amount.Value)
}
if result.Summary.TradeCount != 1 {
t.Errorf("summary trade count = %d, want 1", result.Summary.TradeCount)
}
}
func containsStatus(haystack []domainbacktest.RunStatus, needle domainbacktest.RunStatus) bool {
for _, s := range haystack {
if s == needle {
return true
}
}
return false
}

View file

@ -0,0 +1,58 @@
package backtest
import (
"context"
"fmt"
"git.toki-labs.com/toki/alt/packages/domain/backtest"
"git.toki-labs.com/toki/alt/packages/domain/market"
)
// BuiltInStrategyID is the identifier of the worker's default bundled strategy.
// Command workflows and YAML scenarios reference it so a backtest can run
// end-to-end on imported bars without an operator authoring a custom strategy.
const BuiltInStrategyID backtest.StrategyID = "strategy-v1"
// BuiltInStrategyPort resolves the worker's bundled strategies. It is the
// production StrategyPort wired in cmd/alt-worker. Unknown ids return an error
// so a misconfigured run fails fast instead of silently producing an empty
// result.
type BuiltInStrategyPort struct{}
func NewBuiltInStrategyPort() *BuiltInStrategyPort {
return &BuiltInStrategyPort{}
}
func (p *BuiltInStrategyPort) GetStrategy(ctx context.Context, id backtest.StrategyID) (backtest.Strategy, error) {
switch id {
case BuiltInStrategyID:
return buyAndHoldStrategy{id: id}, nil
default:
return nil, fmt.Errorf("unknown strategy %q", id)
}
}
// buyAndHoldStrategy buys a single unit of each instrument the first time it is
// seen and then holds. It is intentionally minimal and deterministic so command
// validation can assert a stable terminal result from imported daily bars.
type buyAndHoldStrategy struct {
id backtest.StrategyID
}
func (s buyAndHoldStrategy) ID() backtest.StrategyID { return s.id }
func (s buyAndHoldStrategy) Decide(input backtest.StrategyInput) ([]backtest.OrderIntent, error) {
// Hold once a position exists. The engine passes the pre-trade portfolio for
// the current bar, so an existing position means this instrument was already
// bought on an earlier bar.
if _, held := input.Portfolio.Position(input.Bar.InstrumentID); held {
return nil, nil
}
return []backtest.OrderIntent{
{
InstrumentID: input.Bar.InstrumentID,
Side: backtest.OrderSideBuy,
Quantity: market.Quantity{Amount: market.Decimal{Value: "1"}},
},
}, nil
}

View file

@ -0,0 +1,76 @@
package backtest
import (
"context"
"testing"
"git.toki-labs.com/toki/alt/packages/domain/backtest"
"git.toki-labs.com/toki/alt/packages/domain/market"
)
func TestBuiltInStrategyPortResolvesKnownStrategy(t *testing.T) {
port := NewBuiltInStrategyPort()
strat, err := port.GetStrategy(context.Background(), BuiltInStrategyID)
if err != nil {
t.Fatalf("unexpected error resolving built-in strategy: %v", err)
}
if strat.ID() != BuiltInStrategyID {
t.Errorf("strategy id = %q, want %q", strat.ID(), BuiltInStrategyID)
}
}
func TestBuiltInStrategyPortRejectsUnknownStrategy(t *testing.T) {
port := NewBuiltInStrategyPort()
if _, err := port.GetStrategy(context.Background(), "does-not-exist"); err == nil {
t.Fatal("expected error for unknown strategy id, got nil")
}
}
func TestBuyAndHoldBuysOnceThenHolds(t *testing.T) {
port := NewBuiltInStrategyPort()
strat, err := port.GetStrategy(context.Background(), BuiltInStrategyID)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
instID := market.InstrumentID("KRX:005930")
cash := market.Price{Currency: market.CurrencyKRW, Amount: market.Decimal{Value: "1000000"}}
bar := market.Bar{
InstrumentID: instID,
Timeframe: market.TimeframeDaily,
Close: market.Price{Currency: market.CurrencyKRW, Amount: market.Decimal{Value: "1000"}},
}
// No position yet: the strategy buys a single unit.
orders, err := strat.Decide(backtest.StrategyInput{Bar: bar, Portfolio: backtest.NewPortfolioState(cash)})
if err != nil {
t.Fatalf("unexpected error on first decide: %v", err)
}
if len(orders) != 1 {
t.Fatalf("expected 1 buy order on first bar, got %d", len(orders))
}
if orders[0].Side != backtest.OrderSideBuy || orders[0].InstrumentID != instID {
t.Errorf("unexpected order %+v", orders[0])
}
if orders[0].Quantity.Amount.Value != "1" {
t.Errorf("expected quantity 1, got %s", orders[0].Quantity.Amount.Value)
}
// Apply the buy and decide again: with a position held, the strategy holds.
portfolio, err := backtest.NewPortfolioState(cash).ApplyFill(backtest.Fill{
InstrumentID: instID,
Side: backtest.OrderSideBuy,
Quantity: market.Quantity{Amount: market.Decimal{Value: "1"}},
Price: bar.Close,
})
if err != nil {
t.Fatalf("failed to apply fill: %v", err)
}
held, err := strat.Decide(backtest.StrategyInput{Bar: bar, Portfolio: portfolio})
if err != nil {
t.Fatalf("unexpected error on hold decide: %v", err)
}
if len(held) != 0 {
t.Errorf("expected no orders while holding, got %d", len(held))
}
}

View file

@ -71,6 +71,17 @@ func backtestHandlers(deps Deps) []sessionHandler {
)
},
},
{
requestType: protoSocket.TypeNameOf(&altv1.GetBacktestRunRequest{}),
register: func(client *protoSocket.WsClient) {
protoSocket.AddRequestListenerTyped[*altv1.GetBacktestRunRequest, *altv1.GetBacktestRunResponse](
&client.Communicator,
func(req *altv1.GetBacktestRunRequest) (*altv1.GetBacktestRunResponse, error) {
return handleGetBacktestRun(deps, req)
},
)
},
},
{
requestType: protoSocket.TypeNameOf(&altv1.ListBacktestRunsRequest{}),
register: func(client *protoSocket.WsClient) {
@ -137,6 +148,27 @@ func handleStartBacktest(deps Deps, req *altv1.StartBacktestRequest) (*altv1.Sta
return &altv1.StartBacktestResponse{Run: runToProto(run)}, nil
}
// handleGetBacktestRun answers a single-run poll. It reads through the analysis
// store's run detail so polling reuses the same store-backed read path as
// list/detail instead of growing a second run-store dependency on Deps.
func handleGetBacktestRun(deps Deps, req *altv1.GetBacktestRunRequest) (*altv1.GetBacktestRunResponse, error) {
if req.GetRunId() == "" {
return &altv1.GetBacktestRunResponse{Error: invalidRequest("run_id is required")}, nil
}
if deps.Analysis == nil {
return &altv1.GetBacktestRunResponse{Error: unavailableError("backtest analysis is not available")}, nil
}
ctx, cancel := context.WithTimeout(context.Background(), handlerTimeout)
defer cancel()
detail, err := deps.Analysis.GetRunDetail(ctx, backtest.RunID(req.GetRunId()))
if err != nil {
return &altv1.GetBacktestRunResponse{Error: backendErrorInfo(err)}, nil
}
return &altv1.GetBacktestRunResponse{Run: runToProto(detail.Run)}, nil
}
func handleListBacktestRuns(deps Deps, req *altv1.ListBacktestRunsRequest) (*altv1.ListBacktestRunsResponse, error) {
if deps.Analysis == nil {
return &altv1.ListBacktestRunsResponse{Error: unavailableError("backtest analysis is not available")}, nil

View file

@ -155,6 +155,52 @@ func TestHandleListBacktestRunsEmpty(t *testing.T) {
}
}
func TestHandleGetBacktestRunSuccess(t *testing.T) {
store := &fakeAnalysisStore{detail: storage.RunDetail{
Run: backtest.Run{ID: "run-1", Status: backtest.RunStatusSucceeded},
}}
deps := BacktestDeps{Analysis: store}
resp, err := handleGetBacktestRun(deps, &altv1.GetBacktestRunRequest{RunId: "run-1"})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if resp.GetRun().GetId() != "run-1" {
t.Errorf("run id mismatch: %q", resp.GetRun().GetId())
}
if resp.GetRun().GetStatus() != altv1.BacktestRunStatus_BACKTEST_RUN_STATUS_SUCCEEDED {
t.Errorf("expected succeeded status, got %v", resp.GetRun().GetStatus())
}
}
func TestHandleGetBacktestRunNotFound(t *testing.T) {
store := &fakeAnalysisStore{err: storage.ErrRunNotFound}
deps := BacktestDeps{Analysis: store}
resp, err := handleGetBacktestRun(deps, &altv1.GetBacktestRunRequest{RunId: "missing"})
if err != nil {
t.Fatalf("expected typed not_found response, got error: %v", err)
}
requireBacktestError(t, resp.GetError(), backtestErrorNotFound)
}
func TestHandleGetBacktestRunRequiresRunID(t *testing.T) {
deps := BacktestDeps{Analysis: &fakeAnalysisStore{}}
resp, err := handleGetBacktestRun(deps, &altv1.GetBacktestRunRequest{})
if err != nil {
t.Fatalf("expected typed validation response, got error: %v", err)
}
requireBacktestError(t, resp.GetError(), backtestErrorInvalidRequest)
}
func TestHandleGetBacktestRunUnavailable(t *testing.T) {
resp, err := handleGetBacktestRun(BacktestDeps{}, &altv1.GetBacktestRunRequest{RunId: "run-1"})
if err != nil {
t.Fatalf("expected typed unavailable response, got error: %v", err)
}
requireBacktestError(t, resp.GetError(), backtestErrorUnavailable)
}
func TestHandleGetBacktestRunDetailWithResult(t *testing.T) {
store := &fakeAnalysisStore{detail: storage.RunDetail{
Run: backtest.Run{ID: "run-1", Status: backtest.RunStatusSucceeded},
@ -288,6 +334,7 @@ func TestBacktestHandlersCoverAllRequests(t *testing.T) {
required := []string{
"alt.v1.StartBacktestRequest",
"alt.v1.GetBacktestRunRequest",
"alt.v1.ListBacktestRunsRequest",
"alt.v1.GetBacktestRunDetailRequest",
"alt.v1.GetBacktestResultRequest",