update operator workflow tests and backtest runtime
This commit is contained in:
parent
473934212c
commit
96e7c85c6f
31 changed files with 1325 additions and 209 deletions
|
|
@ -34,44 +34,49 @@ task=m-command-first-operator-workflow/03+02_market_status_scenario, plan=0, tag
|
|||
|
||||
| 항목 | 완료 여부 |
|
||||
|------|---------|
|
||||
| [CLI-1] Import Before Status Reads | [ ] |
|
||||
| [CLI-2] Status Evidence Tests | [ ] |
|
||||
| [CLI-3] Handoff Discoverability | [ ] |
|
||||
| [CLI-4] Verification | [ ] |
|
||||
| [CLI-1] Import Before Status Reads | [x] |
|
||||
| [CLI-2] Status Evidence Tests | [x] |
|
||||
| [CLI-3] Handoff Discoverability | [x] |
|
||||
| [CLI-4] Verification | [x] |
|
||||
|
||||
## 구현 체크리스트
|
||||
|
||||
- [ ] [CLI-1] Update `market_data_status_query.yaml` to run import before instrument/bar status reads.
|
||||
- [ ] [CLI-2] Update expected JSONL and tests to require import counts, instrument count, bar count, and typed unavailable/error keys. 검증: `alt operator scenario validate/run` 또는 동등한 command가 instrument/bar count와 typed unavailable/error key를 출력한다.
|
||||
- [ ] [CLI-3] Update handoff matrix and CLI dev hints so the command-first status scenario is discoverable.
|
||||
- [ ] [CLI-4] Run CLI verification and `git diff --check`.
|
||||
- [ ] CODE_REVIEW-*-G??.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채운다. 이 항목이 완료되기 전에는 구현이 완료된 것이 아니다.
|
||||
- [x] [CLI-1] Update `market_data_status_query.yaml` to run import before instrument/bar status reads.
|
||||
- [x] [CLI-2] Update expected JSONL and tests to require import counts, instrument count, bar count, and typed unavailable/error keys. 검증: `alt operator scenario validate/run` 또는 동등한 command가 instrument/bar count와 typed unavailable/error key를 출력한다.
|
||||
- [x] [CLI-3] Update handoff matrix and CLI dev hints so the command-first status scenario is discoverable.
|
||||
- [x] [CLI-4] Run CLI verification and `git diff --check`.
|
||||
- [x] CODE_REVIEW-*-G??.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채운다. 이 항목이 완료되기 전에는 구현이 완료된 것이 아니다.
|
||||
|
||||
## 코드리뷰 전용 체크리스트
|
||||
|
||||
> **[REVIEW AGENT ONLY]** 이 체크리스트는 코드리뷰 에이전트만 사용한다.
|
||||
> 구현 에이전트는 이 섹션을 수정하거나 체크하지 않는다.
|
||||
|
||||
- [ ] `코드리뷰 결과`에 `PASS`, `WARN`, `FAIL` 중 하나의 판정을 append한다.
|
||||
- [ ] 판정과 `차원별 평가`, Required/Suggested/Nit 분류가 서로 일치한다.
|
||||
- [ ] active `CODE_REVIEW-*-G??.md`를 `code_review_cloud_G07_N.log`로 아카이브한다.
|
||||
- [ ] active `PLAN-*-G??.md`를 `plan_cloud_G07_M.log`로 아카이브한다.
|
||||
- [ ] `.gitignore`의 Agent-Ops 관리 block이 `agent-task/**/*.md`와 `agent-task/**/*.log`를 unignore하여 plan/review/archive 산출물이 추적 가능한지 확인한다.
|
||||
- [ ] PASS이면 `agent-ops/skills/common/code-review/templates/complete-log-template.md` 기준으로 `complete.log`를 작성하고 active `.md` 파일을 남기지 않는다.
|
||||
- [ ] PASS이면 active task 디렉터리 `agent-task/m-command-first-operator-workflow/03+02_market_status_scenario/`를 `agent-task/archive/YYYY/MM/m-command-first-operator-workflow/03+02_market_status_scenario/`로 이동하고 최종 archive 경로에서 이 체크리스트를 갱신한다.
|
||||
- [ ] PASS이고 task group이 `m-<milestone-slug>`이면 런타임이 읽을 완료 이벤트 메타데이터를 보고하고, roadmap 수정이나 `update-roadmap` 직접 호출을 하지 않는다.
|
||||
- [ ] PASS split 작업이면 이동 후 빈 active parent `agent-task/m-command-first-operator-workflow/`를 제거하거나, 남은 sibling/file이 있어 유지했다고 확인한다.
|
||||
- [x] `코드리뷰 결과`에 `PASS`, `WARN`, `FAIL` 중 하나의 판정을 append한다.
|
||||
- [x] 판정과 `차원별 평가`, Required/Suggested/Nit 분류가 서로 일치한다.
|
||||
- [x] active `CODE_REVIEW-*-G??.md`를 `code_review_cloud_G07_N.log`로 아카이브한다.
|
||||
- [x] active `PLAN-*-G??.md`를 `plan_cloud_G07_M.log`로 아카이브한다.
|
||||
- [x] `.gitignore`의 Agent-Ops 관리 block이 `agent-task/**/*.md`와 `agent-task/**/*.log`를 unignore하여 plan/review/archive 산출물이 추적 가능한지 확인한다.
|
||||
- [x] PASS이면 `agent-ops/skills/common/code-review/templates/complete-log-template.md` 기준으로 `complete.log`를 작성하고 active `.md` 파일을 남기지 않는다.
|
||||
- [x] PASS이면 active task 디렉터리 `agent-task/m-command-first-operator-workflow/03+02_market_status_scenario/`를 `agent-task/archive/YYYY/MM/m-command-first-operator-workflow/03+02_market_status_scenario/`로 이동하고 최종 archive 경로에서 이 체크리스트를 갱신한다.
|
||||
- [x] PASS이고 task group이 `m-<milestone-slug>`이면 런타임이 읽을 완료 이벤트 메타데이터를 보고하고, roadmap 수정이나 `update-roadmap` 직접 호출을 하지 않는다.
|
||||
- [x] PASS split 작업이면 이동 후 빈 active parent `agent-task/m-command-first-operator-workflow/`를 제거하거나, 남은 sibling/file이 있어 유지했다고 확인한다.
|
||||
- [ ] WARN/FAIL이고 user-review gate가 트리거되지 않았으면 다음 active `PLAN-cloud-G07.md`와 `CODE_REVIEW-cloud-G07.md`를 작성하고 `complete.log`를 작성하지 않는다.
|
||||
- [ ] USER_REVIEW이면 `agent-ops/skills/common/code-review/templates/user-review-template.md` 기준으로 `USER_REVIEW.md`를 작성하고 active `PLAN-*.md`, `CODE_REVIEW-*.md`, `complete.log`를 남기지 않는다.
|
||||
- [ ] USER_REVIEW가 사용자 결정으로 완료/PASS 해소되면 `USER_REVIEW.md`를 해소 상태로 갱신하고 `complete.log`를 작성한 뒤 task directory를 archive로 이동한다.
|
||||
|
||||
## 계획 대비 변경 사항
|
||||
|
||||
_구현 에이전트가 계획과 다르게 구현한 부분을 이유와 함께 기록한다._
|
||||
- import step id를 계획의 `import_kr_daily_bars` 그대로 사용. (계획 본문 일부에서 `import_kr_daily_bars`/`list_kr_instruments` 표기와 일치)
|
||||
- `client_test.go`의 변경은 계획이 적은 "expose last import request"가 아니라 호출 순서 기록(`recordCall`/`callOrder`) 추가로 대체. 사유: import-before-read 증거로는 마지막 요청 캡처(`lastImportDailyBarsReq`는 이미 존재)보다 import→list_instruments→list_bars 호출 순서 단언이 더 직접적이다. 마지막 import request 캡처는 KIS window 단언으로 함께 사용.
|
||||
- handoff_test.go에 추가한 테스트 이름은 `TestStatusScenarioFixtureImportsBeforeReads`로, 고정 검증 명령의 `-run ...|Handoff` 필터에는 잡히지 않지만 전체 테스트(`go test ./apps/cli/...`)에서 실행되어 PASS한다. 계획의 중간 검증 명령(고정 계약)은 그대로 유지했다.
|
||||
- `bin/dev`는 read-only 문구가 없어 (CLI-3 계획의 "update text only if it says read-only") 변경하지 않음.
|
||||
|
||||
## 주요 설계 결정
|
||||
|
||||
_구현 에이전트가 주요 설계 결정 사항을 기록한다._
|
||||
- expected JSONL은 기존 `kis_daily_import_smoke.jsonl`과 동일하게 illustrative seed 값(`instrument_count=1`, `bar_count=42`)을 사용. handoff 문서가 명시하듯 line별 `scenario`/`step`/`action`/`status`와 protobuf field 존재 여부가 계약이고 count 값은 seed 의존이다.
|
||||
- typed unavailable 검증은 import step이 `ErrorInfo{Code:"unavailable"}`를 돌려줄 때 `expect.status: error`/`error_code: unavailable`로 step이 통과(exit 0)하며 출력에 `error.code=unavailable`이 남는 것을 단언. transport_error로 떨어지지 않음을 증거로 남긴다.
|
||||
- 호출 순서 기록은 fakeAPI에 mutex 보호 `calls` slice로 최소 침습 추가했고, 기존 핸들러 동작은 유지.
|
||||
|
||||
## 사용자 리뷰 요청
|
||||
|
||||
|
|
@ -106,33 +111,52 @@ _구현 에이전트가 각 중간 검증 및 최종 검증 명령 실행 후
|
|||
### CLI-1 중간 검증
|
||||
```bash
|
||||
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/market_data_status_query.yaml
|
||||
(output)
|
||||
scenario=market_data_status_query status=valid steps=4
|
||||
```
|
||||
|
||||
### CLI-2 중간 검증
|
||||
```bash
|
||||
$ go test ./apps/cli/internal/operator -run 'MarketDataStatus|MarketStatusTypedUnavailable|Handoff' -count=1
|
||||
(output)
|
||||
$ go test ./apps/cli/internal/operator -run 'MarketDataStatus|MarketStatusTypedUnavailable|Handoff' -count=1 -v
|
||||
=== RUN TestHeadlessValidationHandoffCoversRequiredScenarios
|
||||
--- PASS: TestHeadlessValidationHandoffCoversRequiredScenarios (0.00s)
|
||||
=== RUN TestHandoffDocumentedScenarioFixturesExist
|
||||
--- PASS: TestHandoffDocumentedScenarioFixturesExist (0.00s)
|
||||
=== RUN TestRunScenarioMarketDataStatus
|
||||
--- PASS: TestRunScenarioMarketDataStatus (0.00s)
|
||||
=== RUN TestRunScenarioMarketDataStatusImportsBeforeReads
|
||||
--- PASS: TestRunScenarioMarketDataStatusImportsBeforeReads (0.00s)
|
||||
=== RUN TestRunScenarioMarketStatusTypedUnavailable
|
||||
--- PASS: TestRunScenarioMarketStatusTypedUnavailable (0.00s)
|
||||
PASS
|
||||
ok git.toki-labs.com/toki/alt/apps/cli/internal/operator 0.016s
|
||||
```
|
||||
|
||||
참고: 새 handoff 테스트 `TestStatusScenarioFixtureImportsBeforeReads`는 위 고정 필터에는 잡히지 않으나 최종 검증의 `go test ./apps/cli/...`에서 실행되어 PASS한다.
|
||||
|
||||
### CLI-3 중간 검증
|
||||
```bash
|
||||
$ go test ./apps/cli/internal/operator -run Handoff -count=1
|
||||
(output)
|
||||
ok git.toki-labs.com/toki/alt/apps/cli/internal/operator 0.042s
|
||||
```
|
||||
|
||||
### CLI-4 중간 검증
|
||||
```bash
|
||||
$ git diff --check
|
||||
(output)
|
||||
(no output — clean)
|
||||
```
|
||||
|
||||
### 최종 검증
|
||||
```bash
|
||||
$ go test ./apps/cli/... -count=1
|
||||
? git.toki-labs.com/toki/alt/apps/cli/cmd/alt [no test files]
|
||||
ok git.toki-labs.com/toki/alt/apps/cli/internal/cli 0.013s
|
||||
ok git.toki-labs.com/toki/alt/apps/cli/internal/operator 0.182s
|
||||
|
||||
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/market_data_status_query.yaml
|
||||
scenario=market_data_status_query status=valid steps=4
|
||||
|
||||
$ git diff --check
|
||||
(output)
|
||||
(no output — clean)
|
||||
```
|
||||
|
||||
---
|
||||
|
|
@ -140,3 +164,17 @@ $ git diff --check
|
|||
> **[IMPLEMENTING AGENT — BEFORE SAVING] Have you filled in every implementation-owned section: completion table, implementation checklist, changes from plan, design decisions, and verification output?**
|
||||
> If anything is blank, go back and fill it in before saving this file.
|
||||
> Leave review-agent-only sections unchanged.
|
||||
|
||||
## 코드리뷰 결과
|
||||
|
||||
- 종합 판정: PASS
|
||||
- 차원별 평가:
|
||||
- correctness: Pass
|
||||
- completeness: Pass
|
||||
- test coverage: Pass
|
||||
- API contract: Pass
|
||||
- code quality: Pass
|
||||
- plan deviation: Pass
|
||||
- verification trust: Pass
|
||||
- 발견된 문제: 없음
|
||||
- 다음 단계: PASS 절차로 `complete.log`를 작성하고 task directory를 archive로 이동한다.
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
# Complete - m-command-first-operator-workflow/03+02_market_status_scenario
|
||||
|
||||
## 완료 일시
|
||||
|
||||
2026-06-03
|
||||
|
||||
## 요약
|
||||
|
||||
Market status scenario import-before-read workflow completed in 1 review loop; final verdict PASS.
|
||||
|
||||
## 루프 이력
|
||||
|
||||
| Plan | Review | Verdict | 메모 |
|
||||
|------|--------|---------|------|
|
||||
| `plan_cloud_G07_0.log` | `code_review_cloud_G07_0.log` | PASS | `market_data_status_query` now imports daily bars before status reads, with fixture/docs/tests/verification updated. |
|
||||
|
||||
## 구현/정리 내용
|
||||
|
||||
- Added `import_daily_bars` before instrument/bar status reads in `market_data_status_query.yaml`.
|
||||
- Updated expected JSONL, handoff evidence, and runner tests to cover import counts, read counts, call order, and typed `unavailable` output.
|
||||
- Added fake API call-order capture for command-first scenario assertions.
|
||||
|
||||
## 최종 검증
|
||||
|
||||
- `go test ./apps/cli/internal/operator -run 'MarketDataStatus|MarketStatusTypedUnavailable|Handoff' -count=1 -v` - PASS; focused market status, typed unavailable, and handoff tests passed.
|
||||
- `go test ./apps/cli/... -count=1` - PASS; `cmd/alt` had no test files, `internal/cli` and `internal/operator` passed.
|
||||
- `go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/market_data_status_query.yaml` - PASS; `scenario=market_data_status_query status=valid steps=4`.
|
||||
- `git diff --check` - PASS; no output.
|
||||
|
||||
## Roadmap Completion
|
||||
|
||||
- Milestone: `agent-roadmap/phase/operator-surface/milestones/command-first-operator-workflow.md`
|
||||
- Completed task ids:
|
||||
- `market-status-scenario`: PASS; evidence=`agent-task/archive/2026/06/m-command-first-operator-workflow/03+02_market_status_scenario/plan_cloud_G07_0.log`, `agent-task/archive/2026/06/m-command-first-operator-workflow/03+02_market_status_scenario/code_review_cloud_G07_0.log`; verification=`go test ./apps/cli/... -count=1`, `go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/market_data_status_query.yaml`, `git diff --check`
|
||||
- Not completed task ids: 없음
|
||||
|
||||
## 잔여 Nit
|
||||
|
||||
- 없음
|
||||
|
||||
## 후속 작업
|
||||
|
||||
- 없음
|
||||
|
|
@ -0,0 +1,217 @@
|
|||
<!-- task=m-command-first-operator-workflow/04+02_backtest_runtime_flow plan=0 tag=API -->
|
||||
|
||||
# Code Review Reference - API
|
||||
|
||||
> **[IMPLEMENTING AGENT — READ FIRST] Filling in this file is the mandatory final step of implementation.**
|
||||
> The task is NOT complete until every implementation-owned section below is filled in.
|
||||
> Complete the `구현 체크리스트`; the final checklist item is mandatory before saving.
|
||||
> Fill implementation-owned sections, then stop with active files in place and report ready for review.
|
||||
> If implementation is blocked by a user-only decision, user-owned external environment prerequisite, or scope conflict, fill `사용자 리뷰 요청` with evidence and stop with active files in place; code-review decides whether to write `USER_REVIEW.md`. Evidence gaps that a follow-up agent can close by rerunning commands or collecting artifacts are normal follow-up issues, not user-review blockers by themselves.
|
||||
> Finalization (`코드리뷰 결과`, log rename, `complete.log`, archive moves, `코드리뷰 전용 체크리스트`) is review-agent-only, even after compaction/resume.
|
||||
> Follow the ownership table at the bottom of this file for which sections you own.
|
||||
|
||||
## 개요
|
||||
|
||||
date=2026-06-03
|
||||
task=m-command-first-operator-workflow/04+02_backtest_runtime_flow, plan=0, tag=API
|
||||
|
||||
## Roadmap Targets
|
||||
|
||||
- Milestone: `agent-roadmap/phase/operator-surface/milestones/command-first-operator-workflow.md`
|
||||
- Task ids:
|
||||
- `backtest-command-flow`: imported daily bars를 입력으로 backtest start/poll/result를 command workflow에서 확인할 수 있다.
|
||||
- Completion mode: check-on-pass
|
||||
|
||||
## 이 파일을 읽는 리뷰 에이전트에게
|
||||
|
||||
> **[REVIEW AGENT ONLY]** 아래 종결 절차는 코드리뷰 에이전트 전용이다. 구현 에이전트는 이 섹션을 실행하지 않는다.
|
||||
|
||||
각 항목의 구현을 실제 소스 파일과 대조하고, `검증 결과` 섹션의 출력이 코드와 일치하는지 확인하세요.
|
||||
|
||||
---
|
||||
|
||||
## 구현 항목별 완료 여부
|
||||
|
||||
| 항목 | 완료 여부 |
|
||||
|------|---------|
|
||||
| [API-1] GetBacktestRun Rail | [x] |
|
||||
| [API-2] Worker Runtime Wiring | [x] |
|
||||
| [API-3] Execution Evidence Tests | [x] |
|
||||
| [API-4] Backtest YAML Flow | [x] |
|
||||
| [API-5] Verification | [x] |
|
||||
|
||||
## 구현 체크리스트
|
||||
|
||||
- [x] [API-1] Add API and worker `GetBacktestRun` forwarding/handler support for polling.
|
||||
- [x] [API-2] Wire worker `alt-worker` with `BacktestStarter`, `RegisterRunBacktestHandler`, `StorageBarSource`, result store, and a built-in strategy resolver.
|
||||
- [x] [API-3] Add/update tests for built-in strategy resolver, worker execution wiring, and typed run polling.
|
||||
- [x] [API-4] Update backtest YAML scenarios to import daily bars before start/poll/result. 검증: start/poll/result scenario가 run id, terminal status, result summary key를 출력한다.
|
||||
- [x] [API-5] Run API/worker/CLI verification and `git diff --check`.
|
||||
- [x] CODE_REVIEW-*-G??.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채운다. 이 항목이 완료되기 전에는 구현이 완료된 것이 아니다.
|
||||
|
||||
## 코드리뷰 전용 체크리스트
|
||||
|
||||
> **[REVIEW AGENT ONLY]** 이 체크리스트는 코드리뷰 에이전트만 사용한다.
|
||||
> 구현 에이전트는 이 섹션을 수정하거나 체크하지 않는다.
|
||||
|
||||
- [x] `코드리뷰 결과`에 `PASS`, `WARN`, `FAIL` 중 하나의 판정을 append한다.
|
||||
- [x] 판정과 `차원별 평가`, Required/Suggested/Nit 분류가 서로 일치한다.
|
||||
- [x] active `CODE_REVIEW-*-G??.md`를 `code_review_cloud_G08_N.log`로 아카이브한다.
|
||||
- [x] active `PLAN-*-G??.md`를 `plan_cloud_G08_M.log`로 아카이브한다.
|
||||
- [x] `.gitignore`의 Agent-Ops 관리 block이 `agent-task/**/*.md`와 `agent-task/**/*.log`를 unignore하여 plan/review/archive 산출물이 추적 가능한지 확인한다.
|
||||
- [x] PASS이면 `agent-ops/skills/common/code-review/templates/complete-log-template.md` 기준으로 `complete.log`를 작성하고 active `.md` 파일을 남기지 않는다.
|
||||
- [x] PASS이면 active task 디렉터리 `agent-task/m-command-first-operator-workflow/04+02_backtest_runtime_flow/`를 `agent-task/archive/YYYY/MM/m-command-first-operator-workflow/04+02_backtest_runtime_flow/`로 이동하고 최종 archive 경로에서 이 체크리스트를 갱신한다.
|
||||
- [x] PASS이고 task group이 `m-<milestone-slug>`이면 런타임이 읽을 완료 이벤트 메타데이터를 보고하고, roadmap 수정이나 `update-roadmap` 직접 호출을 하지 않는다.
|
||||
- [x] PASS split 작업이면 이동 후 빈 active parent `agent-task/m-command-first-operator-workflow/`를 제거하거나, 남은 sibling/file이 있어 유지했다고 확인한다.
|
||||
- [ ] WARN/FAIL이고 user-review gate가 트리거되지 않았으면 다음 active `PLAN-cloud-G08.md`와 `CODE_REVIEW-cloud-G08.md`를 작성하고 `complete.log`를 작성하지 않는다.
|
||||
- [ ] USER_REVIEW이면 `agent-ops/skills/common/code-review/templates/user-review-template.md` 기준으로 `USER_REVIEW.md`를 작성하고 active `PLAN-*.md`, `CODE_REVIEW-*.md`, `complete.log`를 남기지 않는다.
|
||||
- [ ] USER_REVIEW가 사용자 결정으로 완료/PASS 해소되면 `USER_REVIEW.md`를 해소 상태로 갱신하고 `complete.log`를 작성한 뒤 task directory를 archive로 이동한다.
|
||||
|
||||
## 계획 대비 변경 사항
|
||||
|
||||
- [API-1] 계획은 `services/api/internal/socket/server_test.go`의 capability sync 가능성을 열어 두었으나, API capability set은 핸들러 등록과 무관하게 고정 목록(`capabilitiesForSession`)이고 `GetBacktestRun`은 기존 `backtest-read`/`backtest-start` capability 묶음 안에 들어가므로 capability 변경이 불필요했다. 따라서 `server_test.go`는 수정하지 않았고, 대신 `backtest_test.go`의 등록 커버리지(`TestBacktestHandlersRegisteredInSession`)와 socket validation 케이스에 `GetBacktestRunRequest`를 추가했다.
|
||||
- [API-2] worker `server_test.go`는 이미 `TestWorkerHelloCapabilitiesReflectDeps`의 `starter deps`/`all deps` 케이스로 Starter가 있을 때 `backtest-start`/`worker-execution` capability가 노출됨을 검증하므로 추가 수정 없이 그대로 둔다.
|
||||
- [API-2] 계획이 선호한 "store-backed deps wiring helper 추출 + Starter != nil 단위 테스트" 옵션을 채택했다. `cmd/alt-worker/main.go`에서 `storeBackedDeps(runner, store)` 헬퍼를 분리하고 `cmd/alt-worker/main_test.go`(신규)에서 nil pool로 wiring만 검증한다(생성 경로는 DB를 건드리지 않음).
|
||||
- [API-3] 실행 증거 테스트는 `fixture_test.go` 확장 대신 신규 파일 `services/worker/internal/backtest/flow_test.go`(`package backtest_test`)로 작성했다. import→start→poll→result를 한 흐름에서 동기 실행(`WithStarterLaunch(fn->fn())`)으로 검증하며, 이는 계획이 허용한 "new worker backtest flow test" 경로다. jobs와 backtest 패키지를 함께 import해야 해 import cycle을 피하려 external test package(`backtest_test`)를 사용했다.
|
||||
- [API-4] `runner_error_test.go` 대신 기존 backtest 러너 테스트 파일 `runner_backtest_test.go`에 `TestRunBacktestImportThenStartFlow`를 추가해 import 선행 흐름(run_id/summary/import count)을 fake API로 검증했다. 기존 구조 우선 원칙에 따라 backtest 러너 테스트가 모여 있는 파일을 택했다.
|
||||
|
||||
## 주요 설계 결정
|
||||
|
||||
- [API-1] worker `GetBacktestRun` 핸들러는 신규 run store 의존성을 `Deps`에 추가하지 않고 기존 `deps.Analysis.GetRunDetail`을 재사용한다(계획 선호안). 단일 run 조회 polling이 list/detail과 같은 store-backed read 경로를 공유하고, `not_found`/`unavailable`/`timeout`도 동일한 `backendErrorInfo` 매핑으로 일관되게 typed error를 반환한다.
|
||||
- [API-1] API는 thin control plane 규칙대로 `handleGetBacktestRun`이 run_id shape만 검증하고 worker로 forwarding하며 transport 실패는 `workerErrorInfo`로 typed `ErrorInfo` 변환한다. 새 메서드는 `sendTyped` 공유 경로를 그대로 쓰는 one-line forwarder다.
|
||||
- [API-2] 번들 전략 resolver `BuiltInStrategyPort`는 `strategy-v1`(buy-and-hold: 종목별 첫 bar에서 1주 매수 후 hold)만 알고 미지 id는 error를 반환한다. command flow가 imported bars로부터 결정적(deterministic) terminal result를 단언할 수 있도록 최소·결정적으로 유지했다.
|
||||
- [API-2] 실행은 비동기 규칙을 지켜 `BacktestStarter`가 pending run을 기록하고 import job과 동일한 `Runner`로 실행을 위임한다(두 번째 실행 엔진을 만들지 않음). worker entrypoint에서만 store-backed deps를 연결한다.
|
||||
- [API-4] backtest 시나리오의 import step은 status 시나리오와 동일한 symbol/날짜(`005930`, `20240527`~`20240528`)를 사용하고 `strategy_id: strategy-v1`을 유지해 번들 resolver와 정렬했다.
|
||||
|
||||
## 사용자 리뷰 요청
|
||||
|
||||
_기본값은 `없음`이다. 구현 중 사용자 결정, 사용자 소유 외부 환경/secret/서비스 준비, 또는 계획 범위 변경 없이는 안전하게 진행할 수 없으면 아래 항목을 실제 내용으로 교체하고, 구현을 중단한 뒤 active 파일을 그대로 둔 채 리뷰를 요청한다. 후속 에이전트가 명령 재실행이나 산출물 수집으로 해소할 수 있는 검증 증거 공백만으로는 사용자 리뷰 요청을 작성하지 않는다._
|
||||
|
||||
- 상태: 없음
|
||||
- 사유 유형: 없음
|
||||
- 결정 필요: 없음
|
||||
- 차단 근거: 없음
|
||||
- 실행한 검증/명령: 없음
|
||||
- 자동 후속 불가 이유: 없음
|
||||
- 재개 조건: 없음
|
||||
|
||||
## 리뷰어를 위한 체크포인트
|
||||
|
||||
- `GetBacktestRunRequest` is registered and forwarded through API to worker.
|
||||
- Worker process advertises backtest start only when `Starter` is actually wired.
|
||||
- Backtest execution uses `StorageBarSource` and a bounded built-in strategy resolver, not API internals.
|
||||
- YAML backtest scenarios import bars before start/poll/result.
|
||||
- Result output includes run id, terminal status, and summary keys.
|
||||
|
||||
## 검증 결과
|
||||
|
||||
_구현 에이전트가 각 중간 검증 및 최종 검증 명령 실행 후 출력을 여기에 붙여 넣는다._
|
||||
|
||||
필수 규칙:
|
||||
- 검증 명령은 고정된 계약이다. 임의로 대체하지 않는다.
|
||||
- 대체가 필요하면 `계획 대비 변경 사항`에 이유와 대체 명령을 기록한다.
|
||||
- `검증 결과`에는 실제 stdout/stderr를 붙여 넣는다.
|
||||
- 사용자 리뷰 요청으로 명령을 끝까지 실행하지 못했다면 `사용자 리뷰 요청`에 실행한 명령, 실제 출력, 미실행 명령의 사유를 기록한다.
|
||||
- mobile/UI hang, timeout, 또는 2분 무진행은 blind retry를 중단하고 focused rerun 명령과 screenshot/window/UI-tree evidence path를 남기며, 불가능하면 정확한 사유를 남긴다.
|
||||
|
||||
### API-1 중간 검증
|
||||
```bash
|
||||
$ go test ./services/api/internal/workerclient ./services/api/internal/socket ./services/worker/internal/socket -run 'GetBacktestRun|BacktestHandlers|Capabilities' -count=1
|
||||
ok git.toki-labs.com/toki/alt/services/api/internal/workerclient 0.027s
|
||||
ok git.toki-labs.com/toki/alt/services/api/internal/socket 0.018s
|
||||
ok git.toki-labs.com/toki/alt/services/worker/internal/socket 0.017s
|
||||
```
|
||||
|
||||
### API-2 중간 검증
|
||||
```bash
|
||||
$ go test ./services/worker/internal/backtest ./services/worker/internal/jobs ./services/worker/internal/socket -count=1
|
||||
ok git.toki-labs.com/toki/alt/services/worker/internal/backtest 0.004s
|
||||
ok git.toki-labs.com/toki/alt/services/worker/internal/jobs 0.003s
|
||||
ok git.toki-labs.com/toki/alt/services/worker/internal/socket 0.077s
|
||||
```
|
||||
|
||||
### API-3 중간 검증
|
||||
```bash
|
||||
$ go test ./services/worker/... -run 'Backtest.*Flow|StorageBarSource|RunBacktest|BacktestStarter' -count=1
|
||||
ok git.toki-labs.com/toki/alt/services/worker/internal/backtest 0.004s
|
||||
ok git.toki-labs.com/toki/alt/services/worker/internal/jobs 0.008s
|
||||
ok git.toki-labs.com/toki/alt/services/worker/internal/socket 0.011s
|
||||
(그 외 패키지는 [no test files]/[no tests to run])
|
||||
```
|
||||
|
||||
### API-4 중간 검증
|
||||
```bash
|
||||
$ go test ./apps/cli/internal/operator -run 'Backtest|Handoff' -count=1
|
||||
ok git.toki-labs.com/toki/alt/apps/cli/internal/operator 0.148s
|
||||
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_run_polling.yaml
|
||||
scenario=backtest_run_polling status=valid steps=3
|
||||
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_result_summary.yaml
|
||||
scenario=backtest_result_summary status=valid steps=3
|
||||
```
|
||||
|
||||
### API-5 중간 검증
|
||||
```bash
|
||||
$ git diff --check
|
||||
(출력 없음 / no warnings)
|
||||
```
|
||||
|
||||
### 최종 검증
|
||||
```bash
|
||||
$ go test ./services/api/... -count=1
|
||||
? git.toki-labs.com/toki/alt/services/api/cmd/alt-api [no test files]
|
||||
ok git.toki-labs.com/toki/alt/services/api/internal/config 0.003s
|
||||
ok git.toki-labs.com/toki/alt/services/api/internal/contracts 0.026s
|
||||
ok git.toki-labs.com/toki/alt/services/api/internal/socket 0.042s
|
||||
ok git.toki-labs.com/toki/alt/services/api/internal/workerclient 0.435s
|
||||
|
||||
$ go test ./services/worker/... -count=1
|
||||
ok git.toki-labs.com/toki/alt/services/worker/cmd/alt-worker 0.010s
|
||||
? git.toki-labs.com/toki/alt/services/worker/cmd/alt-worker-data-check [no test files]
|
||||
ok git.toki-labs.com/toki/alt/services/worker/cmd/alt-worker-migrate 0.004s
|
||||
ok git.toki-labs.com/toki/alt/services/worker/internal/backtest 0.004s
|
||||
ok git.toki-labs.com/toki/alt/services/worker/internal/config 0.005s
|
||||
ok git.toki-labs.com/toki/alt/services/worker/internal/contracts 0.009s
|
||||
ok git.toki-labs.com/toki/alt/services/worker/internal/jobs 0.004s
|
||||
ok git.toki-labs.com/toki/alt/services/worker/internal/marketdata/datacheck 0.006s
|
||||
ok git.toki-labs.com/toki/alt/services/worker/internal/marketdata/importer 0.004s
|
||||
ok git.toki-labs.com/toki/alt/services/worker/internal/providers/kis 0.042s
|
||||
ok git.toki-labs.com/toki/alt/services/worker/internal/rediskeys 0.004s
|
||||
ok git.toki-labs.com/toki/alt/services/worker/internal/socket 0.070s
|
||||
? git.toki-labs.com/toki/alt/services/worker/internal/storage [no test files]
|
||||
ok git.toki-labs.com/toki/alt/services/worker/internal/storage/postgres 0.366s
|
||||
? git.toki-labs.com/toki/alt/services/worker/internal/storage/postgres/sqlc [no test files]
|
||||
|
||||
$ go test ./apps/cli/... -count=1
|
||||
? git.toki-labs.com/toki/alt/apps/cli/cmd/alt [no test files]
|
||||
ok git.toki-labs.com/toki/alt/apps/cli/internal/cli 0.016s
|
||||
ok git.toki-labs.com/toki/alt/apps/cli/internal/operator 0.195s
|
||||
|
||||
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_run_polling.yaml
|
||||
scenario=backtest_run_polling status=valid steps=3
|
||||
|
||||
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_result_summary.yaml
|
||||
scenario=backtest_result_summary status=valid steps=3
|
||||
|
||||
$ git diff --check
|
||||
(출력 없음 / no warnings)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
> **[IMPLEMENTING AGENT — BEFORE SAVING] Have you filled in every implementation-owned section: completion table, implementation checklist, changes from plan, design decisions, and verification output?**
|
||||
> If anything is blank, go back and fill it in before saving this file.
|
||||
> Leave review-agent-only sections unchanged.
|
||||
|
||||
## 코드리뷰 결과
|
||||
|
||||
- 종합 판정: PASS
|
||||
- 차원별 평가:
|
||||
- Correctness: Pass
|
||||
- Completeness: Pass
|
||||
- Test coverage: Pass
|
||||
- API contract: Pass
|
||||
- Code quality: Pass
|
||||
- Plan deviation: Pass
|
||||
- Verification trust: Pass
|
||||
- 발견된 문제: 없음
|
||||
- 다음 단계: PASS이므로 active plan/review를 아카이브하고 `complete.log` 작성 후 task directory를 archive로 이동한다.
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
# Complete - m-command-first-operator-workflow/04+02_backtest_runtime_flow
|
||||
|
||||
## 완료 일시
|
||||
|
||||
2026-06-03
|
||||
|
||||
## 요약
|
||||
|
||||
Backtest runtime flow task completed on the first review loop; final verdict PASS.
|
||||
|
||||
## 루프 이력
|
||||
|
||||
| Plan | Review | Verdict | 메모 |
|
||||
|------|--------|---------|------|
|
||||
| `plan_cloud_G08_0.log` | `code_review_cloud_G08_0.log` | PASS | Added API/worker polling rail, store-backed worker execution wiring, built-in strategy resolver, command-flow tests, and import-first backtest fixtures. |
|
||||
|
||||
## 구현/정리 내용
|
||||
|
||||
- Added API and worker `GetBacktestRun` forwarding/handler support for command polling.
|
||||
- Wired `alt-worker` store-backed deps with `BacktestStarter`, `RegisterRunBacktestHandler`, `StorageBarSource`, result store, and the bundled `strategy-v1` resolver.
|
||||
- Added focused worker flow, strategy, wiring, API forwarding, and CLI fixture tests for imported bars -> start/poll/result.
|
||||
- Updated backtest operator YAML fixtures and expected JSONL to import daily bars before run start/result checks.
|
||||
|
||||
## 최종 검증
|
||||
|
||||
- `go test ./services/api/... -count=1` - PASS; API command package had no tests, config/contracts/socket/workerclient packages passed.
|
||||
- `go test ./services/worker/... -count=1` - PASS; worker command/internal packages passed, with no-test packages reported for data-check and storage/sqlc surfaces.
|
||||
- `go test ./apps/cli/... -count=1` - PASS; CLI command package had no tests, internal cli/operator packages passed.
|
||||
- `go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_run_polling.yaml` - PASS; `scenario=backtest_run_polling status=valid steps=3`.
|
||||
- `go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_result_summary.yaml` - PASS; `scenario=backtest_result_summary status=valid steps=3`.
|
||||
- `git diff --check` - PASS; no output.
|
||||
|
||||
## Roadmap Completion
|
||||
|
||||
- Milestone: `agent-roadmap/phase/operator-surface/milestones/command-first-operator-workflow.md`
|
||||
- Completed task ids:
|
||||
- `backtest-command-flow`: PASS; evidence=`agent-task/archive/2026/06/m-command-first-operator-workflow/04+02_backtest_runtime_flow/plan_cloud_G08_0.log`, `agent-task/archive/2026/06/m-command-first-operator-workflow/04+02_backtest_runtime_flow/code_review_cloud_G08_0.log`; verification=`go test ./services/api/... -count=1`, `go test ./services/worker/... -count=1`, `go test ./apps/cli/... -count=1`, `go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_run_polling.yaml`, `go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_result_summary.yaml`, `git diff --check`
|
||||
- Not completed task ids: 없음
|
||||
|
||||
## 잔여 Nit
|
||||
|
||||
- 없음
|
||||
|
||||
## 후속 작업
|
||||
|
||||
- 없음
|
||||
|
|
@ -1,156 +0,0 @@
|
|||
<!-- task=m-command-first-operator-workflow/04+02_backtest_runtime_flow plan=0 tag=API -->
|
||||
|
||||
# Code Review Reference - API
|
||||
|
||||
> **[IMPLEMENTING AGENT — READ FIRST] Filling in this file is the mandatory final step of implementation.**
|
||||
> The task is NOT complete until every implementation-owned section below is filled in.
|
||||
> Complete the `구현 체크리스트`; the final checklist item is mandatory before saving.
|
||||
> Fill implementation-owned sections, then stop with active files in place and report ready for review.
|
||||
> If implementation is blocked by a user-only decision, user-owned external environment prerequisite, or scope conflict, fill `사용자 리뷰 요청` with evidence and stop with active files in place; code-review decides whether to write `USER_REVIEW.md`. Evidence gaps that a follow-up agent can close by rerunning commands or collecting artifacts are normal follow-up issues, not user-review blockers by themselves.
|
||||
> Finalization (`코드리뷰 결과`, log rename, `complete.log`, archive moves, `코드리뷰 전용 체크리스트`) is review-agent-only, even after compaction/resume.
|
||||
> Follow the ownership table at the bottom of this file for which sections you own.
|
||||
|
||||
## 개요
|
||||
|
||||
date=2026-06-03
|
||||
task=m-command-first-operator-workflow/04+02_backtest_runtime_flow, plan=0, tag=API
|
||||
|
||||
## Roadmap Targets
|
||||
|
||||
- Milestone: `agent-roadmap/phase/operator-surface/milestones/command-first-operator-workflow.md`
|
||||
- Task ids:
|
||||
- `backtest-command-flow`: imported daily bars를 입력으로 backtest start/poll/result를 command workflow에서 확인할 수 있다.
|
||||
- Completion mode: check-on-pass
|
||||
|
||||
## 이 파일을 읽는 리뷰 에이전트에게
|
||||
|
||||
> **[REVIEW AGENT ONLY]** 아래 종결 절차는 코드리뷰 에이전트 전용이다. 구현 에이전트는 이 섹션을 실행하지 않는다.
|
||||
|
||||
각 항목의 구현을 실제 소스 파일과 대조하고, `검증 결과` 섹션의 출력이 코드와 일치하는지 확인하세요.
|
||||
|
||||
---
|
||||
|
||||
## 구현 항목별 완료 여부
|
||||
|
||||
| 항목 | 완료 여부 |
|
||||
|------|---------|
|
||||
| [API-1] GetBacktestRun Rail | [ ] |
|
||||
| [API-2] Worker Runtime Wiring | [ ] |
|
||||
| [API-3] Execution Evidence Tests | [ ] |
|
||||
| [API-4] Backtest YAML Flow | [ ] |
|
||||
| [API-5] Verification | [ ] |
|
||||
|
||||
## 구현 체크리스트
|
||||
|
||||
- [ ] [API-1] Add API and worker `GetBacktestRun` forwarding/handler support for polling.
|
||||
- [ ] [API-2] Wire worker `alt-worker` with `BacktestStarter`, `RegisterRunBacktestHandler`, `StorageBarSource`, result store, and a built-in strategy resolver.
|
||||
- [ ] [API-3] Add/update tests for built-in strategy resolver, worker execution wiring, and typed run polling.
|
||||
- [ ] [API-4] Update backtest YAML scenarios to import daily bars before start/poll/result. 검증: start/poll/result scenario가 run id, terminal status, result summary key를 출력한다.
|
||||
- [ ] [API-5] Run API/worker/CLI verification and `git diff --check`.
|
||||
- [ ] CODE_REVIEW-*-G??.md의 구현 에이전트 소유 섹션을 실제 구현 내용과 검증 출력으로 채운다. 이 항목이 완료되기 전에는 구현이 완료된 것이 아니다.
|
||||
|
||||
## 코드리뷰 전용 체크리스트
|
||||
|
||||
> **[REVIEW AGENT ONLY]** 이 체크리스트는 코드리뷰 에이전트만 사용한다.
|
||||
> 구현 에이전트는 이 섹션을 수정하거나 체크하지 않는다.
|
||||
|
||||
- [ ] `코드리뷰 결과`에 `PASS`, `WARN`, `FAIL` 중 하나의 판정을 append한다.
|
||||
- [ ] 판정과 `차원별 평가`, Required/Suggested/Nit 분류가 서로 일치한다.
|
||||
- [ ] active `CODE_REVIEW-*-G??.md`를 `code_review_cloud_G08_N.log`로 아카이브한다.
|
||||
- [ ] active `PLAN-*-G??.md`를 `plan_cloud_G08_M.log`로 아카이브한다.
|
||||
- [ ] `.gitignore`의 Agent-Ops 관리 block이 `agent-task/**/*.md`와 `agent-task/**/*.log`를 unignore하여 plan/review/archive 산출물이 추적 가능한지 확인한다.
|
||||
- [ ] PASS이면 `agent-ops/skills/common/code-review/templates/complete-log-template.md` 기준으로 `complete.log`를 작성하고 active `.md` 파일을 남기지 않는다.
|
||||
- [ ] PASS이면 active task 디렉터리 `agent-task/m-command-first-operator-workflow/04+02_backtest_runtime_flow/`를 `agent-task/archive/YYYY/MM/m-command-first-operator-workflow/04+02_backtest_runtime_flow/`로 이동하고 최종 archive 경로에서 이 체크리스트를 갱신한다.
|
||||
- [ ] PASS이고 task group이 `m-<milestone-slug>`이면 런타임이 읽을 완료 이벤트 메타데이터를 보고하고, roadmap 수정이나 `update-roadmap` 직접 호출을 하지 않는다.
|
||||
- [ ] PASS split 작업이면 이동 후 빈 active parent `agent-task/m-command-first-operator-workflow/`를 제거하거나, 남은 sibling/file이 있어 유지했다고 확인한다.
|
||||
- [ ] WARN/FAIL이고 user-review gate가 트리거되지 않았으면 다음 active `PLAN-cloud-G08.md`와 `CODE_REVIEW-cloud-G08.md`를 작성하고 `complete.log`를 작성하지 않는다.
|
||||
- [ ] USER_REVIEW이면 `agent-ops/skills/common/code-review/templates/user-review-template.md` 기준으로 `USER_REVIEW.md`를 작성하고 active `PLAN-*.md`, `CODE_REVIEW-*.md`, `complete.log`를 남기지 않는다.
|
||||
- [ ] USER_REVIEW가 사용자 결정으로 완료/PASS 해소되면 `USER_REVIEW.md`를 해소 상태로 갱신하고 `complete.log`를 작성한 뒤 task directory를 archive로 이동한다.
|
||||
|
||||
## 계획 대비 변경 사항
|
||||
|
||||
_구현 에이전트가 계획과 다르게 구현한 부분을 이유와 함께 기록한다._
|
||||
|
||||
## 주요 설계 결정
|
||||
|
||||
_구현 에이전트가 주요 설계 결정 사항을 기록한다._
|
||||
|
||||
## 사용자 리뷰 요청
|
||||
|
||||
_기본값은 `없음`이다. 구현 중 사용자 결정, 사용자 소유 외부 환경/secret/서비스 준비, 또는 계획 범위 변경 없이는 안전하게 진행할 수 없으면 아래 항목을 실제 내용으로 교체하고, 구현을 중단한 뒤 active 파일을 그대로 둔 채 리뷰를 요청한다. 후속 에이전트가 명령 재실행이나 산출물 수집으로 해소할 수 있는 검증 증거 공백만으로는 사용자 리뷰 요청을 작성하지 않는다._
|
||||
|
||||
- 상태: 없음
|
||||
- 사유 유형: 없음
|
||||
- 결정 필요: 없음
|
||||
- 차단 근거: 없음
|
||||
- 실행한 검증/명령: 없음
|
||||
- 자동 후속 불가 이유: 없음
|
||||
- 재개 조건: 없음
|
||||
|
||||
## 리뷰어를 위한 체크포인트
|
||||
|
||||
- `GetBacktestRunRequest` is registered and forwarded through API to worker.
|
||||
- Worker process advertises backtest start only when `Starter` is actually wired.
|
||||
- Backtest execution uses `StorageBarSource` and a bounded built-in strategy resolver, not API internals.
|
||||
- YAML backtest scenarios import bars before start/poll/result.
|
||||
- Result output includes run id, terminal status, and summary keys.
|
||||
|
||||
## 검증 결과
|
||||
|
||||
_구현 에이전트가 각 중간 검증 및 최종 검증 명령 실행 후 출력을 여기에 붙여 넣는다._
|
||||
|
||||
필수 규칙:
|
||||
- 검증 명령은 고정된 계약이다. 임의로 대체하지 않는다.
|
||||
- 대체가 필요하면 `계획 대비 변경 사항`에 이유와 대체 명령을 기록한다.
|
||||
- `검증 결과`에는 실제 stdout/stderr를 붙여 넣는다.
|
||||
- 사용자 리뷰 요청으로 명령을 끝까지 실행하지 못했다면 `사용자 리뷰 요청`에 실행한 명령, 실제 출력, 미실행 명령의 사유를 기록한다.
|
||||
- mobile/UI hang, timeout, 또는 2분 무진행은 blind retry를 중단하고 focused rerun 명령과 screenshot/window/UI-tree evidence path를 남기며, 불가능하면 정확한 사유를 남긴다.
|
||||
|
||||
### API-1 중간 검증
|
||||
```bash
|
||||
$ go test ./services/api/internal/workerclient ./services/api/internal/socket ./services/worker/internal/socket -run 'GetBacktestRun|BacktestHandlers|Capabilities' -count=1
|
||||
(output)
|
||||
```
|
||||
|
||||
### API-2 중간 검증
|
||||
```bash
|
||||
$ go test ./services/worker/internal/backtest ./services/worker/internal/jobs ./services/worker/internal/socket -count=1
|
||||
(output)
|
||||
```
|
||||
|
||||
### API-3 중간 검증
|
||||
```bash
|
||||
$ go test ./services/worker/... -run 'Backtest.*Flow|StorageBarSource|RunBacktest|BacktestStarter' -count=1
|
||||
(output)
|
||||
```
|
||||
|
||||
### API-4 중간 검증
|
||||
```bash
|
||||
$ go test ./apps/cli/internal/operator -run 'Backtest|Handoff' -count=1
|
||||
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_run_polling.yaml
|
||||
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_result_summary.yaml
|
||||
(output)
|
||||
```
|
||||
|
||||
### API-5 중간 검증
|
||||
```bash
|
||||
$ git diff --check
|
||||
(output)
|
||||
```
|
||||
|
||||
### 최종 검증
|
||||
```bash
|
||||
$ go test ./services/api/... -count=1
|
||||
$ go test ./services/worker/... -count=1
|
||||
$ go test ./apps/cli/... -count=1
|
||||
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_run_polling.yaml
|
||||
$ go run ./apps/cli/cmd/alt operator scenario validate --file apps/cli/testdata/operator/backtest_result_summary.yaml
|
||||
$ git diff --check
|
||||
(output)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
> **[IMPLEMENTING AGENT — BEFORE SAVING] Have you filled in every implementation-owned section: completion table, implementation checklist, changes from plan, design decisions, and verification output?**
|
||||
> If anything is blank, go back and fill it in before saving this file.
|
||||
> Leave review-agent-only sections unchanged.
|
||||
|
|
@ -39,6 +39,21 @@ type fakeAPI struct {
|
|||
startBacktestReq *altv1.StartBacktestRequest
|
||||
getBacktestRunReq *altv1.GetBacktestRunRequest
|
||||
importDailyBarsReq *altv1.ImportDailyBarsRequest
|
||||
calls []string
|
||||
}
|
||||
|
||||
// recordCall appends an action name in receive order so tests can assert that a
|
||||
// command-first scenario issued its import before any status read.
|
||||
func (f *fakeAPI) recordCall(action string) {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
f.calls = append(f.calls, action)
|
||||
}
|
||||
|
||||
func (f *fakeAPI) callOrder() []string {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
return append([]string(nil), f.calls...)
|
||||
}
|
||||
|
||||
func (f *fakeAPI) setInstReq(req *altv1.ListInstrumentsRequest) {
|
||||
|
|
@ -128,6 +143,7 @@ func startFakeAPIServer(t *testing.T, api *fakeAPI) string {
|
|||
}, nil
|
||||
})
|
||||
protoSocket.AddRequestListenerTyped[*altv1.ListInstrumentsRequest, *altv1.ListInstrumentsResponse](&c.Communicator, func(req *altv1.ListInstrumentsRequest) (*altv1.ListInstrumentsResponse, error) {
|
||||
api.recordCall("list_instruments")
|
||||
api.setInstReq(req)
|
||||
if api.instResp != nil {
|
||||
return api.instResp, nil
|
||||
|
|
@ -135,6 +151,7 @@ func startFakeAPIServer(t *testing.T, api *fakeAPI) string {
|
|||
return &altv1.ListInstrumentsResponse{}, nil
|
||||
})
|
||||
protoSocket.AddRequestListenerTyped[*altv1.ListBarsRequest, *altv1.ListBarsResponse](&c.Communicator, func(req *altv1.ListBarsRequest) (*altv1.ListBarsResponse, error) {
|
||||
api.recordCall("list_bars")
|
||||
api.setBarsReq(req)
|
||||
if api.barsResp != nil {
|
||||
return api.barsResp, nil
|
||||
|
|
@ -183,6 +200,7 @@ func startFakeAPIServer(t *testing.T, api *fakeAPI) string {
|
|||
return &altv1.CompareBacktestRunsResponse{}, nil
|
||||
})
|
||||
protoSocket.AddRequestListenerTyped[*altv1.ImportDailyBarsRequest, *altv1.ImportDailyBarsResponse](&c.Communicator, func(req *altv1.ImportDailyBarsRequest) (*altv1.ImportDailyBarsResponse, error) {
|
||||
api.recordCall("import_daily_bars")
|
||||
api.setImportDailyBarsReq(req)
|
||||
if api.importDailyBarsResp != nil {
|
||||
return api.importDailyBarsResp, nil
|
||||
|
|
|
|||
|
|
@ -110,6 +110,64 @@ func TestExpectedOutputFixturesAreValidJSONLines(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
// TestStatusScenarioFixtureImportsBeforeReads guards the command-first status
|
||||
// scenario: its expected fixture must carry an import_daily_bars step line with
|
||||
// instrument_count and bar_count keys before the instrument/bar read lines, so
|
||||
// the handoff evidence shows imported counts rather than a bare status read.
|
||||
func TestStatusScenarioFixtureImportsBeforeReads(t *testing.T) {
|
||||
const scenario = "market_data_status_query"
|
||||
f, err := os.Open(expectedFixturePath(scenario))
|
||||
if err != nil {
|
||||
t.Fatalf("open expected fixture: %v", err)
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
scanner := bufio.NewScanner(f)
|
||||
importLine := -1
|
||||
instLine := -1
|
||||
barLine := -1
|
||||
lineNo := 0
|
||||
for scanner.Scan() {
|
||||
line := strings.TrimSpace(scanner.Text())
|
||||
if line == "" {
|
||||
continue
|
||||
}
|
||||
lineNo++
|
||||
|
||||
var rec map[string]any
|
||||
if err := json.Unmarshal([]byte(line), &rec); err != nil {
|
||||
t.Fatalf("line %d not valid JSON: %v", lineNo, err)
|
||||
}
|
||||
switch rec["action"] {
|
||||
case "import_daily_bars":
|
||||
importLine = lineNo
|
||||
if _, ok := rec["instrument_count"]; !ok {
|
||||
t.Errorf("import step line %d missing instrument_count", lineNo)
|
||||
}
|
||||
if _, ok := rec["bar_count"]; !ok {
|
||||
t.Errorf("import step line %d missing bar_count", lineNo)
|
||||
}
|
||||
case "list_instruments":
|
||||
instLine = lineNo
|
||||
case "list_bars":
|
||||
barLine = lineNo
|
||||
}
|
||||
}
|
||||
if err := scanner.Err(); err != nil {
|
||||
t.Fatalf("scan fixture: %v", err)
|
||||
}
|
||||
|
||||
if importLine < 0 {
|
||||
t.Fatal("status fixture has no import_daily_bars step line")
|
||||
}
|
||||
if instLine < 0 || barLine < 0 {
|
||||
t.Fatal("status fixture missing instrument/bar read step lines")
|
||||
}
|
||||
if importLine > instLine || importLine > barLine {
|
||||
t.Errorf("import line %d must precede read lines (instruments=%d, bars=%d)", importLine, instLine, barLine)
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandoffDocumentedScenarioFixturesExist guards the handoff matrix against
|
||||
// referencing fixture paths that do not exist. Each required scenario must have
|
||||
// an input YAML fixture and an expected output fixture on disk, and the matrix
|
||||
|
|
|
|||
|
|
@ -59,6 +59,68 @@ func TestRunBacktestRequestOutputsRunID(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
func TestRunBacktestImportThenStartFlow(t *testing.T) {
|
||||
api := &fakeAPI{
|
||||
importDailyBarsResp: &altv1.ImportDailyBarsResponse{Provider: "kis", InstrumentCount: 1, BarCount: 42},
|
||||
startBacktestResp: &altv1.StartBacktestResponse{
|
||||
Run: &altv1.BacktestRun{
|
||||
Id: "run-12345",
|
||||
Status: altv1.BacktestRunStatus_BACKTEST_RUN_STATUS_PENDING,
|
||||
},
|
||||
},
|
||||
}
|
||||
url := startFakeAPIServer(t, api)
|
||||
sc := &Scenario{
|
||||
Name: "backtest_run_request",
|
||||
Steps: []Step{
|
||||
{
|
||||
ID: "import_kr_daily_bars",
|
||||
Action: ActionImportDailyBars,
|
||||
Request: Request{
|
||||
Provider: "kis",
|
||||
SelectorKind: "watchlist",
|
||||
Market: "kr",
|
||||
Venue: "krx",
|
||||
Symbols: []string{"005930"},
|
||||
FromYYYYMMDD: "20240527",
|
||||
ToYYYYMMDD: "20240528",
|
||||
},
|
||||
Expect: Expect{Status: "ok"},
|
||||
},
|
||||
{
|
||||
ID: "start_run",
|
||||
Action: ActionStartBacktest,
|
||||
Request: Request{
|
||||
StrategyID: "strategy-v1",
|
||||
Market: "kr",
|
||||
Timeframe: "daily",
|
||||
FromUnixMs: 1746057600000,
|
||||
ToUnixMs: 1747267200000,
|
||||
},
|
||||
Expect: Expect{Status: "ok"},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
out, code := runScenario(t, sc, url)
|
||||
if code != codeOK {
|
||||
t.Fatalf("exit code = %d, want 0 (out=%q)", code, out)
|
||||
}
|
||||
// The import step runs before the start step and reports imported counts.
|
||||
if !strings.Contains(out, "instrument_count=1") || !strings.Contains(out, "bar_count=42") {
|
||||
t.Errorf("output %q missing import counts", out)
|
||||
}
|
||||
if !strings.Contains(out, "run.id=run-12345") || !strings.Contains(out, "run.status=pending") {
|
||||
t.Errorf("output %q missing start run id/status", out)
|
||||
}
|
||||
if api.lastImportDailyBarsReq() == nil {
|
||||
t.Error("server did not receive an import request before start")
|
||||
}
|
||||
if api.lastStartBacktestReq() == nil {
|
||||
t.Error("server did not receive a start backtest request")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunBacktestPollingStopsOnSucceeded(t *testing.T) {
|
||||
var mu sync.Mutex
|
||||
pollCount := 0
|
||||
|
|
|
|||
|
|
@ -45,9 +45,10 @@ func TestRunScenarioAPISmoke(t *testing.T) {
|
|||
|
||||
func TestRunScenarioMarketDataStatus(t *testing.T) {
|
||||
url := startFakeAPIServer(t, &fakeAPI{
|
||||
helloCaps: []string{"market-read"},
|
||||
instResp: &altv1.ListInstrumentsResponse{Instruments: []*altv1.Instrument{{Id: "KRX:005930"}}},
|
||||
barsResp: &altv1.ListBarsResponse{Bars: []*altv1.Bar{{InstrumentId: "KRX:005930"}}},
|
||||
helloCaps: []string{"market-read"},
|
||||
importDailyBarsResp: &altv1.ImportDailyBarsResponse{Provider: "kis", InstrumentCount: 1, BarCount: 42},
|
||||
instResp: &altv1.ListInstrumentsResponse{Instruments: []*altv1.Instrument{{Id: "KRX:005930"}}},
|
||||
barsResp: &altv1.ListBarsResponse{Bars: []*altv1.Bar{{InstrumentId: "KRX:005930"}}},
|
||||
})
|
||||
sc, err := LoadScenario(fixture(t, "market_data_status_query.yaml"))
|
||||
if err != nil {
|
||||
|
|
@ -60,8 +61,17 @@ func TestRunScenarioMarketDataStatus(t *testing.T) {
|
|||
}
|
||||
for _, want := range []string{
|
||||
"scenario=market_data_status_query",
|
||||
// import-before-read evidence: import counts are in the status output.
|
||||
"step=import_kr_daily_bars",
|
||||
"action=import_daily_bars",
|
||||
"provider=kis",
|
||||
"instrument_count=1",
|
||||
"bar_count=42",
|
||||
// list/read status counts still report.
|
||||
"step=list_kr_instruments",
|
||||
"action=list_instruments",
|
||||
"step=list_daily_bars",
|
||||
"action=list_bars",
|
||||
"status=ok",
|
||||
"count=1",
|
||||
} {
|
||||
|
|
@ -71,6 +81,88 @@ func TestRunScenarioMarketDataStatus(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
// TestRunScenarioMarketDataStatusImportsBeforeReads proves the command-first
|
||||
// status scenario issues its import_daily_bars call before the instrument and
|
||||
// bar status reads, so the reported counts describe freshly imported data.
|
||||
func TestRunScenarioMarketDataStatusImportsBeforeReads(t *testing.T) {
|
||||
api := &fakeAPI{
|
||||
helloCaps: []string{"market-read"},
|
||||
importDailyBarsResp: &altv1.ImportDailyBarsResponse{Provider: "kis", InstrumentCount: 1, BarCount: 42},
|
||||
instResp: &altv1.ListInstrumentsResponse{Instruments: []*altv1.Instrument{{Id: "KRX:005930"}}},
|
||||
barsResp: &altv1.ListBarsResponse{Bars: []*altv1.Bar{{InstrumentId: "KRX:005930"}}},
|
||||
}
|
||||
url := startFakeAPIServer(t, api)
|
||||
sc, err := LoadScenario(fixture(t, "market_data_status_query.yaml"))
|
||||
if err != nil {
|
||||
t.Fatalf("load scenario: %v", err)
|
||||
}
|
||||
|
||||
out, code := runScenario(t, sc, url)
|
||||
if code != codeOK {
|
||||
t.Fatalf("exit code = %d, want 0 (out=%q)", code, out)
|
||||
}
|
||||
|
||||
// The import actually carried the fixture's KIS window, not a default read.
|
||||
imp := api.lastImportDailyBarsReq()
|
||||
if imp == nil {
|
||||
t.Fatal("server did not receive an import_daily_bars request")
|
||||
}
|
||||
if imp.GetProvider() != "kis" || imp.GetFromYyyymmdd() != "20240527" || imp.GetToYyyymmdd() != "20240528" {
|
||||
t.Errorf("import request = %+v, want kis 20240527..20240528", imp)
|
||||
}
|
||||
|
||||
// Call order: import precedes both status reads.
|
||||
wantOrder := []string{"import_daily_bars", "list_instruments", "list_bars"}
|
||||
if got := api.callOrder(); !equalStrings(got, wantOrder) {
|
||||
t.Errorf("call order = %v, want %v (out=%q)", got, wantOrder, out)
|
||||
}
|
||||
}
|
||||
|
||||
// TestRunScenarioMarketStatusTypedUnavailable proves the status scenario surfaces
|
||||
// a typed unavailable ErrorInfo in its machine-readable output when the import
|
||||
// step fails, instead of degrading into a transport error.
|
||||
func TestRunScenarioMarketStatusTypedUnavailable(t *testing.T) {
|
||||
url := startFakeAPIServer(t, &fakeAPI{
|
||||
importDailyBarsResp: &altv1.ImportDailyBarsResponse{
|
||||
Error: &altv1.ErrorInfo{Code: "unavailable", Message: "market worker offline"},
|
||||
},
|
||||
})
|
||||
sc := &Scenario{
|
||||
Name: "market_status_typed_unavailable",
|
||||
Steps: []Step{{
|
||||
ID: "import_kr_daily_bars",
|
||||
Action: ActionImportDailyBars,
|
||||
Request: Request{
|
||||
Provider: "kis",
|
||||
SelectorKind: "watchlist",
|
||||
Symbols: []string{"005930"},
|
||||
},
|
||||
Expect: Expect{Status: "error", ErrorCode: "unavailable"},
|
||||
}},
|
||||
}
|
||||
|
||||
out, code := runScenario(t, sc, url)
|
||||
if code != codeOK {
|
||||
t.Fatalf("exit code = %d, want 0 (out=%q)", code, out)
|
||||
}
|
||||
if !strings.Contains(out, "status=ok") || !strings.Contains(out, "error.code=unavailable") {
|
||||
t.Errorf("output %q missing expected typed unavailable markers", out)
|
||||
}
|
||||
}
|
||||
|
||||
// equalStrings reports whether two string slices have the same elements in order.
|
||||
func equalStrings(a, b []string) bool {
|
||||
if len(a) != len(b) {
|
||||
return false
|
||||
}
|
||||
for i := range a {
|
||||
if a[i] != b[i] {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func TestRunScenarioMarketEmptyResult(t *testing.T) {
|
||||
url := startFakeAPIServer(t, &fakeAPI{instResp: &altv1.ListInstrumentsResponse{}})
|
||||
sc := &Scenario{
|
||||
|
|
|
|||
|
|
@ -1,6 +1,18 @@
|
|||
name: backtest_result_summary
|
||||
timeout: 5s
|
||||
steps:
|
||||
- id: import_kr_daily_bars
|
||||
action: import_daily_bars
|
||||
request:
|
||||
provider: kis
|
||||
selector_kind: watchlist
|
||||
market: kr
|
||||
venue: krx
|
||||
symbols: ["005930"]
|
||||
from_yyyymmdd: "20240527"
|
||||
to_yyyymmdd: "20240528"
|
||||
expect:
|
||||
status: ok
|
||||
- id: start
|
||||
action: start_backtest
|
||||
request:
|
||||
|
|
|
|||
|
|
@ -1,6 +1,18 @@
|
|||
name: backtest_run_polling
|
||||
timeout: 5s
|
||||
steps:
|
||||
- id: import_kr_daily_bars
|
||||
action: import_daily_bars
|
||||
request:
|
||||
provider: kis
|
||||
selector_kind: watchlist
|
||||
market: kr
|
||||
venue: krx
|
||||
symbols: ["005930"]
|
||||
from_yyyymmdd: "20240527"
|
||||
to_yyyymmdd: "20240528"
|
||||
expect:
|
||||
status: ok
|
||||
- id: start
|
||||
action: start_backtest
|
||||
request:
|
||||
|
|
|
|||
|
|
@ -1,6 +1,18 @@
|
|||
name: backtest_run_request
|
||||
timeout: 5s
|
||||
steps:
|
||||
- id: import_kr_daily_bars
|
||||
action: import_daily_bars
|
||||
request:
|
||||
provider: kis
|
||||
selector_kind: watchlist
|
||||
market: kr
|
||||
venue: krx
|
||||
symbols: ["005930"]
|
||||
from_yyyymmdd: "20240527"
|
||||
to_yyyymmdd: "20240528"
|
||||
expect:
|
||||
status: ok
|
||||
- id: start_run
|
||||
action: start_backtest
|
||||
request:
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
{"action":"import_daily_bars","bar_count":42,"instrument_count":1,"provider":"kis","scenario":"backtest_result_summary","status":"ok","step":"import_kr_daily_bars","type":"step"}
|
||||
{"action":"start_backtest","run_id":"run-0001","run_status":"pending","scenario":"backtest_result_summary","status":"ok","step":"start","type":"step"}
|
||||
{"action":"get_backtest_result","ending_equity":"1125000","run_id":"run-0001","scenario":"backtest_result_summary","starting_cash":"1000000","status":"ok","step":"result","total_return":"0.125","trade_count":12,"type":"step"}
|
||||
{"exit_code":0,"passed":2,"scenario":"backtest_result_summary","status":"ok","steps":2,"type":"summary"}
|
||||
{"exit_code":0,"passed":3,"scenario":"backtest_result_summary","status":"ok","steps":3,"type":"summary"}
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
{"action":"import_daily_bars","bar_count":42,"instrument_count":1,"provider":"kis","scenario":"backtest_run_polling","status":"ok","step":"import_kr_daily_bars","type":"step"}
|
||||
{"action":"start_backtest","run_id":"run-0001","run_status":"pending","scenario":"backtest_run_polling","status":"ok","step":"start","type":"step"}
|
||||
{"action":"poll_backtest_run","run_id":"run-0001","run_status":"succeeded","scenario":"backtest_run_polling","status":"ok","step":"poll","type":"step"}
|
||||
{"exit_code":0,"passed":2,"scenario":"backtest_run_polling","status":"ok","steps":2,"type":"summary"}
|
||||
{"exit_code":0,"passed":3,"scenario":"backtest_run_polling","status":"ok","steps":3,"type":"summary"}
|
||||
|
|
|
|||
|
|
@ -1,2 +1,3 @@
|
|||
{"action":"import_daily_bars","bar_count":42,"instrument_count":1,"provider":"kis","scenario":"backtest_run_request","status":"ok","step":"import_kr_daily_bars","type":"step"}
|
||||
{"action":"start_backtest","run_id":"run-0001","run_status":"pending","scenario":"backtest_run_request","status":"ok","step":"start_run","type":"step"}
|
||||
{"exit_code":0,"passed":1,"scenario":"backtest_run_request","status":"ok","steps":1,"type":"summary"}
|
||||
{"exit_code":0,"passed":2,"scenario":"backtest_run_request","status":"ok","steps":2,"type":"summary"}
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
{"action":"hello","scenario":"market_data_status_query","status":"ok","step":"hello","type":"step"}
|
||||
{"action":"import_daily_bars","bar_count":42,"instrument_count":1,"provider":"kis","scenario":"market_data_status_query","status":"ok","step":"import_kr_daily_bars","type":"step"}
|
||||
{"action":"list_instruments","count":1,"scenario":"market_data_status_query","status":"ok","step":"list_kr_instruments","type":"step"}
|
||||
{"action":"list_bars","count":10,"scenario":"market_data_status_query","status":"ok","step":"list_daily_bars","type":"step"}
|
||||
{"exit_code":0,"passed":3,"scenario":"market_data_status_query","status":"ok","steps":3,"type":"summary"}
|
||||
{"exit_code":0,"passed":4,"scenario":"market_data_status_query","status":"ok","steps":4,"type":"summary"}
|
||||
|
|
|
|||
|
|
@ -26,7 +26,7 @@ ALT 운영 UI 게이트(`agent-ops/rules/project/rules.md`의 "운영 UI 구현
|
|||
| Scenario | Command | Input fixture | Expected output fixture | Expected exit code | Checked protobuf/view-model field | Remaining Flutter wireframe dependency |
|
||||
|----------|---------|---------------|-------------------------|--------------------|-----------------------------------|----------------------------------------|
|
||||
| `api_connection_smoke` | `alt operator scenario run --file testdata/operator/api_connection_smoke.yaml --api-url ws://127.0.0.1:8080/socket --output jsonl` | `testdata/operator/api_connection_smoke.yaml` | `testdata/operator/expected/api_connection_smoke.jsonl` | `0` (성공), `3` (transport 실패 시) | `HelloResponse.capabilities` (`alt.v1`); summary `status`/`exit_code` | connection status / capability 표시 화면 layout 미정 |
|
||||
| `market_data_status_query` | `alt operator scenario run --file testdata/operator/market_data_status_query.yaml --api-url ws://127.0.0.1:8080/socket --output jsonl` | `testdata/operator/market_data_status_query.yaml` | `testdata/operator/expected/market_data_status_query.jsonl` | `0` (성공/empty), `1` (mismatch) | `ListInstrumentsResponse.instruments`, `ListBarsResponse.bars` (step `count`), `ErrorInfo.code` | market data status card / 빈 상태·unavailable 시각화 미정 |
|
||||
| `market_data_status_query` | `alt operator scenario run --file testdata/operator/market_data_status_query.yaml --api-url ws://127.0.0.1:8080/socket --output jsonl` | `testdata/operator/market_data_status_query.yaml` | `testdata/operator/expected/market_data_status_query.jsonl` | `0` (성공/empty), `1` (mismatch) | command-first: `import_daily_bars`를 먼저 실행해 `ImportDailyBarsResponse.instrument_count`/`ImportDailyBarsResponse.bar_count`를 확인한 뒤 `ListInstrumentsResponse.instruments`, `ListBarsResponse.bars` (step `count`) 조회; typed 실패 시 `ErrorInfo.code` (`unavailable`) | market data status card / 수집 결과(종목수·바개수)·빈 상태·unavailable 시각화 미정 |
|
||||
| `backtest_run_request` | `alt operator scenario run --file testdata/operator/backtest_run_request.yaml --api-url ws://127.0.0.1:8080/socket --output jsonl` | `testdata/operator/backtest_run_request.yaml` | `testdata/operator/expected/backtest_run_request.jsonl` | `0` (성공), `1` (typed error/mismatch) | `StartBacktestResponse.run.id`, `BacktestRun.status` (step `run_id`/`run_status`), `ErrorInfo.code` | backtest 생성 form 입력/검증 화면 미정 |
|
||||
| `backtest_run_polling` | `alt operator scenario run --file testdata/operator/backtest_run_polling.yaml --api-url ws://127.0.0.1:8080/socket --output jsonl` | `testdata/operator/backtest_run_polling.yaml` | `testdata/operator/expected/backtest_run_polling.jsonl` | `0` (terminal 도달), `3` (polling timeout) | `GetBacktestRunResponse.run.status` 전이(pending→running→succeeded/failed/canceled) | run 진행 상태/전이 표시(진행바, 상태 칩) 화면 미정 |
|
||||
| `backtest_result_summary` | `alt operator scenario run --file testdata/operator/backtest_result_summary.yaml --api-url ws://127.0.0.1:8080/socket --output jsonl` | `testdata/operator/backtest_result_summary.yaml` | `testdata/operator/expected/backtest_result_summary.jsonl` | `0` (성공), `1` (missing run/typed error) | `GetBacktestResultResponse.result.summary` (`starting_cash`, `ending_equity`, `total_return`, `trade_count`) | result summary metrics / chart 시각화 화면 미정 |
|
||||
|
|
|
|||
|
|
@ -1,7 +1,10 @@
|
|||
# Market data status scenario. It greets the API, then reads instruments and
|
||||
# bars so an operator can confirm market data status from the terminal without a
|
||||
# screen. `alt operator scenario run --file <this> --api-url <ws-url>` executes
|
||||
# it; `alt operator scenario validate --file <this>` checks it offline.
|
||||
# Market data status scenario. It greets the API, imports KIS daily bars, then
|
||||
# reads instruments and bars so an operator can confirm imported market data
|
||||
# status from the terminal without a screen. Running the import first makes this
|
||||
# a command-first status workflow rather than a generic read of whatever data
|
||||
# already happens to be stored. `alt operator scenario run --file <this> --api-url
|
||||
# <ws-url>` executes it; `alt operator scenario validate --file <this>` checks it
|
||||
# offline.
|
||||
name: market_data_status_query
|
||||
timeout: 5s
|
||||
steps:
|
||||
|
|
@ -11,6 +14,18 @@ steps:
|
|||
status: ok
|
||||
capabilities:
|
||||
- market-read
|
||||
- id: import_kr_daily_bars
|
||||
action: import_daily_bars
|
||||
request:
|
||||
provider: kis
|
||||
selector_kind: watchlist
|
||||
market: kr
|
||||
venue: krx
|
||||
symbols: ["005930"]
|
||||
from_yyyymmdd: "20240527"
|
||||
to_yyyymmdd: "20240528"
|
||||
expect:
|
||||
status: ok
|
||||
- id: list_kr_instruments
|
||||
action: list_instruments
|
||||
request:
|
||||
|
|
|
|||
|
|
@ -36,6 +36,14 @@ func backtestHandlers(worker workerclient.WorkerClient) []sessionHandler {
|
|||
})
|
||||
},
|
||||
},
|
||||
{
|
||||
requestType: protoSocket.TypeNameOf(&altv1.GetBacktestRunRequest{}),
|
||||
register: func(client *protoSocket.WsClient) {
|
||||
protoSocket.AddRequestListenerTyped[*altv1.GetBacktestRunRequest, *altv1.GetBacktestRunResponse](&client.Communicator, func(req *altv1.GetBacktestRunRequest) (*altv1.GetBacktestRunResponse, error) {
|
||||
return handleGetBacktestRun(worker, req)
|
||||
})
|
||||
},
|
||||
},
|
||||
{
|
||||
requestType: protoSocket.TypeNameOf(&altv1.ListBacktestRunsRequest{}),
|
||||
register: func(client *protoSocket.WsClient) {
|
||||
|
|
@ -93,6 +101,28 @@ func handleStartBacktest(worker workerclient.WorkerClient, req *altv1.StartBackt
|
|||
return res, nil
|
||||
}
|
||||
|
||||
func handleGetBacktestRun(worker workerclient.WorkerClient, req *altv1.GetBacktestRunRequest) (*altv1.GetBacktestRunResponse, error) {
|
||||
if req.GetRunId() == "" {
|
||||
return &altv1.GetBacktestRunResponse{Error: invalidRequest("run_id is required")}, nil
|
||||
}
|
||||
if worker == nil {
|
||||
return &altv1.GetBacktestRunResponse{Error: workerUnavailable()}, nil
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), workerRequestTimeout)
|
||||
defer cancel()
|
||||
if err := worker.Connect(ctx); err != nil {
|
||||
return &altv1.GetBacktestRunResponse{Error: workerErrorInfo(err)}, nil
|
||||
}
|
||||
res, err := worker.GetBacktestRun(ctx, req)
|
||||
if err != nil {
|
||||
return &altv1.GetBacktestRunResponse{Error: workerErrorInfo(err)}, nil
|
||||
}
|
||||
if res == nil {
|
||||
return &altv1.GetBacktestRunResponse{Error: internalError("worker returned no run response")}, nil
|
||||
}
|
||||
return res, nil
|
||||
}
|
||||
|
||||
func handleListBacktestRuns(worker workerclient.WorkerClient, req *altv1.ListBacktestRunsRequest) (*altv1.ListBacktestRunsResponse, error) {
|
||||
if worker == nil {
|
||||
return &altv1.ListBacktestRunsResponse{Error: workerUnavailable()}, nil
|
||||
|
|
|
|||
|
|
@ -16,6 +16,7 @@ import (
|
|||
// the last forwarded request and returns canned responses/errors.
|
||||
type fakeWorkerClient struct {
|
||||
startReq *altv1.StartBacktestRequest
|
||||
getRunReq *altv1.GetBacktestRunRequest
|
||||
listReq *altv1.ListBacktestRunsRequest
|
||||
detailReq *altv1.GetBacktestRunDetailRequest
|
||||
resultReq *altv1.GetBacktestResultRequest
|
||||
|
|
@ -25,6 +26,7 @@ type fakeWorkerClient struct {
|
|||
importReq *altv1.ImportDailyBarsRequest
|
||||
|
||||
startRes *altv1.StartBacktestResponse
|
||||
getRunRes *altv1.GetBacktestRunResponse
|
||||
listRes *altv1.ListBacktestRunsResponse
|
||||
detailRes *altv1.GetBacktestRunDetailResponse
|
||||
resultRes *altv1.GetBacktestResultResponse
|
||||
|
|
@ -59,6 +61,10 @@ func (f *fakeWorkerClient) StartBacktest(ctx context.Context, req *altv1.StartBa
|
|||
f.startReq = req
|
||||
return f.startRes, f.err
|
||||
}
|
||||
func (f *fakeWorkerClient) GetBacktestRun(ctx context.Context, req *altv1.GetBacktestRunRequest) (*altv1.GetBacktestRunResponse, error) {
|
||||
f.getRunReq = req
|
||||
return f.getRunRes, f.err
|
||||
}
|
||||
func (f *fakeWorkerClient) ListBacktestRuns(ctx context.Context, req *altv1.ListBacktestRunsRequest) (*altv1.ListBacktestRunsResponse, error) {
|
||||
f.listReq = req
|
||||
return f.listRes, f.err
|
||||
|
|
@ -199,6 +205,47 @@ func TestHandleListBacktestRunsForwards(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
func TestHandleGetBacktestRunForwards(t *testing.T) {
|
||||
fake := &fakeWorkerClient{getRunRes: &altv1.GetBacktestRunResponse{Run: &altv1.BacktestRun{Id: "run-1", Status: altv1.BacktestRunStatus_BACKTEST_RUN_STATUS_RUNNING}}}
|
||||
|
||||
resp, err := handleGetBacktestRun(fake, &altv1.GetBacktestRunRequest{RunId: "run-1"})
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if fake.getRunReq.GetRunId() != "run-1" {
|
||||
t.Errorf("run id was not forwarded unchanged, got %q", fake.getRunReq.GetRunId())
|
||||
}
|
||||
if resp.GetRun().GetId() != "run-1" {
|
||||
t.Errorf("unexpected run id: %q", resp.GetRun().GetId())
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleGetBacktestRunRequiresRunID(t *testing.T) {
|
||||
fake := &fakeWorkerClient{}
|
||||
|
||||
resp, err := handleGetBacktestRun(fake, &altv1.GetBacktestRunRequest{})
|
||||
if err != nil {
|
||||
t.Fatalf("expected typed validation response, got error: %v", err)
|
||||
}
|
||||
requireBacktestError(t, resp.GetError(), backtestErrorInvalidRequest)
|
||||
if fake.getRunReq != nil {
|
||||
t.Error("worker must not be called when validation fails")
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleGetBacktestRunNilWorkerResponse(t *testing.T) {
|
||||
fake := &fakeWorkerClient{}
|
||||
|
||||
resp, err := handleGetBacktestRun(fake, &altv1.GetBacktestRunRequest{RunId: "run-1"})
|
||||
if err != nil {
|
||||
t.Fatalf("expected typed internal response, got error: %v", err)
|
||||
}
|
||||
requireBacktestError(t, resp.GetError(), backtestErrorInternal)
|
||||
if fake.getRunReq == nil {
|
||||
t.Error("expected request to be forwarded before nil response was detected")
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleGetBacktestRunDetailRequiresRunID(t *testing.T) {
|
||||
fake := &fakeWorkerClient{}
|
||||
|
||||
|
|
@ -316,6 +363,21 @@ func TestBacktestSocketValidationReturnsTypedErrors(t *testing.T) {
|
|||
return resp.GetError()
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "get run missing run id",
|
||||
run: func(t *testing.T) *altv1.ErrorInfo {
|
||||
t.Helper()
|
||||
resp, err := protoSocket.SendRequestTyped[*altv1.GetBacktestRunRequest, *altv1.GetBacktestRunResponse](
|
||||
&client.Communicator,
|
||||
&altv1.GetBacktestRunRequest{},
|
||||
500*time.Millisecond,
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("request should return typed error response without timeout: %v", err)
|
||||
}
|
||||
return resp.GetError()
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "detail missing run id",
|
||||
run: func(t *testing.T) *altv1.ErrorInfo {
|
||||
|
|
@ -368,7 +430,7 @@ func TestBacktestSocketValidationReturnsTypedErrors(t *testing.T) {
|
|||
requireBacktestError(t, tt.run(t), backtestErrorInvalidRequest)
|
||||
})
|
||||
}
|
||||
if fake.startReq != nil || fake.detailReq != nil || fake.resultReq != nil || fake.compareReq != nil {
|
||||
if fake.startReq != nil || fake.getRunReq != nil || fake.detailReq != nil || fake.resultReq != nil || fake.compareReq != nil {
|
||||
t.Fatal("worker must not be called for socket-level validation failures")
|
||||
}
|
||||
}
|
||||
|
|
@ -439,6 +501,7 @@ func TestBacktestHandlersRegisteredInSession(t *testing.T) {
|
|||
}
|
||||
required := []string{
|
||||
protoSocket.TypeNameOf(&altv1.StartBacktestRequest{}),
|
||||
protoSocket.TypeNameOf(&altv1.GetBacktestRunRequest{}),
|
||||
protoSocket.TypeNameOf(&altv1.ListBacktestRunsRequest{}),
|
||||
protoSocket.TypeNameOf(&altv1.GetBacktestRunDetailRequest{}),
|
||||
protoSocket.TypeNameOf(&altv1.GetBacktestResultRequest{}),
|
||||
|
|
|
|||
|
|
@ -31,6 +31,7 @@ type WorkerClient interface {
|
|||
// Backtest command/query surface. The API forwards client requests onto the
|
||||
// worker unchanged, so each method mirrors a contract request/response pair.
|
||||
StartBacktest(ctx context.Context, req *altv1.StartBacktestRequest) (*altv1.StartBacktestResponse, error)
|
||||
GetBacktestRun(ctx context.Context, req *altv1.GetBacktestRunRequest) (*altv1.GetBacktestRunResponse, error)
|
||||
ListBacktestRuns(ctx context.Context, req *altv1.ListBacktestRunsRequest) (*altv1.ListBacktestRunsResponse, error)
|
||||
GetBacktestRunDetail(ctx context.Context, req *altv1.GetBacktestRunDetailRequest) (*altv1.GetBacktestRunDetailResponse, error)
|
||||
GetBacktestResult(ctx context.Context, req *altv1.GetBacktestResultRequest) (*altv1.GetBacktestResultResponse, error)
|
||||
|
|
@ -132,6 +133,10 @@ func (c *socketClient) StartBacktest(ctx context.Context, req *altv1.StartBackte
|
|||
return sendTyped[*altv1.StartBacktestRequest, *altv1.StartBacktestResponse](c, ctx, req)
|
||||
}
|
||||
|
||||
func (c *socketClient) GetBacktestRun(ctx context.Context, req *altv1.GetBacktestRunRequest) (*altv1.GetBacktestRunResponse, error) {
|
||||
return sendTyped[*altv1.GetBacktestRunRequest, *altv1.GetBacktestRunResponse](c, ctx, req)
|
||||
}
|
||||
|
||||
func (c *socketClient) ListBacktestRuns(ctx context.Context, req *altv1.ListBacktestRunsRequest) (*altv1.ListBacktestRunsResponse, error) {
|
||||
return sendTyped[*altv1.ListBacktestRunsRequest, *altv1.ListBacktestRunsResponse](c, ctx, req)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -122,6 +122,47 @@ func TestWorkerClient_StartBacktest_Success(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
func TestWorkerClient_GetBacktestRun_Success(t *testing.T) {
|
||||
port, cleanup := startFakeWorker(t, func(client *protoSocket.WsClient) {
|
||||
protoSocket.AddRequestListenerTyped[*altv1.GetBacktestRunRequest, *altv1.GetBacktestRunResponse](&client.Communicator, func(req *altv1.GetBacktestRunRequest) (*altv1.GetBacktestRunResponse, error) {
|
||||
return &altv1.GetBacktestRunResponse{
|
||||
Run: &altv1.BacktestRun{
|
||||
Id: req.GetRunId(),
|
||||
Status: altv1.BacktestRunStatus_BACKTEST_RUN_STATUS_SUCCEEDED,
|
||||
},
|
||||
}, nil
|
||||
})
|
||||
})
|
||||
defer cleanup()
|
||||
|
||||
client := New(fmt.Sprintf("ws://127.0.0.1:%d/socket", port))
|
||||
ctx := context.Background()
|
||||
if err := client.Connect(ctx); err != nil {
|
||||
t.Fatalf("failed to connect: %v", err)
|
||||
}
|
||||
defer client.Close()
|
||||
|
||||
res, err := client.GetBacktestRun(ctx, &altv1.GetBacktestRunRequest{RunId: "run-1"})
|
||||
if err != nil {
|
||||
t.Fatalf("GetBacktestRun failed: %v", err)
|
||||
}
|
||||
if res.GetRun().GetId() != "run-1" {
|
||||
t.Errorf("run id did not round-trip, got %q", res.GetRun().GetId())
|
||||
}
|
||||
if res.GetRun().GetStatus() != altv1.BacktestRunStatus_BACKTEST_RUN_STATUS_SUCCEEDED {
|
||||
t.Errorf("status did not round-trip, got %v", res.GetRun().GetStatus())
|
||||
}
|
||||
}
|
||||
|
||||
func TestWorkerClient_GetBacktestRun_Unavailable(t *testing.T) {
|
||||
// A never-connected client must report ErrUnavailable instead of blocking.
|
||||
client := New("ws://127.0.0.1:54321/socket")
|
||||
_, err := client.GetBacktestRun(context.Background(), &altv1.GetBacktestRunRequest{RunId: "run-1"})
|
||||
if !errors.Is(err, ErrUnavailable) {
|
||||
t.Errorf("expected ErrUnavailable, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWorkerClient_ListInstruments_Success(t *testing.T) {
|
||||
port, cleanup := startFakeWorker(t, func(client *protoSocket.WsClient) {
|
||||
protoSocket.AddRequestListenerTyped[*altv1.ListInstrumentsRequest, *altv1.ListInstrumentsResponse](&client.Communicator, func(req *altv1.ListInstrumentsRequest) (*altv1.ListInstrumentsResponse, error) {
|
||||
|
|
|
|||
|
|
@ -8,9 +8,12 @@ import (
|
|||
"os/signal"
|
||||
"syscall"
|
||||
|
||||
"time"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
|
||||
"git.toki-labs.com/toki/alt/packages/domain/market"
|
||||
workerbacktest "git.toki-labs.com/toki/alt/services/worker/internal/backtest"
|
||||
"git.toki-labs.com/toki/alt/services/worker/internal/config"
|
||||
"git.toki-labs.com/toki/alt/services/worker/internal/jobs"
|
||||
"git.toki-labs.com/toki/alt/services/worker/internal/marketdata/importer"
|
||||
|
|
@ -36,20 +39,7 @@ func main() {
|
|||
slog.Error("failed to create database pool; store-backed socket surfaces disabled", "error", err)
|
||||
} else {
|
||||
defer pool.Close()
|
||||
store := postgres.NewStore(pool)
|
||||
kisProvider := kis.NewLiveProvider(kis.NewClient(kis.ConfigFromEnv()))
|
||||
// One importer instance backs both the async job handler and the socket
|
||||
// import command so the operator-facing command and scheduled jobs share
|
||||
// the same provider/store wiring.
|
||||
kisImporter := importer.New(kisProvider, store)
|
||||
jobs.RegisterDailyBarImportHandler(runner, market.ProviderKIS, kisImporter)
|
||||
deps = socket.Deps{
|
||||
Analysis: store,
|
||||
Results: store,
|
||||
Instruments: store,
|
||||
Bars: store,
|
||||
DailyBarImporter: kisImporter,
|
||||
}
|
||||
deps = storeBackedDeps(runner, postgres.NewStore(pool))
|
||||
}
|
||||
|
||||
server := socket.NewServer(cfg, deps)
|
||||
|
|
@ -75,3 +65,34 @@ func main() {
|
|||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
// storeBackedDeps wires every store-backed socket dependency around a single
|
||||
// store, including the backtest execution rail. It is factored out of main so a
|
||||
// unit test can assert the backtest start/read surfaces are wired without a live
|
||||
// database. Backtest execution reads imported daily bars from storage and runs
|
||||
// the bundled strategy resolver; the starter records the pending run and hands
|
||||
// execution to the same runner the import jobs use, keeping a single async job
|
||||
// rail rather than a second execution engine.
|
||||
func storeBackedDeps(runner *jobs.Runner, store *postgres.Store) socket.Deps {
|
||||
kisProvider := kis.NewLiveProvider(kis.NewClient(kis.ConfigFromEnv()))
|
||||
// One importer instance backs both the async job handler and the socket
|
||||
// import command so the operator-facing command and scheduled jobs share the
|
||||
// same provider/store wiring.
|
||||
kisImporter := importer.New(kisProvider, store)
|
||||
jobs.RegisterDailyBarImportHandler(runner, market.ProviderKIS, kisImporter)
|
||||
|
||||
barSource := workerbacktest.NewStorageBarSource(store, store)
|
||||
strategyPort := workerbacktest.NewBuiltInStrategyPort()
|
||||
engine := workerbacktest.NewEngine(barSource, strategyPort, store)
|
||||
jobs.RegisterRunBacktestHandler(runner, store, engine, time.Now)
|
||||
starter := jobs.NewBacktestStarter(runner, store)
|
||||
|
||||
return socket.Deps{
|
||||
Starter: starter,
|
||||
Analysis: store,
|
||||
Results: store,
|
||||
Instruments: store,
|
||||
Bars: store,
|
||||
DailyBarImporter: kisImporter,
|
||||
}
|
||||
}
|
||||
|
|
|
|||
37
services/worker/cmd/alt-worker/main_test.go
Normal file
37
services/worker/cmd/alt-worker/main_test.go
Normal file
|
|
@ -0,0 +1,37 @@
|
|||
package main
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"git.toki-labs.com/toki/alt/services/worker/internal/jobs"
|
||||
"git.toki-labs.com/toki/alt/services/worker/internal/storage/postgres"
|
||||
)
|
||||
|
||||
// TestStoreBackedDepsWiresBacktestSurfaces guards against a regression where the
|
||||
// worker process serves backtest reads but leaves the start surface (Starter)
|
||||
// unwired, which would advertise no backtest-start capability and reject every
|
||||
// start command. The store is built with a nil pool because the wiring path only
|
||||
// constructs adapters; it never touches the database at startup.
|
||||
func TestStoreBackedDepsWiresBacktestSurfaces(t *testing.T) {
|
||||
runner := jobs.NewRunner()
|
||||
deps := storeBackedDeps(runner, postgres.NewStore(nil))
|
||||
|
||||
if deps.Starter == nil {
|
||||
t.Error("expected backtest Starter to be wired")
|
||||
}
|
||||
if deps.Analysis == nil || deps.Results == nil {
|
||||
t.Error("expected backtest read stores to be wired")
|
||||
}
|
||||
if deps.Instruments == nil || deps.Bars == nil {
|
||||
t.Error("expected market stores to be wired")
|
||||
}
|
||||
if deps.DailyBarImporter == nil {
|
||||
t.Error("expected daily bar importer to be wired")
|
||||
}
|
||||
|
||||
// Wiring registers the import and run-backtest job handlers on the runner, so
|
||||
// it must hold at least those two handlers after wiring.
|
||||
if runner.Len() < 2 {
|
||||
t.Errorf("expected wiring to register job handlers, runner holds %d", runner.Len())
|
||||
}
|
||||
}
|
||||
233
services/worker/internal/backtest/flow_test.go
Normal file
233
services/worker/internal/backtest/flow_test.go
Normal file
|
|
@ -0,0 +1,233 @@
|
|||
package backtest_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
domainbacktest "git.toki-labs.com/toki/alt/packages/domain/backtest"
|
||||
"git.toki-labs.com/toki/alt/packages/domain/market"
|
||||
workerbacktest "git.toki-labs.com/toki/alt/services/worker/internal/backtest"
|
||||
"git.toki-labs.com/toki/alt/services/worker/internal/jobs"
|
||||
"git.toki-labs.com/toki/alt/services/worker/internal/storage"
|
||||
)
|
||||
|
||||
// flowStore is an in-memory store that satisfies the instrument, bar, run, and
|
||||
// result ports the backtest command flow touches. It records every run status
|
||||
// upsert so the test can assert the pending -> running -> succeeded transition.
|
||||
type flowStore struct {
|
||||
mu sync.Mutex
|
||||
instruments map[market.InstrumentID]market.Instrument
|
||||
bars map[market.InstrumentID][]market.Bar
|
||||
runs map[domainbacktest.RunID]domainbacktest.Run
|
||||
results map[domainbacktest.RunID]domainbacktest.Result
|
||||
runStatusLog []domainbacktest.RunStatus
|
||||
}
|
||||
|
||||
func newFlowStore() *flowStore {
|
||||
return &flowStore{
|
||||
instruments: make(map[market.InstrumentID]market.Instrument),
|
||||
bars: make(map[market.InstrumentID][]market.Bar),
|
||||
runs: make(map[domainbacktest.RunID]domainbacktest.Run),
|
||||
results: make(map[domainbacktest.RunID]domainbacktest.Result),
|
||||
}
|
||||
}
|
||||
|
||||
func (s *flowStore) UpsertInstrument(ctx context.Context, inst market.Instrument) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
s.instruments[inst.ID] = inst
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *flowStore) GetInstrument(ctx context.Context, id market.InstrumentID) (market.Instrument, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
inst, ok := s.instruments[id]
|
||||
if !ok {
|
||||
return market.Instrument{}, storage.ErrInstrumentNotFound
|
||||
}
|
||||
return inst, nil
|
||||
}
|
||||
|
||||
func (s *flowStore) ListInstruments(ctx context.Context) ([]market.Instrument, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
out := make([]market.Instrument, 0, len(s.instruments))
|
||||
for _, inst := range s.instruments {
|
||||
out = append(out, inst)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (s *flowStore) UpsertBar(ctx context.Context, bar market.Bar) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
s.bars[bar.InstrumentID] = append(s.bars[bar.InstrumentID], bar)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *flowStore) GetBars(ctx context.Context, id market.InstrumentID, timeframe market.Timeframe, from, to time.Time) ([]market.Bar, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
var out []market.Bar
|
||||
for _, bar := range s.bars[id] {
|
||||
if bar.Timeframe != timeframe {
|
||||
continue
|
||||
}
|
||||
if bar.Timestamp.Before(from) || bar.Timestamp.After(to) {
|
||||
continue
|
||||
}
|
||||
out = append(out, bar)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (s *flowStore) UpsertRun(ctx context.Context, run domainbacktest.Run) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
s.runs[run.ID] = run
|
||||
s.runStatusLog = append(s.runStatusLog, run.Status)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *flowStore) GetRun(ctx context.Context, id domainbacktest.RunID) (domainbacktest.Run, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
run, ok := s.runs[id]
|
||||
if !ok {
|
||||
return domainbacktest.Run{}, storage.ErrRunNotFound
|
||||
}
|
||||
return run, nil
|
||||
}
|
||||
|
||||
func (s *flowStore) UpsertResult(ctx context.Context, result domainbacktest.Result) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
s.results[result.RunID] = result
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *flowStore) GetResult(ctx context.Context, id domainbacktest.RunID) (domainbacktest.Result, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
result, ok := s.results[id]
|
||||
if !ok {
|
||||
return domainbacktest.Result{}, storage.ErrResultNotFound
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (s *flowStore) statuses() []domainbacktest.RunStatus {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
out := make([]domainbacktest.RunStatus, len(s.runStatusLog))
|
||||
copy(out, s.runStatusLog)
|
||||
return out
|
||||
}
|
||||
|
||||
// TestBacktestCommandFlowFromImportedBars proves the command-first rail end to
|
||||
// end with storage-backed bars: a started run records a pending run, the runner
|
||||
// executes it through the bundled strategy resolver against imported daily bars,
|
||||
// the run reaches terminal succeeded, and the result carries summary fields the
|
||||
// CLI result step reads. It runs the launch synchronously so no Docker or
|
||||
// goroutine race is involved.
|
||||
func TestBacktestCommandFlowFromImportedBars(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
store := newFlowStore()
|
||||
|
||||
instID := market.InstrumentID("KRX:005930")
|
||||
if err := store.UpsertInstrument(ctx, market.Instrument{ID: instID, Market: market.MarketKR}); err != nil {
|
||||
t.Fatalf("upsert instrument: %v", err)
|
||||
}
|
||||
// Imported daily bars: a 3-day window the run spec covers.
|
||||
for _, b := range []market.Bar{
|
||||
{InstrumentID: instID, Timeframe: market.TimeframeDaily, Timestamp: time.Date(2026, 5, 1, 0, 0, 0, 0, time.UTC), Close: market.Price{Currency: market.CurrencyKRW, Amount: market.Decimal{Value: "1000"}}},
|
||||
{InstrumentID: instID, Timeframe: market.TimeframeDaily, Timestamp: time.Date(2026, 5, 2, 0, 0, 0, 0, time.UTC), Close: market.Price{Currency: market.CurrencyKRW, Amount: market.Decimal{Value: "1100"}}},
|
||||
{InstrumentID: instID, Timeframe: market.TimeframeDaily, Timestamp: time.Date(2026, 5, 3, 0, 0, 0, 0, time.UTC), Close: market.Price{Currency: market.CurrencyKRW, Amount: market.Decimal{Value: "1200"}}},
|
||||
} {
|
||||
if err := store.UpsertBar(ctx, b); err != nil {
|
||||
t.Fatalf("upsert bar: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
engine := workerbacktest.NewEngine(
|
||||
workerbacktest.NewStorageBarSource(store, store),
|
||||
workerbacktest.NewBuiltInStrategyPort(),
|
||||
store,
|
||||
)
|
||||
runner := jobs.NewRunner()
|
||||
jobs.RegisterRunBacktestHandler(runner, store, engine, time.Now)
|
||||
starter := jobs.NewBacktestStarter(runner, store,
|
||||
jobs.WithStarterIDGenerator(func() string { return "run-flow" }),
|
||||
jobs.WithStarterLaunch(func(fn func()) { fn() }), // synchronous for determinism
|
||||
)
|
||||
|
||||
spec := domainbacktest.RunSpec{
|
||||
StrategyID: workerbacktest.BuiltInStrategyID,
|
||||
Market: market.MarketKR,
|
||||
Timeframe: market.TimeframeDaily,
|
||||
From: time.Date(2026, 5, 1, 0, 0, 0, 0, time.UTC),
|
||||
To: time.Date(2026, 5, 3, 0, 0, 0, 0, time.UTC),
|
||||
}
|
||||
|
||||
run, err := starter.StartBacktest(ctx, spec)
|
||||
if err != nil {
|
||||
t.Fatalf("start backtest: %v", err)
|
||||
}
|
||||
if run.ID != "run-flow" {
|
||||
t.Fatalf("unexpected run id %q", run.ID)
|
||||
}
|
||||
|
||||
// Poll: the run reached terminal succeeded after synchronous execution.
|
||||
got, err := store.GetRun(ctx, run.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("get run: %v", err)
|
||||
}
|
||||
if got.Status != domainbacktest.RunStatusSucceeded {
|
||||
t.Fatalf("expected terminal status succeeded, got %q (statuses=%v)", got.Status, store.statuses())
|
||||
}
|
||||
|
||||
// The pending -> running -> succeeded transition is recorded.
|
||||
statuses := store.statuses()
|
||||
wantTransition := []domainbacktest.RunStatus{
|
||||
domainbacktest.RunStatusPending,
|
||||
domainbacktest.RunStatusRunning,
|
||||
domainbacktest.RunStatusSucceeded,
|
||||
}
|
||||
for _, want := range wantTransition {
|
||||
if !containsStatus(statuses, want) {
|
||||
t.Errorf("expected status %q in transition log %v", want, statuses)
|
||||
}
|
||||
}
|
||||
|
||||
// Result: summary fields the CLI result step reads are present and reflect the
|
||||
// buy-and-hold run over the imported bars.
|
||||
result, err := store.GetResult(ctx, run.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("get result: %v", err)
|
||||
}
|
||||
if result.RunID != run.ID {
|
||||
t.Errorf("result run id = %q, want %q", result.RunID, run.ID)
|
||||
}
|
||||
if result.Summary.StartingCash.Amount.Value != "10000000" {
|
||||
t.Errorf("summary starting cash = %q, want 10000000", result.Summary.StartingCash.Amount.Value)
|
||||
}
|
||||
// Buy 1 unit at 1000 on day 1, hold; ending equity = 9,999,000 cash + 1*1200.
|
||||
if result.Summary.EndingEquity.Amount.Value != "10000200" {
|
||||
t.Errorf("summary ending equity = %q, want 10000200", result.Summary.EndingEquity.Amount.Value)
|
||||
}
|
||||
if result.Summary.TradeCount != 1 {
|
||||
t.Errorf("summary trade count = %d, want 1", result.Summary.TradeCount)
|
||||
}
|
||||
}
|
||||
|
||||
func containsStatus(haystack []domainbacktest.RunStatus, needle domainbacktest.RunStatus) bool {
|
||||
for _, s := range haystack {
|
||||
if s == needle {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
58
services/worker/internal/backtest/strategies.go
Normal file
58
services/worker/internal/backtest/strategies.go
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
package backtest
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"git.toki-labs.com/toki/alt/packages/domain/backtest"
|
||||
"git.toki-labs.com/toki/alt/packages/domain/market"
|
||||
)
|
||||
|
||||
// BuiltInStrategyID is the identifier of the worker's default bundled strategy.
|
||||
// Command workflows and YAML scenarios reference it so a backtest can run
|
||||
// end-to-end on imported bars without an operator authoring a custom strategy.
|
||||
const BuiltInStrategyID backtest.StrategyID = "strategy-v1"
|
||||
|
||||
// BuiltInStrategyPort resolves the worker's bundled strategies. It is the
|
||||
// production StrategyPort wired in cmd/alt-worker. Unknown ids return an error
|
||||
// so a misconfigured run fails fast instead of silently producing an empty
|
||||
// result.
|
||||
type BuiltInStrategyPort struct{}
|
||||
|
||||
func NewBuiltInStrategyPort() *BuiltInStrategyPort {
|
||||
return &BuiltInStrategyPort{}
|
||||
}
|
||||
|
||||
func (p *BuiltInStrategyPort) GetStrategy(ctx context.Context, id backtest.StrategyID) (backtest.Strategy, error) {
|
||||
switch id {
|
||||
case BuiltInStrategyID:
|
||||
return buyAndHoldStrategy{id: id}, nil
|
||||
default:
|
||||
return nil, fmt.Errorf("unknown strategy %q", id)
|
||||
}
|
||||
}
|
||||
|
||||
// buyAndHoldStrategy buys a single unit of each instrument the first time it is
|
||||
// seen and then holds. It is intentionally minimal and deterministic so command
|
||||
// validation can assert a stable terminal result from imported daily bars.
|
||||
type buyAndHoldStrategy struct {
|
||||
id backtest.StrategyID
|
||||
}
|
||||
|
||||
func (s buyAndHoldStrategy) ID() backtest.StrategyID { return s.id }
|
||||
|
||||
func (s buyAndHoldStrategy) Decide(input backtest.StrategyInput) ([]backtest.OrderIntent, error) {
|
||||
// Hold once a position exists. The engine passes the pre-trade portfolio for
|
||||
// the current bar, so an existing position means this instrument was already
|
||||
// bought on an earlier bar.
|
||||
if _, held := input.Portfolio.Position(input.Bar.InstrumentID); held {
|
||||
return nil, nil
|
||||
}
|
||||
return []backtest.OrderIntent{
|
||||
{
|
||||
InstrumentID: input.Bar.InstrumentID,
|
||||
Side: backtest.OrderSideBuy,
|
||||
Quantity: market.Quantity{Amount: market.Decimal{Value: "1"}},
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
76
services/worker/internal/backtest/strategies_test.go
Normal file
76
services/worker/internal/backtest/strategies_test.go
Normal file
|
|
@ -0,0 +1,76 @@
|
|||
package backtest
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"git.toki-labs.com/toki/alt/packages/domain/backtest"
|
||||
"git.toki-labs.com/toki/alt/packages/domain/market"
|
||||
)
|
||||
|
||||
func TestBuiltInStrategyPortResolvesKnownStrategy(t *testing.T) {
|
||||
port := NewBuiltInStrategyPort()
|
||||
strat, err := port.GetStrategy(context.Background(), BuiltInStrategyID)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error resolving built-in strategy: %v", err)
|
||||
}
|
||||
if strat.ID() != BuiltInStrategyID {
|
||||
t.Errorf("strategy id = %q, want %q", strat.ID(), BuiltInStrategyID)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuiltInStrategyPortRejectsUnknownStrategy(t *testing.T) {
|
||||
port := NewBuiltInStrategyPort()
|
||||
if _, err := port.GetStrategy(context.Background(), "does-not-exist"); err == nil {
|
||||
t.Fatal("expected error for unknown strategy id, got nil")
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuyAndHoldBuysOnceThenHolds(t *testing.T) {
|
||||
port := NewBuiltInStrategyPort()
|
||||
strat, err := port.GetStrategy(context.Background(), BuiltInStrategyID)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
|
||||
instID := market.InstrumentID("KRX:005930")
|
||||
cash := market.Price{Currency: market.CurrencyKRW, Amount: market.Decimal{Value: "1000000"}}
|
||||
bar := market.Bar{
|
||||
InstrumentID: instID,
|
||||
Timeframe: market.TimeframeDaily,
|
||||
Close: market.Price{Currency: market.CurrencyKRW, Amount: market.Decimal{Value: "1000"}},
|
||||
}
|
||||
|
||||
// No position yet: the strategy buys a single unit.
|
||||
orders, err := strat.Decide(backtest.StrategyInput{Bar: bar, Portfolio: backtest.NewPortfolioState(cash)})
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error on first decide: %v", err)
|
||||
}
|
||||
if len(orders) != 1 {
|
||||
t.Fatalf("expected 1 buy order on first bar, got %d", len(orders))
|
||||
}
|
||||
if orders[0].Side != backtest.OrderSideBuy || orders[0].InstrumentID != instID {
|
||||
t.Errorf("unexpected order %+v", orders[0])
|
||||
}
|
||||
if orders[0].Quantity.Amount.Value != "1" {
|
||||
t.Errorf("expected quantity 1, got %s", orders[0].Quantity.Amount.Value)
|
||||
}
|
||||
|
||||
// Apply the buy and decide again: with a position held, the strategy holds.
|
||||
portfolio, err := backtest.NewPortfolioState(cash).ApplyFill(backtest.Fill{
|
||||
InstrumentID: instID,
|
||||
Side: backtest.OrderSideBuy,
|
||||
Quantity: market.Quantity{Amount: market.Decimal{Value: "1"}},
|
||||
Price: bar.Close,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("failed to apply fill: %v", err)
|
||||
}
|
||||
held, err := strat.Decide(backtest.StrategyInput{Bar: bar, Portfolio: portfolio})
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error on hold decide: %v", err)
|
||||
}
|
||||
if len(held) != 0 {
|
||||
t.Errorf("expected no orders while holding, got %d", len(held))
|
||||
}
|
||||
}
|
||||
|
|
@ -71,6 +71,17 @@ func backtestHandlers(deps Deps) []sessionHandler {
|
|||
)
|
||||
},
|
||||
},
|
||||
{
|
||||
requestType: protoSocket.TypeNameOf(&altv1.GetBacktestRunRequest{}),
|
||||
register: func(client *protoSocket.WsClient) {
|
||||
protoSocket.AddRequestListenerTyped[*altv1.GetBacktestRunRequest, *altv1.GetBacktestRunResponse](
|
||||
&client.Communicator,
|
||||
func(req *altv1.GetBacktestRunRequest) (*altv1.GetBacktestRunResponse, error) {
|
||||
return handleGetBacktestRun(deps, req)
|
||||
},
|
||||
)
|
||||
},
|
||||
},
|
||||
{
|
||||
requestType: protoSocket.TypeNameOf(&altv1.ListBacktestRunsRequest{}),
|
||||
register: func(client *protoSocket.WsClient) {
|
||||
|
|
@ -137,6 +148,27 @@ func handleStartBacktest(deps Deps, req *altv1.StartBacktestRequest) (*altv1.Sta
|
|||
return &altv1.StartBacktestResponse{Run: runToProto(run)}, nil
|
||||
}
|
||||
|
||||
// handleGetBacktestRun answers a single-run poll. It reads through the analysis
|
||||
// store's run detail so polling reuses the same store-backed read path as
|
||||
// list/detail instead of growing a second run-store dependency on Deps.
|
||||
func handleGetBacktestRun(deps Deps, req *altv1.GetBacktestRunRequest) (*altv1.GetBacktestRunResponse, error) {
|
||||
if req.GetRunId() == "" {
|
||||
return &altv1.GetBacktestRunResponse{Error: invalidRequest("run_id is required")}, nil
|
||||
}
|
||||
if deps.Analysis == nil {
|
||||
return &altv1.GetBacktestRunResponse{Error: unavailableError("backtest analysis is not available")}, nil
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), handlerTimeout)
|
||||
defer cancel()
|
||||
|
||||
detail, err := deps.Analysis.GetRunDetail(ctx, backtest.RunID(req.GetRunId()))
|
||||
if err != nil {
|
||||
return &altv1.GetBacktestRunResponse{Error: backendErrorInfo(err)}, nil
|
||||
}
|
||||
return &altv1.GetBacktestRunResponse{Run: runToProto(detail.Run)}, nil
|
||||
}
|
||||
|
||||
func handleListBacktestRuns(deps Deps, req *altv1.ListBacktestRunsRequest) (*altv1.ListBacktestRunsResponse, error) {
|
||||
if deps.Analysis == nil {
|
||||
return &altv1.ListBacktestRunsResponse{Error: unavailableError("backtest analysis is not available")}, nil
|
||||
|
|
|
|||
|
|
@ -155,6 +155,52 @@ func TestHandleListBacktestRunsEmpty(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
func TestHandleGetBacktestRunSuccess(t *testing.T) {
|
||||
store := &fakeAnalysisStore{detail: storage.RunDetail{
|
||||
Run: backtest.Run{ID: "run-1", Status: backtest.RunStatusSucceeded},
|
||||
}}
|
||||
deps := BacktestDeps{Analysis: store}
|
||||
|
||||
resp, err := handleGetBacktestRun(deps, &altv1.GetBacktestRunRequest{RunId: "run-1"})
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if resp.GetRun().GetId() != "run-1" {
|
||||
t.Errorf("run id mismatch: %q", resp.GetRun().GetId())
|
||||
}
|
||||
if resp.GetRun().GetStatus() != altv1.BacktestRunStatus_BACKTEST_RUN_STATUS_SUCCEEDED {
|
||||
t.Errorf("expected succeeded status, got %v", resp.GetRun().GetStatus())
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleGetBacktestRunNotFound(t *testing.T) {
|
||||
store := &fakeAnalysisStore{err: storage.ErrRunNotFound}
|
||||
deps := BacktestDeps{Analysis: store}
|
||||
|
||||
resp, err := handleGetBacktestRun(deps, &altv1.GetBacktestRunRequest{RunId: "missing"})
|
||||
if err != nil {
|
||||
t.Fatalf("expected typed not_found response, got error: %v", err)
|
||||
}
|
||||
requireBacktestError(t, resp.GetError(), backtestErrorNotFound)
|
||||
}
|
||||
|
||||
func TestHandleGetBacktestRunRequiresRunID(t *testing.T) {
|
||||
deps := BacktestDeps{Analysis: &fakeAnalysisStore{}}
|
||||
resp, err := handleGetBacktestRun(deps, &altv1.GetBacktestRunRequest{})
|
||||
if err != nil {
|
||||
t.Fatalf("expected typed validation response, got error: %v", err)
|
||||
}
|
||||
requireBacktestError(t, resp.GetError(), backtestErrorInvalidRequest)
|
||||
}
|
||||
|
||||
func TestHandleGetBacktestRunUnavailable(t *testing.T) {
|
||||
resp, err := handleGetBacktestRun(BacktestDeps{}, &altv1.GetBacktestRunRequest{RunId: "run-1"})
|
||||
if err != nil {
|
||||
t.Fatalf("expected typed unavailable response, got error: %v", err)
|
||||
}
|
||||
requireBacktestError(t, resp.GetError(), backtestErrorUnavailable)
|
||||
}
|
||||
|
||||
func TestHandleGetBacktestRunDetailWithResult(t *testing.T) {
|
||||
store := &fakeAnalysisStore{detail: storage.RunDetail{
|
||||
Run: backtest.Run{ID: "run-1", Status: backtest.RunStatusSucceeded},
|
||||
|
|
@ -288,6 +334,7 @@ func TestBacktestHandlersCoverAllRequests(t *testing.T) {
|
|||
|
||||
required := []string{
|
||||
"alt.v1.StartBacktestRequest",
|
||||
"alt.v1.GetBacktestRunRequest",
|
||||
"alt.v1.ListBacktestRunsRequest",
|
||||
"alt.v1.GetBacktestRunDetailRequest",
|
||||
"alt.v1.GetBacktestResultRequest",
|
||||
|
|
|
|||
Loading…
Reference in a new issue