backtest-kit MinIO S3 Docker replaces the defaultDocumentation Index
Fetch the complete documentation index at: https://mintlify.com/theonetrade/backtest-kit-minio-s3-docker/llms.txt
Use this file to discover all available pages before exploring further.
./dump/ file-based persistence with a two-tier infrastructure layer: MinIO serves as the S3-compatible source of truth for every JSON record, and Redis provides a per-minute SET index that makes newest-first listings O(limit) instead of O(bucket). Sixteen IPersist*Instance adapters wire this infrastructure into the standard backtest-kit persistence contract, meaning strategy code, runners, and the CLI entry point require zero changes — only the persistence layer is swapped.
No schema migrations, no ORM, no SQL: every record is a JSON object stored at a deterministic key path. MinIO’s S3 semantics (versioning, mc mirror, lifecycle policies) handle operational durability; Redis handles time-ordered read efficiency.
Three-Layer Model
| Layer | What it does | Where it lives |
|---|---|---|
| backtest-kit IPersist contract | Defines readXData / writeXData interfaces that strategy code calls | backtest-kit npm package |
| 16 IPersist adapters | Implement each interface; delegate to the appropriate data service | src/config/setup.ts |
| MinIO + Redis | MinIO stores every JSON object; Redis indexes Log, Notification, and Storage entries for newest-first listing | Docker Compose services |
src/config/setup.ts:
src/config/setup.ts
The 16 Persist Adapters
All adapters share a single MinIO bucket namedbacktest-kit. Each entity occupies its own root folder within that bucket.
| Adapter | Folder | Object key pattern | Purpose |
|---|---|---|---|
| Candle | candle-items/ | exchange/symbol/interval/timestamp | OHLCV cache; immutable inserts |
| Signal | signal-items/ | symbol/strategy/exchange | Live signal state per context |
| Schedule | schedule-items/ | symbol/strategy/exchange | Pending scheduled signal |
| Strategy | strategy-items/ | symbol/strategy/exchange | Deferred commit queue snapshot |
| Risk | risk-items/ | riskName/exchange | Active risk positions snapshot |
| Partial | partial-items/ | symbol/strategy/exchange/signalId | Partial profit/loss levels per signal |
| Breakeven | breakeven-items/ | symbol/strategy/exchange/signalId | Breakeven reached flag |
| Storage | storage-items/ | backtest/signalId | Closed/opened signal log per mode † |
| Notification | notification-items/ | backtest/⟲ts_notificationId | Event notifications † |
| Log | log-items/ | ⟲ts_entryId | Strategy log entries † |
| Measure | measure-items/ | bucket/entryKey | LLM/API response cache |
| Interval | interval-items/ | bucket/entryKey | Once-per-interval markers |
| Memory | memory-items/ | signalId/bucket/memoryId | Per-signal memory store |
| Recent | recent-items/ | symbol/strategy/exchange/frame/backtest | Last public signal per context |
| State | state-items/ | signalId/bucket | Per-signal state buckets |
| Session | session-items/ | strategy/exchange/frame/symbol/backtest | One session per running strategy |
† Log, Notification, and Storage entries are also registered in the Redis time index on every write. All other adapters read directly from MinIO with no Redis involvement.
Single-Bucket Model and Key Prefixes
Everything lives in one physical MinIO bucket calledbacktest-kit. BaseStorage is constructed with a name that encodes both the bucket and the entity folder:
BaseStorage.ts
BaseStorage("backtest-kit/candle-items") resolves to:
- Physical bucket:
backtest-kit - Root key prefix:
candle-items/ - Full object path:
backtest-kit/candle-items/ccxt_binance/BTCUSDT/1h/1700000000000
BaseStorage("breakeven-items") — still means a dedicated bucket named breakeven-items with no prefix, which keeps the implementation fully backward compatible with single-bucket setups.
Inverted Timestamp Keys
For entities that need newest-first ordering (Log, Notification), the object key embeds an inverted timestamp instead of a raw Unix millisecond value:LogDataService.ts
Number.MAX_SAFE_INTEGER (9007199254740991) minus a recent Unix millisecond produces a very large but steadily decreasing number, plain lexicographic S3 listing already returns objects newest-first — no sort pass required. This is the foundation of the cold-index fallback: even with Redis flushed, listObjectsV2 returns entries in the right order.
Explore Further
MinIO Storage
Bucket layout, BaseStorage API, auto-bucket creation, and candle insert-only semantics.
Redis Index
Per-minute SETs, backwards walk, cold-index fallback, and steady-state RTT cost.
Write Durability
How S3 read-after-write consistency and deterministic keys satisfy the backtest-kit write durability contract.
Adapters Overview
All 16 adapters, their registration pattern, and the
waitForInit gate.