Documentation Index
Fetch the complete documentation index at: https://mintlify.com/theonetrade/backtest-kit-minio-s3-docker/llms.txt
Use this file to discover all available pages before exploring further.
The Candle adapter stores OHLCV (open/high/low/close/volume) candlestick data as immutable JSON objects in MinIO. Each candle’s object key is a deterministic function of exchange, symbol, interval, and timestamp — meaning an upsert is always a single idempotent PUT with no read-before-write race. Because the market history for a given timestamp never changes, the adapter uses insert-only semantics: it performs a cheap stat check first, and skips the PUT entirely if the object already exists.
Interface
PersistCandleAdapter is registered with a class that implements IPersistCandleInstance:
PersistCandleAdapter.usePersistCandleAdapter(class implements IPersistCandleInstance {
constructor(
readonly symbol: string,
readonly interval: CandleInterval,
readonly exchangeName: string,
) {}
async waitForInit(initial: boolean): Promise<void> {
if (!initial) return;
await waitForInfra();
}
async writeCandlesData(candles: CandleData[]): Promise<void> {
for (const candle of candles) {
await ioc.candleDataService.create({
symbol: this.symbol,
interval: this.interval,
close: candle.close,
high: candle.high,
low: candle.low,
open: candle.open,
timestamp: candle.timestamp,
volume: candle.volume,
});
}
}
async readCandlesData(limit: number, sinceTimestamp: number): Promise<CandleData[] | null> {
const stepMs = INTERVAL_MINUTES[this.interval] * MS_PER_MINUTE;
const result: CandleData[] = [];
for (let i = 0; i < limit; i++) {
const ts = sinceTimestamp + i * stepMs;
const row = await ioc.candleDataService.findBySymbolIntervalTimestamp(
this.symbol, this.interval, ts
);
if (!row) return null;
result.push({
timestamp: row.timestamp,
open: row.open,
high: row.high,
low: row.low,
close: row.close,
volume: row.volume,
});
}
return result;
}
});
| Method | Signature | Description |
|---|
waitForInit | (initial: boolean): Promise<void> | Gates first-touch on Redis readiness via waitForInfra() |
writeCandlesData | (candles: CandleData[]): Promise<void> | Bulk insert-or-skip; iterates the array and calls create() for each |
readCandlesData | (limit: number, sinceTimestamp: number): Promise<CandleData[] | null> | Fetches limit consecutive candles starting at sinceTimestamp; returns null if any candle is missing |
Constructor Context
backtest-kit passes three fields to the constructor when instantiating the adapter for a given context:
constructor(
readonly symbol: string, // e.g. "BTCUSDT"
readonly interval: CandleInterval, // e.g. "1h"
readonly exchangeName: string, // e.g. "ccxt_binance"
)
Valid CandleInterval values are: 1m, 3m, 5m, 15m, 30m, 1h, 2h, 4h, 6h, 8h, 1d.These are used both in the MinIO object key and in the INTERVAL_MINUTES lookup table to compute the millisecond step for sequential reads.
CandleDataService extends BaseStorage("backtest-kit/candle-items"), placing all objects inside the candle-items/ prefix of the backtest-kit bucket. The full object key is assembled by GET_STORAGE_KEY_FN:
src/lib/services/data/CandleDataService.ts
const EXCHANGE_NAME = "ccxt_binance";
const GET_STORAGE_KEY_FN = (
symbol: string,
interval: CandleInterval,
timestamp: number
) => {
return `${EXCHANGE_NAME}/${symbol}/${interval}/${timestamp}`;
};
BTCUSDT at the 1h interval with Unix timestamp 1700000000000 is stored at:
backtest-kit/candle-items/ccxt_binance/BTCUSDT/1h/1700000000000
Insert-Only Semantics
Because candlestick data for a completed bar is immutable, CandleDataService.create() never overwrites an existing object. It performs a stat (HEAD request) first and short-circuits if the object already exists — avoiding a body download and a redundant PUT:
src/lib/services/data/CandleDataService.ts
public create = async (dto: ICandleDto): Promise<ICandleRow> => {
const key = GET_STORAGE_KEY_FN(dto.symbol, dto.interval, dto.timestamp);
const now = new Date();
const row: ICandleRow = {
id: key,
exchangeName: EXCHANGE_NAME,
symbol: dto.symbol,
interval: dto.interval,
timestamp: dto.timestamp,
open: dto.open,
high: dto.high,
low: dto.low,
close: dto.close,
volume: dto.volume,
createDate: now,
updatedDate: now,
};
// Candles are immutable: the row is fully determined by the dto, so an
// existence check (stat) replaces downloading the stored body.
if (await this.has(key)) {
return row;
}
await this.set(key, row);
return row;
};
This insert-only design satisfies backtest-kit’s write durability contract — after writeCandlesData returns, readCandlesData is guaranteed to see the written values. S3’s strong read-after-write consistency for single objects means no transactions are needed.
has() helper on BaseStorage maps to a minioClient.statObject() call, which is a lightweight HEAD request with no body transfer:
src/lib/common/BaseStorage.ts
async has(key: string): Promise<boolean> {
const minioClient = await this.minioService.getClient(this.bucketName);
try {
await minioClient.statObject(this.bucketName, this.rootPrefix + key);
return true;
} catch (error) {
if (isNotFound(error)) return false;
throw error;
}
}
readCandlesData
readCandlesData reconstructs a contiguous time series by computing each expected timestamp arithmetically — no range queries or index scans are needed. Starting from sinceTimestamp, it advances by one interval step per iteration:
async readCandlesData(limit: number, sinceTimestamp: number) {
const stepMs = INTERVAL_MINUTES[this.interval] * MS_PER_MINUTE;
const result: CandleData[] = [];
for (let i = 0; i < limit; i++) {
const ts = sinceTimestamp + i * stepMs;
const row = await ioc.candleDataService.findBySymbolIntervalTimestamp(
this.symbol, this.interval, ts
);
if (!row) {
return null; // gap in data — signal caller to fetch from exchange
}
result.push({
timestamp: row.timestamp,
open: row.open,
high: row.high,
low: row.low,
close: row.close,
volume: row.volume,
});
}
return result;
}
CandleInterval:
| Interval | Minutes | Step (ms) |
|---|
1m | 1 | 60,000 |
3m | 3 | 180,000 |
5m | 5 | 300,000 |
15m | 15 | 900,000 |
30m | 30 | 1,800,000 |
1h | 60 | 3,600,000 |
2h | 120 | 7,200,000 |
4h | 240 | 14,400,000 |
6h | 360 | 21,600,000 |
8h | 480 | 28,800,000 |
1d | 1440 | 86,400,000 |
If any candle in the requested range is missing from MinIO, readCandlesData returns null (not a partial array). The caller — typically the backtest-kit warm-candles utility — interprets null as a signal to fetch the missing bars from the exchange and write them via writeCandlesData.
Schema
The Candle adapter operates on two TypeScript interfaces defined in src/schema/Candle.schema.ts:
src/schema/Candle.schema.ts
import { CandleInterval } from "backtest-kit";
/** Input shape passed to CandleDataService.create() */
interface ICandleDto {
symbol: string;
interval: CandleInterval;
timestamp: number;
open: number;
high: number;
low: number;
close: number;
volume: number;
}
/** Persisted shape stored as JSON in MinIO */
interface ICandleRow extends ICandleDto {
id: string; // deterministic key: exchange/symbol/interval/timestamp
exchangeName: string; // always "ccxt_binance" for this adapter
createDate: Date;
updatedDate: Date;
}
export { ICandleDto, ICandleRow };
ICandleDto carries only the market data fields. ICandleRow adds three server-assigned fields (id, exchangeName, createDate/updatedDate) that are computed at write time rather than supplied by the caller.
createDate and updatedDate are set to new Date() at the moment create() runs. Because candles are immutable and the has() guard prevents re-writes, updatedDate will always equal createDate in practice — it exists for schema consistency with other entity types.