Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/theonetrade/backtest-kit-redis-mongo-docker/llms.txt

Use this file to discover all available pages before exploring further.

Every persistent data domain in backtest-kit—signals, candles, risk positions, schedules, and eleven others—maps to a dedicated MongoDB collection managed by a typed DB service. Each service extends BaseCRUD, which provides a consistent set of Mongoose-backed query methods. Adapters registered with backtest-kit delegate all reads and writes to these services, keeping strategy code entirely decoupled from the database. The upsert pattern (findOneAndUpdate with { upsert: true, new: true }) is applied consistently across all 15 domains to guarantee read-after-write consistency in a single round-trip.

​
The IPersist* Adapter Contract

backtest-kit exposes one IPersist*Instance interface per data domain. Each interface requires at minimum:
  • waitForInit(initial: boolean) — called before the first I/O; used to block on infrastructure readiness
  • readXData(when?: Date) — fetch the current persisted value for this adapter’s context
  • writeXData(value, when: Date) — durably write a new value; must be visible to subsequent reads
Adapters are registered at application startup via the static PersistXAdapter.usePersistXAdapter(...) factory. The adapter constructor receives the context tuple (e.g., symbol, strategyName, exchangeName) that uniquely identifies one row in the underlying MongoDB collection.

​
BaseCRUD: Foundation for All DB Services

All 15 DB services inherit from BaseCRUD, a dependency-injection factory class that wraps a Mongoose Model with a standard query API: 
const FIND_ALL_LIMIT = 1_000;

export const BaseCRUD = factory(
  class {
    readonly loggerService = inject<LoggerService>(TYPES.loggerService);
    constructor(public readonly TargetModel: Model<any>) {}

    public async create(dto: object) { ... }
    public async update(id: string, dto: object) { ... }
    public async findById(id: string) { ... }
    public async findByFilter(filterData: object, sort?: object) { ... }
    public async findAll(filterData: object = {}, limit = FIND_ALL_LIMIT) { ... }
    public async *iterate(filterData: object = {}, sort?: object) { ... }
    public async paginate(
      filterData: object,
      pagination: { limit: number; offset: number },
      sort?: object
    ) { ... }
  }
);
MethodDescription
createInsert a new document
updatePatch an existing document by _id
findByIdPrimary-key lookup; O(log n) on the _id index
findByFilterCompound-field query; used as the cache-miss fallback
findAllRetrieve up to FIND_ALL_LIMIT (1 000) documents matching a filter
iterateAsync generator that streams documents in batches
paginateOffset/limit pagination with optional sort

​
The Upsert Pattern

All mutable DB services use findOneAndUpdate with an upsert to combine create and update into a single atomic operation: 
public upsert = async (
  symbol: string,
  strategyName: string,
  exchangeName: string,
  payload: ISignalRow | null,
) => {
  const filter = { symbol, strategyName, exchangeName };
  const document = await SignalModel.findOneAndUpdate(
    filter,
    { $set: { payload } },
    { upsert: true, new: true, setDefaultsOnInsert: true },
  );
  const result = readTransform(document.toJSON()) as unknown as ISignalRowDoc;
  await this.signalCacheService.setSignalId(result); // always update cache after write
};
The three Mongoose options work together:
  • upsert: true — creates the document if no match is found, using the filter fields as the initial key set
  • new: true — returns the document after the write, not the pre-write snapshot; this is what makes the read-after-write guarantee possible
  • setDefaultsOnInsert: true — applies Mongoose schema defaults to newly inserted documents only, leaving existing documents unchanged
Because new: true is set on every upsert, the returned document always reflects the state that was just written. Any subsequent readXData() call on the same adapter context will see the same value.

​
Insert-Only Pattern: Candle Immutability

Candle (OHLCV) data is immutable once recorded—a candle for a given symbol, interval, and timestamp should never be overwritten by later ingestion. CandleDbService enforces this with $setOnInsert instead of $set: 
public create = async (dto: ICandleDto): Promise<ICandleRow> => {
  const filter = {
    symbol: dto.symbol,
    interval: dto.interval,
    timestamp: dto.timestamp,
  };
  const insertOnly = {
    exchangeName: EXCHANGE_NAME,
    open, high, low, close, volume,
  };
  const document = await CandleModel.findOneAndUpdate(
    filter,
    { $setOnInsert: insertOnly }, // only applied when a new document is inserted
    { upsert: true, new: true, setDefaultsOnInsert: true },
  );
  const result = readTransform(document.toJSON()) as unknown as ICandleRow;
  await this.candleCacheService.setCandleId(result);
  return result;
};
$setOnInsert is a no-op when a matching document already exists. Combined with the filter on { symbol, interval, timestamp }, this makes candle ingestion fully idempotent—running the same candle through the pipeline twice produces exactly one stored document.
Any attempt to update an existing candle’s OHLCV fields via create will silently succeed but leave the stored data unchanged. Use a direct update call (from BaseCRUD) if you need to correct historical data intentionally.

​
Infrastructure Initialization Gate

Before any adapter issues its first database call, it must wait for both MongoDB and Redis to complete their connection handshakes. The waitForInfra helper uses singleshot to ensure this initialization runs exactly once regardless of concurrent adapter construction: 
const waitForInfra = singleshot(
  async () => {
    await Promise.all([
      ioc.mongoService.waitForInit(),
      ioc.redisService.waitForInit(),
    ]);
  }
);
Every adapter implementation calls this inside waitForInit(initial), guarding on the initial flag so infrastructure initialization is only attempted on the very first invocation: 
PersistSignalAdapter.usePersistSignalAdapter(
  class implements IPersistSignalInstance {
    constructor(
      readonly symbol: string,
      readonly strategyName: string,
      readonly exchangeName: string,
    ) {}

    async waitForInit(initial: boolean) {
      if (!initial) return;          // skip on subsequent calls
      await waitForInfra();          // blocks until Mongo + Redis are ready
    }

    async readSignalData(): Promise<ISignalRow | null> {
      const row = await ioc.signalDbService.findByContext(
        this.symbol,
        this.strategyName,
        this.exchangeName,
      );
      return row ? row.payload : null;
    }

    async writeSignalData(signalRow: ISignalRow | null): Promise<void> {
      await ioc.signalDbService.upsert(
        this.symbol,
        this.strategyName,
        this.exchangeName,
        signalRow,
      );
    }
  }
);
The if (!initial) return guard is critical. backtest-kit may call waitForInit multiple times during the lifecycle of a long-running backtest. Skipping subsequent calls avoids redundant Promise.all re-evaluations—though singleshot would handle deduplication anyway.

​
The readTransform Utility

Every DB service applies readTransform to raw Mongoose document output before returning it to callers or writing to the cache. This utility normalizes the document shape by spreading all existing fields and adding an id string property derived from _id, so that all upstream code can reference the primary key as id instead of the raw _id ObjectId. 
const result = readTransform(document.toJSON()) as unknown as ISignalRowDoc;
toJSON() is called before readTransform to serialize the Mongoose document into a plain JavaScript object. This step is required; passing a Mongoose document directly would carry prototype methods and internal state that readTransform does not expect.

​
15 MongoDB Collections at a Glance

Each DB service owns one collection. The table below maps each service to its compound key fields—the fields used in the findOneAndUpdate filter that form the logical primary key for upsert operations.
DB ServiceCollectionCompound Key Fields
candleDbServicecandlessymbol, interval, timestamp
signalDbServicesignalssymbol, strategyName, exchangeName
scheduleDbServiceschedulessymbol, strategyName, exchangeName
riskDbServicerisksriskName, exchangeName
partialDbServicepartialssymbol, strategyName, exchangeName, signalId
breakevenDbServicebreakevenssymbol, strategyName, exchangeName, signalId
storageDbServicestoragesbacktest, signalId
notificationDbServicenotificationsbacktest, notificationId
logDbServicelogsentryId
measureDbServicemeasuresbucket, entryKey
intervalDbServiceintervalsbucket, entryKey
memoryDbServicememoriessignalId, bucketName, memoryId
recentDbServicerecentssymbol, strategyName, exchangeName, frameName, backtest
stateDbServicestatessignalId, bucketName
sessionDbServicesessionsstrategyName, exchangeName, frameName

Build docs developers (and LLMs) love

Get started for freeTalk to us