Measure, Interval, and Memory adapters with soft-delete

Measure, Interval, and Memory are the three key-value cache adapters with soft-delete semantics. Unlike the Storage, Notification, and Log adapters, they have no Redis time index — newest-first ordering is not a requirement for these entities. Instead of writing tombstone objects on removal, they physically delete the S3 object, which means listKeys is a pure prefix LIST with zero body reads. There is never a removed object to skip, and the list result is always the complete set of live entries.

Use-case summary:

Measure — cache LLM or external API responses to avoid repeated calls on the same input.
Interval — store once-per-interval markers, e.g. “run this logic only once per trading day”.
Memory — per-signal scratchpad persisted across process restarts, enabling stateful signal behaviour without coupling to the signal row itself.

Measure adapter

Purpose

The Measure adapter caches LLM or external API responses keyed by an arbitrary string within a named bucket. A typical pattern is to compute a cache key from the prompt or API parameters, call readMeasureData first, and only call the external service if the result is null. The cached response is then written back with writeMeasureData.

Constructor

PersistMeasureAdapter.usePersistMeasureAdapter(
  class implements IPersistMeasureInstance {
    constructor(readonly bucket: string) {}
    // ...
  }
);

bucket is a logical namespace (e.g. "openai-summaries", "coingecko-prices"). Multiple strategy instances can share a bucket to deduplicate identical requests across signals.

Object key pattern

measure-items/  <bucket>/<entryKey>
                 openai-summaries/btc-analysis-2024-06-01

The key is a pure function of (bucket, entryKey) — no timestamp component. An upsert is a single idempotent PUT.

Interface methods

Method	Signature	Description
`waitForInit`	`(initial: boolean) => Promise<void>`	Gates first-touch on Redis readiness.
`readMeasureData`	`(key: string) => Promise<MeasureData \| null>`	Returns the cached payload, or `null` if the key has been removed or never written.
`writeMeasureData`	`(data: MeasureData, key: string, when: Date) => Promise<void>`	Writes or overwrites the cached payload.
`removeMeasureData`	`(key: string) => Promise<void>`	Physically deletes the object. Subsequent reads return `null`.
`listMeasureData`	`() => AsyncGenerator<string>`	Yields every live entry key in the bucket (prefix `LIST`, no body reads).

Soft-delete

removeMeasureData calls softRemove on MeasureDataService, which physically deletes the S3 object:

// From MeasureDataService.ts
public softRemove = async (bucket: string, entryKey: string): Promise<void> => {
  this.loggerService.log("measureDataService softRemove", { bucket, entryKey });
  await this.delete(GET_STORAGE_KEY_FN(bucket, entryKey));
};

If upsert is called with a payload that already has removed: true, it also triggers a physical delete instead of a PUT:

// From MeasureDataService.ts
public upsert = async (
  bucket: string,
  entryKey: string,
  payload: MeasureData
): Promise<void> => {
  const key = GET_STORAGE_KEY_FN(bucket, entryKey);
  // removed==true means logically absent: deleting the object keeps
  // listKeys a pure LIST without reading bodies
  if (payload.removed) {
    await this.delete(key);
    return;
  }
  const now = new Date();
  const row: IMeasureRow = {
    id: key, bucket, entryKey, payload,
    removed: false, createDate: now, updatedDate: now,
  };
  await this.set(key, row);
};

Adapter registration

// From setup.ts
PersistMeasureAdapter.usePersistMeasureAdapter(class implements IPersistMeasureInstance {
  constructor(readonly bucket: string) {}
  async waitForInit(initial: boolean) {
    if (!initial) return;
    await waitForInfra();
  }
  async readMeasureData(key: string): Promise<MeasureData | null> {
    const row = await ioc.measureDataService.findByKey(this.bucket, key);
    if (!row || row.removed) return null;
    return row.payload;
  }
  async writeMeasureData(data: MeasureData, key: string, _when: Date): Promise<void> {
    await ioc.measureDataService.upsert(this.bucket, key, data);
  }
  async removeMeasureData(key: string): Promise<void> {
    await ioc.measureDataService.softRemove(this.bucket, key);
  }
  async *listMeasureData(): AsyncGenerator<string> {
    const keys = await ioc.measureDataService.listKeys(this.bucket);
    for (const key of keys) {
      yield key;
    }
  }
});

listMeasureData is an async generator that yields keys, not values. To read the payload for a given key, call readMeasureData(key) on each yielded string.

Interval adapter

Purpose

The Interval adapter stores once-per-interval markers — lightweight flags that record whether a particular action has already been taken within the current interval (e.g., a day, a week). Strategy code reads the marker, acts if absent, and writes it back so the action is skipped on the next tick within the same interval.

Constructor

PersistIntervalAdapter.usePersistIntervalAdapter(
  class implements IPersistIntervalInstance {
    constructor(readonly bucket: string) {}
    // ...
  }
);

bucket namespaces the markers for a given strategy context (e.g., "daily-rebalance-BTCUSDT").

Object key pattern

interval-items/  <bucket>/<entryKey>
                  daily-rebalance-BTCUSDT/2024-06-01

Like Measure, the key is a pure (bucket, entryKey) function. entryKey is typically a date string or interval identifier derived at the strategy’s candle timestamp.

Interface methods

Method	Signature	Description
`waitForInit`	`(initial: boolean) => Promise<void>`	Gates first-touch on Redis readiness.
`readIntervalData`	`(key: string) => Promise<IntervalData \| null>`	Returns the marker payload, or `null` if absent or removed.
`writeIntervalData`	`(data: IntervalData, key: string, when: Date) => Promise<void>`	Writes or overwrites the marker. `when` is stored in the row but does not affect the key.
`removeIntervalData`	`(key: string) => Promise<void>`	Physically deletes the object.
`listIntervalData`	`() => AsyncGenerator<string>`	Yields every live entry key in the bucket.

Soft-delete

removeIntervalData calls softRemove, which physically deletes the object:

// From IntervalDataService.ts
public softRemove = async (bucket: string, entryKey: string): Promise<void> => {
  this.loggerService.log("intervalDataService softRemove", { bucket, entryKey });
  await this.delete(GET_STORAGE_KEY_FN(bucket, entryKey));
};

The upsert method also detects removed: true payloads and routes them to a physical delete:

// From IntervalDataService.ts
public upsert = async (
  bucket: string,
  entryKey: string,
  payload: IntervalData,
  when: Date
): Promise<void> => {
  const key = GET_STORAGE_KEY_FN(bucket, entryKey);
  // removed==true means logically absent: deleting the object keeps
  // listKeys a pure LIST without reading bodies
  if (payload.removed) {
    await this.delete(key);
    return;
  }
  const now = new Date();
  const row: IIntervalRow = {
    id: key, bucket, entryKey, payload,
    removed: false, when: when.getTime(),
    createDate: now, updatedDate: now,
  };
  await this.set(key, row);
};

Adapter registration

// From setup.ts
PersistIntervalAdapter.usePersistIntervalAdapter(class implements IPersistIntervalInstance {
  constructor(readonly bucket: string) {}
  async waitForInit(initial: boolean) {
    if (!initial) return;
    await waitForInfra();
  }
  async readIntervalData(key: string): Promise<IntervalData | null> {
    const row = await ioc.intervalDataService.findByKey(this.bucket, key);
    if (!row || row.removed) return null;
    return row.payload;
  }
  async writeIntervalData(data: IntervalData, key: string, when: Date): Promise<void> {
    await ioc.intervalDataService.upsert(this.bucket, key, data, when);
  }
  async removeIntervalData(key: string): Promise<void> {
    await ioc.intervalDataService.softRemove(this.bucket, key);
  }
  async *listIntervalData(): AsyncGenerator<string> {
    const keys = await ioc.intervalDataService.listKeys(this.bucket);
    for (const key of keys) {
      yield key;
    }
  }
});

listIntervalData is an async generator that yields keys, not values. Call readIntervalData(key) on each yielded string to fetch the marker payload.

Memory adapter

Purpose

The Memory adapter provides a per-signal scratchpad that survives process restarts. Each signal instance gets its own isolated namespace (signalId / bucketName) containing any number of named memory entries identified by memoryId. This enables stateful signal logic — such as maintaining a running list of observations, a rolling price buffer, or intermediate computation state — without polluting the signal row itself.

Constructor

PersistMemoryAdapter.usePersistMemoryAdapter(
  class implements IPersistMemoryInstance {
    constructor(
      readonly signalId: string,
      readonly bucketName: string,
    ) {}
    // ...
  }
);

Both signalId and bucketName are injected by backtest-kit from the active signal context.

Object key pattern

memory-items/  <signalId>/<bucketName>/<memoryId>
                sig-abc123/observations/entry-001
                sig-abc123/rolling-buffer/slot-0

The three-part key isolates entries by signal identity, logical bucket, and individual memory slot.

Interface methods

Method	Signature	Description
`waitForInit`	`(initial: boolean) => Promise<void>`	Gates first-touch on Redis readiness.
`readMemoryData`	`(memoryId: string) => Promise<MemoryData \| null>`	Returns the payload for the given memory slot, or `null` if absent or removed.
`hasMemoryData`	`(memoryId: string) => Promise<boolean>`	Returns `true` if the object exists in MinIO (stat only, no body download).
`writeMemoryData`	`(data: MemoryData, memoryId: string, when: Date) => Promise<void>`	Writes or overwrites the memory slot.
`removeMemoryData`	`(memoryId: string) => Promise<void>`	Physically deletes the object.
`listMemoryData`	`() => AsyncGenerator<{ memoryId: string; data: MemoryData }>`	Yields every live memory entry with its full payload for this signal.
`dispose`	`() => void`	No-op in this implementation (see tip below).

Soft-delete

removeMemoryData calls softRemove, which physically deletes the object:

// From MemoryDataService.ts
public softRemove = async (
  signalId: string,
  bucketName: string,
  memoryId: string,
): Promise<void> => {
  this.loggerService.log("memoryDataService softRemove", { signalId, bucketName, memoryId });
  await this.delete(GET_STORAGE_KEY_FN(signalId, bucketName, memoryId));
};

The upsert method also routes removed: true payloads directly to a physical delete:

// From MemoryDataService.ts
public upsert = async (
  signalId: string,
  bucketName: string,
  memoryId: string,
  payload: MemoryData,
  when: Date,
): Promise<void> => {
  const key = GET_STORAGE_KEY_FN(signalId, bucketName, memoryId);
  // removed==true means logically absent: deleting the object keeps
  // listEntries free of tombstone bodies
  if (payload.removed) {
    await this.delete(key);
    return;
  }
  const now = new Date();
  const row: IMemoryRow = {
    id: key, signalId, bucketName, memoryId, payload,
    removed: false, when: when.getTime(),
    createDate: now, updatedDate: now,
  };
  await this.set(key, row);
};

Adapter registration

// From setup.ts
PersistMemoryAdapter.usePersistMemoryAdapter(class implements IPersistMemoryInstance {
  constructor(
    readonly signalId: string,
    readonly bucketName: string,
  ) {}
  async waitForInit(initial: boolean) {
    if (!initial) return;
    await waitForInfra();
  }
  async readMemoryData(memoryId: string): Promise<MemoryData | null> {
    const row = await ioc.memoryDataService.findByMemoryId(
      this.signalId, this.bucketName, memoryId
    );
    if (!row || row.removed) return null;
    return row.payload;
  }
  async hasMemoryData(memoryId: string): Promise<boolean> {
    return await ioc.memoryDataService.hasMemoryEntry(
      this.signalId, this.bucketName, memoryId
    );
  }
  async writeMemoryData(data: MemoryData, memoryId: string, when: Date): Promise<void> {
    await ioc.memoryDataService.upsert(
      this.signalId, this.bucketName, memoryId, data, when
    );
  }
  async removeMemoryData(memoryId: string): Promise<void> {
    await ioc.memoryDataService.softRemove(
      this.signalId, this.bucketName, memoryId
    );
  }
  async *listMemoryData(): AsyncGenerator<{ memoryId: string; data: MemoryData }> {
    const rows = await ioc.memoryDataService.listEntries(
      this.signalId, this.bucketName
    );
    for (const row of rows) {
      yield { memoryId: row.memoryId, data: row.payload };
    }
  }
  dispose(): void { void 0; }
});

The dispose() method is a deliberate no-op (void 0). In this MinIO implementation, memory cleanup is entirely explicit — entries persist until removeMemoryData is called. There is no TTL, no background sweep, and no reference-counting. If your strategy needs to clean up on signal close, iterate listMemoryData and call removeMemoryData for each yielded memoryId before the signal exits.

listMemoryData reads full bodies

Unlike listMeasureData and listIntervalData — which are pure prefix LIST operations that yield only keys — listMemoryData downloads the body of every live object in the signalId/bucketName/ prefix. This is intentional: the Memory adapter is designed to support BM25 index rebuilds and other patterns that need all entry payloads at once. For large memory stores, prefer readMemoryData(memoryId) for targeted point reads over listMemoryData when only a subset of entries is needed.

Comparison: list behaviour across the three adapters

Adapter	`list*Data` yields	Body downloaded per item	Backed by
Measure	`string` (entryKey)	❌ No — prefix LIST only	`MeasureDataService.listKeys`
Interval	`string` (entryKey)	❌ No — prefix LIST only	`IntervalDataService.listKeys`
Memory	`{ memoryId, data }`	✅ Yes — full body per entry	`MemoryDataService.listEntries`

For Measure and Interval, fetching the payload requires an explicit readXData(key) call after iterating listXData.

listMeasureData and listIntervalData are async generators yielding keys only. To fetch the value for a key, call readMeasureData(key) or readIntervalData(key) respectively. Iterating the generator alone does not download any object bodies.

Get Started

Architecture

Configuration

Persist Adapters

Strategy Development

Measure, Interval, and Memory adapters with soft-delete

Measure adapter

Purpose

Constructor

Object key pattern

Interface methods

Soft-delete

Adapter registration

Interval adapter

Purpose

Constructor

Object key pattern

Interface methods

Soft-delete

Adapter registration

Memory adapter

Purpose

Constructor

Object key pattern

Interface methods

Soft-delete

Adapter registration

Comparison: list behaviour across the three adapters

Build docs developers (and LLMs) love

Get Started

Architecture

Configuration

Persist Adapters

Strategy Development

Documentation Index

​Measure adapter

​Purpose

​Constructor

​Object key pattern

​Interface methods

​Soft-delete

​Adapter registration

​Interval adapter

​Purpose

​Constructor

​Object key pattern

​Interface methods

​Soft-delete

​Adapter registration

​Memory adapter

​Purpose

​Constructor

​Object key pattern

​Interface methods

​Soft-delete

​Adapter registration

​Comparison: list behaviour across the three adapters

Build docs developers (and LLMs) love

Measure adapter

Purpose

Constructor

Object key pattern

Interface methods

Soft-delete

Adapter registration

Interval adapter

Purpose

Constructor

Object key pattern

Interface methods

Soft-delete

Adapter registration

Memory adapter

Purpose

Constructor

Object key pattern

Interface methods

Soft-delete

Adapter registration

Comparison: list behaviour across the three adapters