Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/backtest-kit/backtest-kit-minio-s3-docker/llms.txt

Use this file to discover all available pages before exploring further.

BaseStorage is a di-factory class that wraps MinIO S3 operations into a simple key/value store interface. Every data service in backtest-kit-minio-s3-docker — candles, signals, strategies, risk, memory entries, and so on — extends BaseStorage. It handles bucket creation (via MinioService.getClient), JSON serialization/deserialization, not-found error normalisation, and batched deletes, so individual data services can focus purely on their domain logic.

​
Constructor and bucket routing

S3 bucket names cannot contain forward slashes, but it is useful to namespace keys within a shared bucket. BaseStorage handles this by splitting the BUCKET_NAME string on /: the first segment becomes the physical MinIO bucket name, and any remaining segments become the rootPrefix that is prepended to every key. 
constructor(public readonly BUCKET_NAME: string) {
  const [bucketName, ...folders] = BUCKET_NAME.split("/");
  this.bucketName = bucketName;
  this.rootPrefix = folders.length ? `${folders.join("/")}/` : "";
}
For example, constructing BaseStorage("backtest-kit/candle-items") produces:
PropertyValue
BUCKET_NAME"backtest-kit/candle-items"
bucketName"backtest-kit"
rootPrefix"candle-items/"
A key like "BTC_1m_1700000000000" is therefore stored in MinIO as candle-items/BTC_1m_1700000000000 inside the backtest-kit bucket. Constructing BaseStorage("backtest-kit") (no slash) sets rootPrefix to "", storing keys directly in the bucket root.

​
Instance properties

​
BUCKET_NAME
string
The original name string passed to the constructor. This is the value as declared by each data service — for example, "backtest-kit/candle-items". It is exposed publicly so that diagnostics code can identify which logical store an instance belongs to.
​
bucketName
string
The physical MinIO bucket name — the first path segment of BUCKET_NAME. This is the string passed to all MinIO SDK calls (putObject, getObject, listObjectsV2, etc.) and to MinioService.getClient() for bucket existence checks.
​
rootPrefix
string
The key prefix prepended to every object key before it is written to or read from MinIO. Empty string when BUCKET_NAME contains no /. Includes a trailing / when non-empty, so object keys are always of the form ${rootPrefix}${key}.

​
Methods

​
set(key, value)
Promise<void>
Serializes value to JSON, encodes it as a UTF-8 Buffer, and uploads it to MinIO via putObject with Content-Type: application/json. Throws if key is an empty string.
async set(key: string, value: unknown): Promise<void>
​
get<T>(key)
Promise<T | null>
Downloads the object identified by key and parses its body as JSON. Returns null immediately if key is null. Returns null (without throwing) if MinIO responds with a not-found error code. Streams the response body via Node.js Readable events and resolves via createAwaiter from functools-kit.
async get<T = unknown>(key: string | null): Promise<T | null>
​
has(key)
Promise<boolean>
Performs a statObject call — metadata only, no body download — and returns true if the object exists. Returns false for not-found errors and re-throws all other errors.
async has(key: string): Promise<boolean>
​
delete(key)
Promise<void>
Removes the object from MinIO using removeObject. No-ops silently if key is null.
async delete(key: string): Promise<void>
​
clear(prefix?)
Promise<void>
Deletes all objects whose keys start with prefix (default: ""). Uses keys() to stream the listing and collects keys into batches of up to DELETE_BATCH_SIZE = 1000 before calling removeObjects. Memory usage is therefore bounded by the batch size, not the total number of objects in the bucket.
async clear(prefix?: string): Promise<void>
​
keys(prefix?, limit?)
AsyncIterableIterator<string>
Yields object keys under prefix using MinIO’s listObjectsV2 with recursive listing enabled. The rootPrefix is stripped from returned keys so callers always see logical keys, not physical MinIO paths. Stops after limit keys if provided.
async *keys(prefix?: string, limit?: number): AsyncIterableIterator<string>
​
values(prefix?, limit?)
AsyncIterableIterator<unknown>
Iterates keys() and calls get() for each key. Skips entries where get() returns null (for example, if an object is removed between the listing and the fetch).
async *values(prefix?: string, limit?: number): AsyncIterableIterator<unknown>
​
iterate(prefix?, limit?)
AsyncIterableIterator<readonly [string, unknown]>
Yields [key, value] tuples, combining keys() and get(). Skips null values just like values(). Useful when you need both the key and the deserialized object together.
async *iterate(prefix?: string, limit?: number): AsyncIterableIterator<readonly [string, unknown]>
​
toArray(prefix?)
Promise<[string, unknown][]>
Collects all entries under prefix into an in-memory array by exhausting iterate(). Convenient for small result sets; for large buckets prefer the async iterator methods to avoid loading everything into memory at once.
async toArray(prefix?: string): Promise<[string, unknown][]>
​
size(prefix?)
Promise<number>
Counts all objects whose keys start with prefix by exhausting keys(). The count is computed client-side because MinIO’s S3 API does not return a total object count during listing.
async size(prefix?: string): Promise<number>

​
DELETE_BATCH_SIZE

const DELETE_BATCH_SIZE = 1_000;
clear() accumulates keys into batches and calls minioClient.removeObjects() once per batch. The S3 DeleteObjects API accepts up to 1000 keys per request, which is also the batch size used here. Batching prevents building an unbounded in-memory array when clearing a bucket with millions of objects and reduces the number of round-trips to the minimum allowed by the protocol.
BaseStorage treats the error codes "NoSuchKey" and "NotFound" as soft not-found signals. get() returns null and has() returns false for these codes rather than throwing. Every other MinIO error is re-thrown immediately. If you see an unexpected null from get(), confirm the bucket and key are correct before assuming a data loss scenario — the object may simply not have been written yet.
Prefer keys(), values(), and iterate() over toArray() when working with large buckets. They are AsyncIterableIterator generators that fetch one MinIO listing page at a time and download objects one-by-one, so peak memory usage stays proportional to the number of objects in a single listing page rather than the total bucket size.

Build docs developers (and LLMs) love

Get started for freeTalk to us