Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/theonetrade/backtest-ollama-crontab/llms.txt

Use this file to discover all available pages before exploring further.

The packages/core package defines two Mongoose-backed MongoDB collections that form the persistence backbone of the pipeline. Raw Telegram signals land in parser-items after being scraped and parsed; once the local Ollama LLM has evaluated each row, the risk verdict is written to screen-items. A shared BaseCRUD factory provides all standard read/write operations for both collections without duplication.

parser-items Collection

The parser-items collection stores every trading signal extracted from Telegram. Each document maps one-to-one with a single Telegram message from a monitored channel. The visited flag starts as false and is flipped to true only after the LLM screening job has processed the row, making it the primary cursor for the 15-minute crontab.

TypeScript Interfaces

/** Input shape — used when creating a new parser document */
interface IParserDto {
  channel: string;
  source: string;
  messageId: number;
  publishedAt: Date;
  note: string;

  symbol: string;
  direction: "long" | "short";
  entry: { from: number; to: number };
  targets: number[];
  stoploss: number;

  content: unknown;
}

/** Output shape — returned by BaseCRUD methods after readTransform */
interface IParserRow extends IParserDto {
  id: string;
  visited: boolean;
  createDate: Date;
  updatedDate: Date;
}

Field Reference

channel
string
required
Telegram channel name that published the signal. Carries a single-field ascending index (channel: 1) and participates in the unique compound index { channel: 1, messageId: 1 }.
source
string
required
Source identifier string for the originating feed or bot account. Indexed (source: 1).
messageId
number
required
Telegram-assigned numeric message ID within the channel. Indexed (messageId: 1). Together with channel it forms the unique compound index that prevents duplicate ingestion.
publishedAt
Date
required
UTC timestamp at which the message was published in the Telegram channel. Indexed (publishedAt: 1) and used as the descending sort key in the composite index { symbol: 1, publishedAt: -1 }.
note
string
required
Raw, unprocessed message text as received from Telegram — the full original string before field extraction.
symbol
string
required
Normalised trading pair, e.g. "SOLUSDT". Indexed (symbol: 1) and the leading key of the composite index { symbol: 1, publishedAt: -1 }.
direction
"long" | "short"
required
Trade direction extracted from the signal. Validated against the enum ["long", "short"].
entry
object
required
Entry price zone as a nested sub-document.
targets
number[]
required
Ordered array of take-profit price levels as published in the signal.
stoploss
number
required
Stop-loss price level.
content
unknown
required
Full parsed signal object stored as Schema.Types.Mixed. Preserves the complete structured output of the parser for debugging and audit purposes.
visited
boolean
required
Set to false on document creation (Mongoose default: false). The 15-minute crontab queries { visited: false } to find unprocessed rows; ParserDbService.markVisited() flips this to true once the LLM job completes. Indexed (visited: 1).
id
string
MongoDB ObjectId serialised as a hex string. Added by readTransform on IParserRow; not present on IParserDto.
createDate
Date
Auto-set by Mongoose timestamps (createdAt alias). Present on IParserRow only.
updatedDate
Date
Auto-set by Mongoose timestamps (updatedAt alias). Present on IParserRow only.

MongoDB Indexes

Index definitionOptionsPurpose
{ channel: 1, messageId: 1 }unique: trueIdempotent upsert — prevents duplicate messages
{ symbol: 1, publishedAt: -1 }Fast lookups of recent signals by symbol
{ channel: 1 }Filter by channel
{ source: 1 }Filter by source
{ messageId: 1 }Direct message lookup
{ publishedAt: 1 }Date-range queries
{ symbol: 1 }Symbol filter
{ visited: 1 }Crontab queue scan

screen-items Collection

The screen-items collection holds the LLM risk-assessment verdict for each parser row. A screen document is created exactly once per parser row (enforced by the unique index on parserItemId). The entry zone is flattened from the nested entry.{ from, to } shape of parser-items into top-level entryFrom/entryTo fields for easier query expressions.

TypeScript Interfaces

/** Input shape — used when creating a new screen document */
interface IScreenDto {
  parserItemId: string;
  channel: string;
  source: string;
  publishedAt: Date;
  symbol: string;

  direction: "long" | "short";
  entryFrom: number;
  entryTo: number;
  targets: number[];
  stoploss: number;

  riskSureLevel: "low" | "low_medium" | "medium" | "medium_high" | "high";
  riskConfidence: "reliable" | "not_reliable";
  riskAction: "skip" | "follow";
  riskDescription: string;
  riskReasoning: string;

  note: string;
  content: unknown;
}

/** Output shape — returned by BaseCRUD methods after readTransform */
interface IScreenRow extends IScreenDto {
  id: string;
  createDate: Date;
  updatedDate: Date;
}

Field Reference

parserItemId
string
required
Foreign key referencing the parent parser-items document by its id string. Enforced as unique by the index { parserItemId: 1 } — one screen result per parser row.
channel
string
required
Copied from the parent parser row. Indexed.
source
string
required
Copied from the parent parser row. Indexed.
publishedAt
Date
required
Copied from the parent parser row. Indexed and used in the composite { symbol: 1, publishedAt: -1 } index.
symbol
string
required
Copied from the parent parser row. Indexed and the leading key of the composite { symbol: 1, publishedAt: -1 } index.
direction
"long" | "short"
required
Trade direction, copied from the parent parser row.
entryFrom
number
required
Lower bound of the entry zone, flattened from entry.from on the parser row.
entryTo
number
required
Upper bound of the entry zone, flattened from entry.to on the parser row.
targets
number[]
required
Take-profit levels, copied from the parent parser row.
stoploss
number
required
Stop-loss price, copied from the parent parser row.
riskAction
"skip" | "follow"
required
The primary LLM verdict. "follow" means the model judges the signal safe to act on; "skip" means it should be ignored. Indexed (riskAction: 1). Strategy files read this field to gate position entry.
riskSureLevel
"low" | "low_medium" | "medium" | "medium_high" | "high"
required
The model’s self-reported confidence that candle-level manipulation was detected. Stored for audit and backtesting analysis. Indexed (riskSureLevel: 1).
ValueMeaning
lowNo manipulation traces; volume structure is organic
low_mediumOne recent anomaly without price displacement
mediumOne older anomaly or anomaly with price displacement
medium_highRepeated accumulation pattern detected
highMultiple accumulation events with clear price displacement
riskConfidence
"reliable" | "not_reliable"
required
The model’s confidence in its own evaluation. "reliable" means data was unambiguous; "not_reliable" means data was contradictory, sparse, or absent. Stored for audit. Indexed (riskConfidence: 1).
riskDescription
string
required
A 2–3 sentence human-readable summary of the verdict written by the LLM, naming the applied rule and key numeric values.
riskReasoning
string
required
Step-by-step reasoning produced by the LLM as a single newline-separated string. Covers metrics used, the applied rule, and the derivation of riskSureLevel/riskConfidence.
note
string
required
Raw message text, copied from the parent parser row.
content
unknown
required
Full parsed signal object (Schema.Types.Mixed), copied from the parent parser row.
id
string
MongoDB ObjectId as a hex string added by readTransform. Present on IScreenRow only.
createDate
Date
Auto-set Mongoose timestamp. Present on IScreenRow only.
updatedDate
Date
Auto-set Mongoose timestamp. Present on IScreenRow only.

MongoDB Indexes

Index definitionOptionsPurpose
{ parserItemId: 1 }unique: trueOne screen result per parser row
{ symbol: 1, publishedAt: -1 }Fast recent-signal lookups by symbol
{ channel: 1 }Filter by channel
{ source: 1 }Filter by source
{ publishedAt: 1 }Date-range queries
{ symbol: 1 }Symbol filter
{ riskAction: 1 }Quick follow/skip partition
{ riskSureLevel: 1 }Audit-log queries
{ riskConfidence: 1 }Audit-log queries

RiskOutlineFormat Zod Schema

The RiskOutlineFormat zod schema, defined in packages/core/src/contract/RiskOutline.ts, describes the exact JSON object the Ollama LLM must return. The five fields map directly onto the corresponding risk* columns of screen-items.
import { str } from "functools-kit";
import { z } from "zod";

export const RiskOutlineFormat = z.object({
  action: z
    .enum(["skip", "follow"])
    .describe(
      str.newline(
        "Решение об открытии позиции:",
        "follow — открыть позицию в направлении канала (direction из draft signal)",
        "skip — пропустить сигнал, не открывать позицию",
      )
    ),
  sure_level: z
    .enum(["low", "low_medium", "medium", "medium_high", "high"])
    .describe(
      str.newline(
        "Уровень уверенности в наличии манипуляции в свечах (для audit log):",
        "low — следов нет, структура объёма органична",
        "low_medium — одна свежая аномалия без смещения",
        "medium — одна аномалия давно или со смещением цены",
        "medium_high — повторяющееся накопление",
        "high — множественное накопление с явным смещением цены",
      )
    ),
  confidence: z
    .enum(["reliable", "not_reliable"])
    .describe(
      str.newline(
        "Уверенность в оценке (для audit log):",
        "reliable — данные однозначны",
        "not_reliable — данные противоречивы, слабые или отсутствуют",
      )
    ),
  description: z
    .string()
    .describe(
      str.newline(
        "Краткое объяснение action для трейдера (2-3 предложения).",
        "Назови применённое правило и числа. Пример:",
        "'Action skip. SHORT на спящем активе (avgRangePct 0.045% < 0.07%) — stop-hunt мишень. Sure_level high, confidence reliable.'"
      )
    ),
  reasoning: z
    .string()
    .describe(
      str.newline(
        "Детальное обоснование (одна строка с переводами через \\n, НЕ объект, НЕ массив):",
        "Шаг 1: avgRangePct=X%, momentum24hPct=Y% (из метрик)",
        "Шаг 2: применённое правило → action",
        "Шаг 3: sure_level и confidence",
      )
    ),
});

export type RiskOutlineContract = z.infer<typeof RiskOutlineFormat>;
The LLM is instructed to produce reasoning as a single newline-separated string, not a JSON object or array. The zod schema enforces z.string() to catch malformed responses before they reach the database write path.
The inferred TypeScript type RiskOutlineContract is:
type RiskOutlineContract = {
  action: "skip" | "follow";
  sure_level: "low" | "low_medium" | "medium" | "medium_high" | "high";
  confidence: "reliable" | "not_reliable";
  description: string;
  reasoning: string;
};
The SignalLogicService maps these fields onto IScreenDto as follows:
RiskOutlineContract fieldIScreenDto / IScreenRow field
actionriskAction
sure_levelriskSureLevel
confidenceriskConfidence
descriptionriskDescription
reasoningriskReasoning

BaseCRUD — Shared Database Base Class

BaseCRUD is a di-factory factory function located at packages/core/src/lib/common/BaseCRUD.ts. Both ParserDbService and ScreenDbService extend it by passing their respective Mongoose model. Every method runs readTransform on the raw toJSON() output to normalise the _id → id renaming and strip internal Mongoose fields.
BaseCRUD is created with factory() from di-factory, so it is a mixin constructor — subclasses call BaseCRUD(Model) to get a class they can extend, not a singleton.

Methods

create(dto)
Promise<object>
Inserts a new document using Model.create(dto) and returns the transformed row.
async create(dto: object): Promise<object>
update(id, dto)
Promise<object>
Calls Model.findByIdAndUpdate with { new: true, runValidators: true }. Strips the id key from dto before updating to prevent ObjectId conflicts. Throws Error("<ModelName> not found") if no document matches.
async update(id: string, dto: object): Promise<object>
findById(id)
Promise<object>
Finds a document by its MongoDB _id. Throws Error("<ModelName> not found") if the document does not exist.
async findById(id: string): Promise<object>
findByFilter(filterData, sort?)
Promise<object | null>
Calls Model.findOne(filterData, null, { sort }). Returns the transformed document or null if no match exists.
async findByFilter(filterData: object, sort?: object): Promise<object | null>
findAll(filterData?, limit?)
Promise<object[]>
Calls Model.find(filterData).sort({ date: -1 }).limit(limit). Default filterData is {} and default limit is 1000.
async findAll(filterData?: object, limit?: number): Promise<object[]>
iterate(filterData?, sort?)
AsyncGenerator<object>
An async generator that streams documents from MongoDB one at a time. Suitable for large result sets where loading everything into memory is undesirable.
async *iterate(filterData?: object, sort?: object): AsyncGenerator<object>
for await (const row of core.parserDbService.iterate({ visited: false })) {
  // process one row at a time — no 1 000-row cap
}
paginate(filterData, pagination, sort?)
Promise<{ rows: object[]; total: number }>
Returns a page of results plus the full collection count for the given filter. Uses .skip(offset).limit(limit) internally.
async paginate(
  filterData: object,
  pagination: { limit: number; offset: number },
  sort?: object
): Promise<{ rows: object[]; total: number }>
const page = await core.screenDbService.paginate(
  { riskAction: "follow" },
  { limit: 50, offset: 0 },
  { publishedAt: -1 }
);
console.log(page.total, page.rows.length);

ParserDbService — Extended Methods

ParserDbService (at packages/core/src/lib/services/db/ParserDbService.ts) extends BaseCRUD with upsert semantics and several typed query helpers.
create(dto: IParserDto)
Promise<IParserRow>
Uses $setOnInsert with an upsert on { channel, messageId }. If a document already exists for that pair it is returned unchanged — preventing duplicate ingestion.
findAllByVisited(visited: boolean)
Promise<IParserRow[]>
Shorthand for findAll({ visited }). Used by the crontab to fetch the unprocessed queue.
findAllByPublishedAt(from: Date, to: Date)
Promise<IParserRow[]>
Returns all rows whose publishedAt falls within the closed [from, to] range.
findLast4HourRow(symbol: string, when: Date)
Promise<IParserRow | null>
Returns the most recent parser row for symbol within the 4-hour window ending at when, sorted by publishedAt descending.
markVisited(rowId: string)
Promise<boolean>
Sets visited: true on a single document via $set. Returns true if a document was updated, false otherwise.

ScreenDbService — Extended Methods

ScreenDbService (at packages/core/src/lib/services/db/ScreenDbService.ts) uses the same upsert pattern keyed on parserItemId.
create(dto: IScreenDto)
Promise<IScreenRow>
Uses $setOnInsert with an upsert on { parserItemId }. Idempotent — calling it twice for the same parser row is safe.
findByParserItem(parserItemId: string)
Promise<IScreenRow | null>
Returns the screen row associated with the given parser row ID, or null if screening has not yet run.
findLast4HourRow(symbol: string, when: Date)
Promise<IScreenRow | null>
Returns the most recent screen row for symbol within the 4-hour window ending at when, sorted by publishedAt descending. Called by SignalMainService.getLast4HourSignal.

Build docs developers (and LLMs) love