TheDocumentation Index
Fetch the complete documentation index at: https://mintlify.com/theonetrade/backtest-ollama-crontab/llms.txt
Use this file to discover all available pages before exploring further.
packages/core package defines two Mongoose-backed MongoDB collections that form the persistence backbone of the pipeline. Raw Telegram signals land in parser-items after being scraped and parsed; once the local Ollama LLM has evaluated each row, the risk verdict is written to screen-items. A shared BaseCRUD factory provides all standard read/write operations for both collections without duplication.
parser-items Collection
The parser-items collection stores every trading signal extracted from Telegram. Each document maps one-to-one with a single Telegram message from a monitored channel. The visited flag starts as false and is flipped to true only after the LLM screening job has processed the row, making it the primary cursor for the 15-minute crontab.
TypeScript Interfaces
Field Reference
Telegram channel name that published the signal. Carries a single-field ascending index (
channel: 1) and participates in the unique compound index { channel: 1, messageId: 1 }.Source identifier string for the originating feed or bot account. Indexed (
source: 1).Telegram-assigned numeric message ID within the channel. Indexed (
messageId: 1). Together with channel it forms the unique compound index that prevents duplicate ingestion.UTC timestamp at which the message was published in the Telegram channel. Indexed (
publishedAt: 1) and used as the descending sort key in the composite index { symbol: 1, publishedAt: -1 }.Raw, unprocessed message text as received from Telegram — the full original string before field extraction.
Normalised trading pair, e.g.
"SOLUSDT". Indexed (symbol: 1) and the leading key of the composite index { symbol: 1, publishedAt: -1 }.Trade direction extracted from the signal. Validated against the enum
["long", "short"].Entry price zone as a nested sub-document.
Ordered array of take-profit price levels as published in the signal.
Stop-loss price level.
Full parsed signal object stored as
Schema.Types.Mixed. Preserves the complete structured output of the parser for debugging and audit purposes.Set to
false on document creation (Mongoose default: false). The 15-minute crontab queries { visited: false } to find unprocessed rows; ParserDbService.markVisited() flips this to true once the LLM job completes. Indexed (visited: 1).MongoDB
ObjectId serialised as a hex string. Added by readTransform on IParserRow; not present on IParserDto.Auto-set by Mongoose timestamps (
createdAt alias). Present on IParserRow only.Auto-set by Mongoose timestamps (
updatedAt alias). Present on IParserRow only.MongoDB Indexes
| Index definition | Options | Purpose |
|---|---|---|
{ channel: 1, messageId: 1 } | unique: true | Idempotent upsert — prevents duplicate messages |
{ symbol: 1, publishedAt: -1 } | — | Fast lookups of recent signals by symbol |
{ channel: 1 } | — | Filter by channel |
{ source: 1 } | — | Filter by source |
{ messageId: 1 } | — | Direct message lookup |
{ publishedAt: 1 } | — | Date-range queries |
{ symbol: 1 } | — | Symbol filter |
{ visited: 1 } | — | Crontab queue scan |
screen-items Collection
The screen-items collection holds the LLM risk-assessment verdict for each parser row. A screen document is created exactly once per parser row (enforced by the unique index on parserItemId). The entry zone is flattened from the nested entry.{ from, to } shape of parser-items into top-level entryFrom/entryTo fields for easier query expressions.
TypeScript Interfaces
Field Reference
Foreign key referencing the parent
parser-items document by its id string. Enforced as unique by the index { parserItemId: 1 } — one screen result per parser row.Copied from the parent parser row. Indexed.
Copied from the parent parser row. Indexed.
Copied from the parent parser row. Indexed and used in the composite
{ symbol: 1, publishedAt: -1 } index.Copied from the parent parser row. Indexed and the leading key of the composite
{ symbol: 1, publishedAt: -1 } index.Trade direction, copied from the parent parser row.
Lower bound of the entry zone, flattened from
entry.from on the parser row.Upper bound of the entry zone, flattened from
entry.to on the parser row.Take-profit levels, copied from the parent parser row.
Stop-loss price, copied from the parent parser row.
The primary LLM verdict.
"follow" means the model judges the signal safe to act on; "skip" means it should be ignored. Indexed (riskAction: 1). Strategy files read this field to gate position entry.The model’s self-reported confidence that candle-level manipulation was detected. Stored for audit and backtesting analysis. Indexed (
riskSureLevel: 1).| Value | Meaning |
|---|---|
low | No manipulation traces; volume structure is organic |
low_medium | One recent anomaly without price displacement |
medium | One older anomaly or anomaly with price displacement |
medium_high | Repeated accumulation pattern detected |
high | Multiple accumulation events with clear price displacement |
The model’s confidence in its own evaluation.
"reliable" means data was unambiguous; "not_reliable" means data was contradictory, sparse, or absent. Stored for audit. Indexed (riskConfidence: 1).A 2–3 sentence human-readable summary of the verdict written by the LLM, naming the applied rule and key numeric values.
Step-by-step reasoning produced by the LLM as a single newline-separated string. Covers metrics used, the applied rule, and the derivation of
riskSureLevel/riskConfidence.Raw message text, copied from the parent parser row.
Full parsed signal object (
Schema.Types.Mixed), copied from the parent parser row.MongoDB
ObjectId as a hex string added by readTransform. Present on IScreenRow only.Auto-set Mongoose timestamp. Present on
IScreenRow only.Auto-set Mongoose timestamp. Present on
IScreenRow only.MongoDB Indexes
| Index definition | Options | Purpose |
|---|---|---|
{ parserItemId: 1 } | unique: true | One screen result per parser row |
{ symbol: 1, publishedAt: -1 } | — | Fast recent-signal lookups by symbol |
{ channel: 1 } | — | Filter by channel |
{ source: 1 } | — | Filter by source |
{ publishedAt: 1 } | — | Date-range queries |
{ symbol: 1 } | — | Symbol filter |
{ riskAction: 1 } | — | Quick follow/skip partition |
{ riskSureLevel: 1 } | — | Audit-log queries |
{ riskConfidence: 1 } | — | Audit-log queries |
RiskOutlineFormat Zod Schema
The RiskOutlineFormat zod schema, defined in packages/core/src/contract/RiskOutline.ts, describes the exact JSON object the Ollama LLM must return. The five fields map directly onto the corresponding risk* columns of screen-items.
The LLM is instructed to produce
reasoning as a single newline-separated string, not a JSON object or array. The zod schema enforces z.string() to catch malformed responses before they reach the database write path.RiskOutlineContract is:
SignalLogicService maps these fields onto IScreenDto as follows:
RiskOutlineContract field | IScreenDto / IScreenRow field |
|---|---|
action | riskAction |
sure_level | riskSureLevel |
confidence | riskConfidence |
description | riskDescription |
reasoning | riskReasoning |
BaseCRUD — Shared Database Base Class
BaseCRUD is a di-factory factory function located at packages/core/src/lib/common/BaseCRUD.ts. Both ParserDbService and ScreenDbService extend it by passing their respective Mongoose model. Every method runs readTransform on the raw toJSON() output to normalise the _id → id renaming and strip internal Mongoose fields.
BaseCRUD is created with factory() from di-factory, so it is a mixin constructor — subclasses call BaseCRUD(Model) to get a class they can extend, not a singleton.Methods
Inserts a new document using
Model.create(dto) and returns the transformed row.Calls
Model.findByIdAndUpdate with { new: true, runValidators: true }. Strips the id key from dto before updating to prevent ObjectId conflicts. Throws Error("<ModelName> not found") if no document matches.Finds a document by its MongoDB
_id. Throws Error("<ModelName> not found") if the document does not exist.Calls
Model.findOne(filterData, null, { sort }). Returns the transformed document or null if no match exists.Calls
Model.find(filterData).sort({ date: -1 }).limit(limit). Default filterData is {} and default limit is 1000.An async generator that streams documents from MongoDB one at a time. Suitable for large result sets where loading everything into memory is undesirable.
Returns a page of results plus the full collection count for the given filter. Uses
.skip(offset).limit(limit) internally.ParserDbService — Extended Methods
ParserDbService (at packages/core/src/lib/services/db/ParserDbService.ts) extends BaseCRUD with upsert semantics and several typed query helpers.
Uses
$setOnInsert with an upsert on { channel, messageId }. If a document already exists for that pair it is returned unchanged — preventing duplicate ingestion.Shorthand for
findAll({ visited }). Used by the crontab to fetch the unprocessed queue.Returns all rows whose
publishedAt falls within the closed [from, to] range.Returns the most recent parser row for
symbol within the 4-hour window ending at when, sorted by publishedAt descending.Sets
visited: true on a single document via $set. Returns true if a document was updated, false otherwise.ScreenDbService — Extended Methods
ScreenDbService (at packages/core/src/lib/services/db/ScreenDbService.ts) uses the same upsert pattern keyed on parserItemId.
Uses
$setOnInsert with an upsert on { parserItemId }. Idempotent — calling it twice for the same parser row is safe.Returns the screen row associated with the given parser row ID, or
null if screening has not yet run.Returns the most recent screen row for
symbol within the 4-hour window ending at when, sorted by publishedAt descending. Called by SignalMainService.getLast4HourSignal.