Backtesting: vectorbt, Backtrader, and Lean Engines

Backtesting is the process of replaying a strategy’s signal logic against historical price data to estimate how it would have performed. The Hedge Fund Backend’s Backtest Engine resolves all of a strategy’s referenced artifacts — features, model predictions, signal logic — feeds them to vectorbt, Backtrader, or Lean, and persists the resulting equity curve, trade list, and a comprehensive suite of performance metrics. A completed backtest is a prerequisite for validation and strategy promotion.

BacktestPipeline

When you execute a backtest, the pipeline runs the following resolution steps before any simulation begins:

Strategy config
   │
   ├── feature_ids ──► Feature Store → FeatureDataset Parquets → X matrix
   │
   ├── model_id ─────► MLflow artifact store → load fitted model → predictions ŷ
   │
   ├── signal_logic_id ─► Rule tree evaluation → signal series (BUY/SELL/HOLD)
   │
   └── symbol + timeframe ─► OHLCV prices
         │
         ▼
     Backtest Engine (vectorbt | backtrader | lean)
         │
         ▼
     compute_metrics() → BacktestMetrics
         │
         ▼
     Persist equity curve + trades to S3 as Parquet
         │
         ▼
     Write metrics dict to Backtest.metrics (JSONB)

The execution request that triggers this pipeline. The engine field accepts "vectorbt", "backtrader", or "lean":

// POST /api/backtests
{
  "strategy_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "engine": "vectorbt",
  "initial_capital": 100000.0,
  "commission": 0.0005,
  "slippage": 0.0005,
  "config": {
    "symbol": "AAPL",
    "timeframe": "1d",
    "start_date": "2020-01-01T00:00:00Z",
    "end_date": "2024-01-01T00:00:00Z",
    "feature_ids": ["f1000000-0000-0000-0000-000000000001"],
    "model_id": "m1000000-0000-0000-0000-000000000001",
    "signal_logic_id": "s1000000-0000-0000-0000-000000000001",
    "bars_per_year": 252,
    "risk_free_rate": 0.05,
    "size_type": "percent"
  }
}

Backtest Engines

vectorbt

Fast vectorized simulation. Operates on NumPy arrays in a single pass — backtests over years of daily data complete in milliseconds. Supports discrete (BUY/SELL/HOLD) and numeric (±1) signal modes. Preferred engine for parameter sweeps and AutoML comparisons where speed matters.

Backtrader

Event-driven simulation. Processes bars one at a time, firing order execution, fill, and commission events. More realistic for strategies that require order management, position sizing logic, or custom broker models. A custom BacktraderStrategy class is built dynamically from the signal series.

Lean

QuantConnect Lean engine. Production-grade, multi-asset simulation with institutional-quality fill modelling, margin, and corporate-action handling. Use "engine": "lean" in the execution request to target this adapter.

All three engines call the same compute_metrics() function after their run completes, ensuring that metrics definitions are identical regardless of which engine was used.

The size_type field in BacktestRunConfig controls position sizing:

"percent" — allocates a fixed percentage of portfolio equity per signal
"shares" — allocates a fixed share count
"signal_weight" — uses the raw numeric prediction magnitude as the position weight (numeric output mode only)

BacktestResult Fields

After execution, the Backtest record exposes:

Field	Type	Description
`status`	`string`	`pending` → `running` → `completed` \| `failed`
`metrics`	`object`	Flat dict of all computed metrics (see below)
`equity_curve_uri`	`string`	S3 URI to a Parquet file with columns `(timestamp, equity)`
`trades_uri`	`string`	S3 URI to a Parquet file with columns `(entry, exit, pnl, return_pct, side)`
`initial_capital`	`float`	Starting capital in base currency
`commission`	`float`	Per-trade commission as a fraction of trade value
`slippage`	`float`	Per-trade slippage as a fraction of trade value

Metrics Reference

The compute_metrics() function (in app/engines/backtest_engine/metrics.py) stores 20 values flat in the Backtest.metrics JSONB column. Two top-level keys (total_return, bars_in_market) are unprefixed; the remaining 18 carry perf_, risk_, or trade_ prefixes:

Top-Level Keys (unprefixed)

Metric	Key	Description
Total Return	`total_return`	`(final_equity / initial_capital) - 1`
Bars in Market	`bars_in_market`	Number of bars where a non-zero position was held

Performance Metrics

Metric	Key	Description
CAGR	`perf_cagr`	Compound Annual Growth Rate: `(final/initial)^(1/n_years) - 1`
Sharpe Ratio	`perf_sharpe_ratio`	Annualised excess return / annualised volatility
Sortino Ratio	`perf_sortino_ratio`	Annualised excess return / downside deviation
Calmar Ratio	`perf_calmar_ratio`	CAGR / Max Drawdown

Risk Metrics

Metric	Key	Description
Max Drawdown	`risk_max_drawdown`	Largest peak-to-trough decline as a positive fraction
Max DD Duration	`risk_max_drawdown_duration`	Longest time (in bars) spent below previous high
VaR 95%	`risk_var_95`	1-bar 95% Value at Risk (5th percentile of returns)
CVaR 95%	`risk_cvar_95`	Expected Shortfall at 95% — mean return below VaR 95%
VaR 99%	`risk_var_99`	1-bar 99% Value at Risk (1st percentile of returns)
CVaR 99%	`risk_cvar_99`	Expected Shortfall at 99%
Annualised Vol	`risk_volatility_annualised`	`std(daily_returns) × √bars_per_year`

Trading Metrics

Metric	Key	Description
Total Trades	`trade_total_trades`	Number of closed round-trip trades
Win Rate	`trade_win_rate`	Fraction of trades with positive PnL
Profit Factor	`trade_profit_factor`	Gross profit / gross loss
Avg Win	`trade_avg_win`	Mean PnL of winning trades
Avg Loss	`trade_avg_loss`	Mean PnL of losing trades
Expectancy	`trade_expectancy`	`win_rate × avg_win + (1 - win_rate) × avg_loss`
Annualised Turnover	`trade_turnover_annualised`	Fraction of portfolio rotated per year

A full structured response looks like:

{
  "id": "b1000000-0000-0000-0000-000000000001",
  "status": "completed",
  "engine": "vectorbt",
  "initial_capital": 100000.0,
  "structured_metrics": {
    "total_return": 0.412,
    "bars_in_market": 187,
    "performance": {
      "cagr": 0.091,
      "sharpe_ratio": 1.24,
      "sortino_ratio": 1.87,
      "calmar_ratio": 0.73
    },
    "risk": {
      "max_drawdown": 0.124,
      "max_drawdown_duration": 43,
      "var_95": -0.0182,
      "cvar_95": -0.0261,
      "var_99": -0.0318,
      "cvar_99": -0.0447,
      "volatility_annualised": 0.138
    },
    "trading": {
      "total_trades": 94,
      "win_rate": 0.553,
      "profit_factor": 1.48,
      "avg_win": 812.4,
      "avg_loss": -551.2,
      "expectancy": 202.8,
      "turnover_annualised": 0.74
    }
  }
}

bars_per_year defaults to 252 for daily strategies, 52 for weekly, and 12 for monthly. Always set this correctly — Sharpe, Sortino, CAGR, and volatility are all annualisation-dependent.

Parameter Sweep

To find the optimal configuration across many parameter combinations, use the sweep endpoint:

// POST /api/backtests/sweep
{
  "base_config": {
    "strategy_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "engine": "vectorbt",
    "initial_capital": 100000.0,
    "commission": 0.0005,
    "slippage": 0.0005
  },
  "param_grid": {
    "commission":  [0.0001, 0.0005, 0.001],
    "slippage":    [0.0002, 0.0005],
    "size_type":   ["percent", "signal_weight"]
  },
  "rank_by": "perf_sharpe_ratio",
  "direction": "maximize"
}

The engine generates all 12 parameter combinations, dispatches them to Celery workers in parallel, and returns a ranked leaderboard when all tasks complete. This is the recommended way to tune execution parameters without running backtests sequentially.

Backtest Comparison

Compare up to 10 backtest runs side-by-side to highlight which configuration wins on each metric:

// POST /api/backtests/compare
{
  "backtest_ids": [
    "b1000000-0000-0000-0000-000000000001",
    "b1000000-0000-0000-0000-000000000002",
    "b1000000-0000-0000-0000-000000000003"
  ]
}

The response metric_diff object contains, for each metric, the value from every run and the ID of the winning run:

{
  "runs": [ { "...": "summary per run" } ],
  "metric_diff": {
    "perf_sharpe_ratio": {
      "values": {
        "b1000000-...": 1.24,
        "b1000000-...": 0.98,
        "b1000000-...": 1.41
      },
      "best_id": "b1000000-0000-0000-0000-000000000003",
      "higher_is_better": true
    },
    "risk_max_drawdown": {
      "values": {
        "b1000000-...": 0.124,
        "b1000000-...": 0.089,
        "b1000000-...": 0.152
      },
      "best_id": "b1000000-0000-0000-0000-000000000002",
      "higher_is_better": false
    }
  }
}

A strong in-sample backtest is not a reliable predictor of live performance. Always follow backtesting with walk-forward analysis or CPCV validation before promoting a strategy. See Strategy Validation for details.

Async Execution

For multi-year daily backtests or any Backtrader run (which is inherently event-driven and slower), pass async_mode: true to POST /api/backtests/{id}/execute. The endpoint returns a Celery task_id immediately. Poll GET /api/backtests/{id} to check status.

API Reference

For the complete endpoint reference — including artifact download, equity curve streaming, and sweep status polling — see the Backtests API.

Get Started

Core Concepts

Guides

Backtesting: vectorbt, Backtrader, and Lean Engines

BacktestPipeline

Backtest Engines

vectorbt

Backtrader

Lean

BacktestResult Fields

Metrics Reference

Top-Level Keys (unprefixed)

Performance Metrics

Risk Metrics

Trading Metrics

Parameter Sweep

Backtest Comparison

Async Execution

API Reference

Build docs developers (and LLMs) love

Get Started

Core Concepts

Guides

Documentation Index

​BacktestPipeline

​Backtest Engines

vectorbt

Backtrader

Lean

​BacktestResult Fields

​Metrics Reference

​Top-Level Keys (unprefixed)

​Performance Metrics

​Risk Metrics

​Trading Metrics

​Parameter Sweep

​Backtest Comparison

​Async Execution

​API Reference

Build docs developers (and LLMs) love

BacktestPipeline

Backtest Engines

BacktestResult Fields

Metrics Reference

Top-Level Keys (unprefixed)

Performance Metrics

Risk Metrics

Trading Metrics

Parameter Sweep

Backtest Comparison

Async Execution

API Reference