VINS-Fusion estimates pose by tightly fusing camera images and IMU measurements through a sliding-window nonlinear optimizer. This page explains the core pipeline stages and key design decisions.Documentation Index
Fetch the complete documentation index at: https://mintlify.com/HKUST-Aerial-Robotics/Vins-Fusion/llms.txt
Use this file to discover all available pages before exploring further.
Pipeline stages
Feature tracking
The
FeatureTracker class tracks 2D features across frames using Lucas-Kanade optical flow. Forward-backward flow checking (flow_back: 1) improves tracking reliability. Up to max_cnt features are tracked, with a minimum pixel distance of min_dist between them.IMU pre-integration
Between consecutive keyframes, IMU measurements are pre-integrated using
IntegrationBase. Pre-integration avoids re-propagating raw measurements during optimization — only the pre-integrated delta pose, velocity, and biases are used as residuals.Initialization
Before entering the main optimization, VINS-Fusion solves for an initial structure-from-motion solution, aligns it with IMU data to recover gravity direction and scale, and initializes accelerometer/gyroscope biases. The
Estimator transitions from INITIAL to NON_LINEAR state once initialization succeeds.Sliding-window optimization
The estimator maintains a window of
WINDOW_SIZE = 10 keyframes. Each optimization round jointly optimizes poses, velocities, IMU biases, feature depths, and optionally camera-IMU extrinsics and time offset. Ceres Solver handles the nonlinear least-squares problem.Key classes
| Class | Role |
|---|---|
Estimator | Central state machine: receives IMU and image data, runs optimization |
FeatureTracker | Optical flow tracking, feature extraction |
FeatureManager | Manages the 3D feature map across the sliding window |
IntegrationBase | IMU pre-integration between keyframes |
Failure detection and recovery
Estimator::failureDetection() monitors for degenerate states such as large bias estimates or excessive position jumps. On failure, clearState() resets the estimator and re-initializes from scratch. You can also trigger a manual restart by publishing true to the /vins_restart topic.
The sliding window size of 10 frames is fixed at compile time (
WINDOW_SIZE = 10 in parameters.h). Larger windows improve accuracy at the cost of computation time.Related papers
- VINS-Mono (IEEE Transactions on Robotics): foundational algorithm description
- Online Temporal Calibration (IROS 2018, best student paper): camera-IMU time offset estimation