Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/ragavsachdeva/CYWS-3D/llms.txt

Use this file to discover all available pages before exploring further.

The inference script ties together every component of CYWS-3D — the Model, the CorrespondenceExtractor, and an optional depth predictor — to produce change-detection bounding boxes for a batch of image pairs. Given a config file and a metadata YAML that describes your input images, the script loads weights, runs the full pipeline, applies post-processing (area filtering, NMS, cross-image matching), and saves an annotated side-by-side PNG for each item in the batch.

​
Running Inference

The simplest invocation uses all defaults and only supplies the required checkpoint path: 
python inference.py --load_weights_from checkpoints/cyws3d.ckpt
You can override every parameter on the command line. The example below points to a custom metadata file, raises the confidence threshold, and limits output to three bounding boxes per image: 
python inference.py \
  --config_file config.yml \
  --input_metadata my_data/pairs.yml \
  --load_weights_from checkpoints/cyws3d.ckpt \
  --filter_predictions_with_area_under 600 \
  --keep_matching_bboxes_only true \
  --max_predictions_to_display 3 \
  --minimum_confidence_threshold 0.25
The CLI is powered by jsonargparse. Every parameter maps directly to a --param_name value flag. Boolean flags accept true / false (case-insensitive).

​
Parameters

​
config_file
str
default:"config.yml"
Path to the YAML configuration file that controls model architecture and training settings. The default config.yml at the repository root is suitable for inference with the released checkpoint.
​
input_metadata
str
default:"demo_data/input_metadata.yml"
Path to the batch metadata YAML file that describes each image pair. See Input Metadata Format below for the full schema including all six registration strategies.
​
load_weights_from
str
default:"None"
Path to a .ckpt checkpoint file. This parameter is required for any real inference run — without it the model uses randomly initialised weights.
Omitting load_weights_from does not raise an error but will produce meaningless predictions. Always supply a trained checkpoint.
​
filter_predictions_with_area_under
int
default:"400"
Discard predicted bounding boxes whose pixel area (in the 224×224 coordinate space) is smaller than this value. Raising this threshold removes small, noisy detections; lowering it retains fine-grained changes.
​
keep_matching_bboxes_only
bool
default:"true"
When true, the post-processor runs keep_matching_bboxes to retain only bounding boxes that have a plausible cross-image correspondence. Disabling this flag returns all surviving detections regardless of whether the change appears in both views.
​
max_predictions_to_display
int
default:"5"
Maximum number of bounding boxes drawn per image in the output PNG. Boxes are already ranked by confidence after NMS, so only the top-N detections appear in the visualisation. This limit does not affect the raw model output.
​
minimum_confidence_threshold
float
default:"0.1"
Minimum detection score required when keep_matching_bboxes_only is true. Boxes below this threshold are excluded before the cross-image matching step. Increase this value to reduce false positives at the cost of recall.

​
Output

For every item i in the batch the script writes a PNG to the working directory: 
prediction_0.png
prediction_1.png
...
Each file is a side-by-side composite of the two input images with yellow bounding boxes drawn around detected changes. Boxes are clipped to the top max_predictions_to_display detections per image after all filtering steps.
If keep_matching_bboxes_only is enabled, the bounding box in image 1 and its counterpart in image 2 are spatially correlated — the model has verified that the flagged region appears (or disappears) across both views.
The visualisation helper also writes per-pair correspondence images (correspondences_{i}.png) that show the SuperPoint + SuperGlue keypoint matches used to establish the geometric relationship between the two views. These are useful for diagnosing cases where the matching step fails.

​
Input Metadata Format

The metadata YAML contains a top-level batch key whose value is a list of per-pair dictionaries. The registration_strategy field determines which additional keys are required.

​
Strategy 1 — 3d (full camera poses + intrinsics)

batch:
  - image1: path/to/image1.jpg
    image2: path/to/image2.jpg
    depth1: path/to/depth1.tiff          # optional; predicted by ZoeDepth if absent
    depth2: path/to/depth2.tiff          # optional; predicted by ZoeDepth if absent
    intrinsics1: path/to/intrinsics1.npy # 3×3 camera matrix
    intrinsics2: path/to/intrinsics2.npy
    position1: path/to/position1.npy     # (3,) translation vector
    position2: path/to/position2.npy
    rotation1: path/to/rotation1.npy     # (3,3) rotation matrix
    rotation2: path/to/rotation2.npy
    registration_strategy: "3d"

​
Strategy 2 — 3d without pre-computed depth (auto depth prediction)

batch:
  - image1: path/to/image1.jpg
    image2: path/to/image2.jpg
    intrinsics1: path/to/intrinsics1.npy
    intrinsics2: path/to/intrinsics2.npy
    position1: path/to/position1.npy
    position2: path/to/position2.npy
    rotation1: path/to/rotation1.npy
    rotation2: path/to/rotation2.npy
    registration_strategy: "3d"
When depth maps are absent for a 3d pair, the pipeline automatically runs ZoeDepth (ZoeD_NK) on both images. Depth prediction requires a GPU and adds latency.

​
Strategy 3 — 2d_from_corr (homography from keypoints)

batch:
  - image1: path/to/image1.jpg
    image2: path/to/image2.jpg
    registration_strategy: "2d_from_corr"
The CorrespondenceExtractor (SuperPoint + SuperGlue) runs automatically and estimates a 2-D homography from the matched keypoints.

​
Strategy 4 — 2d (pre-computed 2-D transform)

batch:
  - image1: path/to/image1.jpg
    image2: path/to/image2.jpg
    transfm2d_1_to_2: path/to/H_1_to_2.npy  # (3,3) homography
    transfm2d_2_to_1: path/to/H_2_to_1.npy  # (3,3) homography
    registration_strategy: "2d"

​
Strategy 5 — identity (no registration)

batch:
  - image1: path/to/image1.jpg
    image2: path/to/image2.jpg
    registration_strategy: "identity"
Use this strategy when the two images are already aligned (e.g. same camera, no viewpoint change). The CorrespondenceExtractor is skipped entirely.

​
Strategy 6 — Mixed batch

batch:
  - image1: scene_a/view1.jpg
    image2: scene_a/view2.jpg
    registration_strategy: "identity"
  - image1: scene_b/view1.jpg
    image2: scene_b/view2.jpg
    intrinsics1: scene_b/K1.npy
    intrinsics2: scene_b/K2.npy
    position1: scene_b/t1.npy
    position2: scene_b/t2.npy
    rotation1: scene_b/R1.npy
    rotation2: scene_b/R2.npy
    registration_strategy: "3d"
Different items in the same batch may use different registration strategies. The pipeline handles each item individually before collating into a single forward pass.

Build docs developers (and LLMs) love

Get started for freeTalk to us