Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/ragavsachdeva/CYWS-3D/llms.txt

Use this file to discover all available pages before exploring further.

The fastest way to see CYWS-3D in action is to run inference.py against the demo images that are already included in the repository. The script reads a YAML metadata file that describes each image pair, runs the full 3D registration and change-detection pipeline, and writes annotated output images to disk. The steps below take you from a fresh clone to your first predictions.
1

Clone the repository and install dependencies

If you have not done so already, clone the repository — including its SuperGluePretrainedNetwork submodule — and follow the Installation guide to create the conda environment and install all dependencies.
git clone --recursive git@github.com:ragavsachdeva/CYWS-3D.git
cd CYWS-3D
conda activate cyws3d
2

Download the pre-trained checkpoint

Download and decompress the model weights into the repository root. The inference script expects a .ckpt file.
wget https://thor.robots.ox.ac.uk/cyws-3d/cyws-3d.ckpt.gz
gzip -d cyws-3d.ckpt.gz
3

Run inference on the demo data

Execute inference.py pointing at the downloaded checkpoint. By default the script reads demo_data/input_metadata.yml, which already contains several example image pairs covering all three registration strategies.
python inference.py --load_weights_from ./cyws-3d.ckpt
The first run will also invoke ZoeDepth to predict depth for any image pairs that do not supply ground-truth depth files, so it may take a minute or two longer than subsequent runs.
4

Inspect the output images

For every item in the batch, inference.py writes an annotated image to the working directory:
prediction_0.png
prediction_1.png
prediction_2.png
...
Each prediction_{i}.png corresponds to the i-th entry in the batch list of your metadata file. The image shows both input views side-by-side with predicted change bounding boxes overlaid, ranked by confidence score.
If you uncomment the plot_correspondences call in inference.py, the script will also write correspondences_{i}.png files showing the SuperGlue feature matches used during 3D registration — useful for debugging poor alignments.

​
Input metadata format

The script’s input is controlled by a YAML file passed via --input_metadata. Each entry in the batch list describes one image pair. The registration_strategy field (3d, 2d, or identity) determines how the two views are aligned before change detection. Optional fields provide ground-truth depth maps, camera intrinsics, and pose information when available. 
batch:
  # 3D scene — RGB only (depth predicted automatically by ZoeDepth)
  - image1: "demo_data/ship_0.jpg"
    image2: "demo_data/ship_2.jpg"
    registration_strategy: "3d"

  # 3D scene — RGB + ground-truth depth
  - image1: "demo_data/ship_0.jpg"
    image2: "demo_data/ship_2.jpg"
    depth1: "demo_data/depth_ship_0.png"
    depth2: "demo_data/depth_ship_2.png"
    registration_strategy: "3d"

  # 3D scene — RGB + depth + full camera parameters
  - image1: "demo_data/24478_831_2_0.png"
    image2: "demo_data/24478_831_2_1.png"
    depth1: "demo_data/depth_24478_831_2_0.tiff"
    depth2: "demo_data/depth_24478_831_2_1.tiff"
    intrinsics1: "demo_data/intrinsics1.npy"
    intrinsics2: "demo_data/intrinsics2.npy"
    position1: "demo_data/position1.npy"
    position2: "demo_data/position2.npy"
    rotation1: "demo_data/rotation1.npy"
    rotation2: "demo_data/rotation2.npy"
    registration_strategy: "3d"

  # 2D scene — RGB only (affine registration estimated internally)
  - image1: "demo_data/coco_aff1.png"
    image2: "demo_data/coco_aff2.png"
    registration_strategy: "2d"

  # 2D scene — RGB + ground-truth affine transformation
  - image1: "demo_data/coco_aff1.png"
    image2: "demo_data/coco_aff2.png"
    transfm2d_1_to_2: "demo_data/transfm2d_1_to_2.npy"
    transfm2d_2_to_1: "demo_data/transfm2d_2_to_1.npy"
    registration_strategy: "2d"

  # Identity — images already aligned, no warping applied
  - image1: "demo_data/4842.png"
    image2: "demo_data/4842_mask3.png"
    registration_strategy: "identity"
All paths in input_metadata.yml are resolved relative to the working directory from which you launch inference.py. Adjust them accordingly if you run the script from a different location.

​
Overriding inference parameters

All inference.py arguments can be overridden from the command line via jsonargparse. The full set of available flags and their defaults is shown below:
FlagDefaultDescription
--config_fileconfig.ymlPath to the model configuration file
--input_metadatademo_data/input_metadata.ymlPath to the batch metadata YAML
--load_weights_fromNonePath to the .ckpt checkpoint (required)
--filter_predictions_with_area_under400Drop predicted boxes whose pixel area is below this threshold
--keep_matching_bboxes_onlyTrueOnly retain boxes that have a matching detection in both views
--max_predictions_to_display5Maximum number of boxes drawn per image
--minimum_confidence_threshold0.1Minimum score for a prediction to be displayed
For example, to use a custom metadata file, lower the confidence threshold, and show up to ten predictions per image: 
python inference.py \
  --load_weights_from ./cyws-3d.ckpt \
  --input_metadata my_metadata.yml \
  --minimum_confidence_threshold 0.2 \
  --max_predictions_to_display 10
To remove very small detections (e.g. artefacts smaller than 200 px²) and show all predicted boxes regardless of cross-view matching: 
python inference.py \
  --load_weights_from ./cyws-3d.ckpt \
  --filter_predictions_with_area_under 200 \
  --keep_matching_bboxes_only false
Setting --minimum_confidence_threshold to a lower value will surface more potential changes but may also increase false positives. For high-precision applications, consider raising it to 0.3 or above.

Build docs developers (and LLMs) love

Get started for freeTalk to us