Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/ragavsachdeva/CYWS-3D/llms.txt

Use this file to discover all available pages before exploring further.

geometry.py provides the mathematical building blocks that the rest of CYWS-3D relies on for spatial reasoning. The module is split into two groups of functions: coordinate utilities that project points between image space, camera space, and 3D world space; and bounding box utilities that filter, de-duplicate, and cross-match predicted change regions across the two views.

​
Coordinate utilities

​
get_index_grid()

Returns a dense normalised coordinate grid covering the full image, which is used internally to unproject every pixel to world space during feature warping.
​
height
int
required
Height of the desired grid in pixels.
​
width
int
required
Width of the desired grid in pixels.
​
batch
int
default:"None"
If provided, the grid is expanded and repeated along a leading batch dimension, producing shape (B, H, W, 2). When None the output is (H, W, 2).
​
type_as
Tensor
default:"None"
If provided, the output tensor is cast to the same dtype and device as this tensor.
Returns a Tensor of shape (H, W, 2) or (B, H, W, 2) with values normalised to [0, 1] along each spatial axis.

​
convert_image_coordinates_to_world()

Unprojects 2D image coordinates to 3D world positions using per-pixel depth and camera matrices.
​
image_coords
Tensor
required
Normalised pixel coordinates, shape (B, N, 2), with values in [0, 1].
​
depth
Tensor
required
Depth at each coordinate, shape (B, N).
​
K_inv
Tensor
required
Inverse intrinsic matrix, shape (B, 3, 3).
​
Rt
Tensor
required
Camera-to-world rigid transform, shape (B, 4, 4).
​
world_points
Tensor
3D world coordinates, shape (B, N, 3).

​
convert_world_to_image_coordinates()

Projects 3D world points into a camera’s image plane.
​
world_points
Tensor
required
3D world positions, shape (B, N, 3).
​
K_inv
Tensor
required
Inverse intrinsic matrix of the target camera, shape (B, 3, 3).
​
Rt
Tensor
required
World-to-camera rigid transform, shape (B, 4, 4).
​
keep_depth
bool
required
When True, the projected depth is appended as a third channel, returning shape (B, N, 3). When False, only the 2D pixel coordinates are returned, shape (B, N, 2).

​
transform_points()

Applies a homogeneous or affine transformation matrix to a set of points.
​
transformation_matrix
Tensor
required
Transform to apply, shape (B, 4, 4) or (4, 4).
​
points
Tensor
required
Input points, shape (B, N, 3).
​
keep_depth
bool
default:"False"
If True, the transformed depth (z-coordinate) is retained in the output.
Returns transformed points as a Tensor, shape (B, N, 2) or (B, N, 3) when keep_depth=True.

​
get_relative_pose()

Computes the relative rotation and translation between two camera poses expressed in a shared world frame.
​
rotation_before
Tensor
required
Rotation matrix of the first camera, shape (B, 3, 3).
​
rotation_after
Tensor
required
Rotation matrix of the second camera, shape (B, 3, 3).
​
position_before
Tensor
required
World-space position of the first camera, shape (B, 3).
​
position_after
Tensor
required
World-space position of the second camera, shape (B, 3).
​
as_single_matrix
bool
default:"False"
Controls the return format (see below).

​
estimate_linear_warp()

Fits a least-squares linear (affine) warp between two sets of corresponding points. Used by the 2D registration path when no explicit transformation matrix is provided in the batch.
​
X
Tensor
required
Source points in homogeneous coordinates, shape (B, N, 3).
​
Y
Tensor
required
Target points in homogeneous coordinates, shape (B, N, 3).
​
M
Tensor
Best-fit warp matrix M of shape (B, 4, 4) such that applying M to X minimises the residual to Y in the least-squares sense:
Y_hat = torch.einsum("bij,bnj->bni", R, X) + T

​
sample_depth_for_given_points()

Samples a dense depth map at a set of normalised 2D coordinates using bilinear interpolation.
​
depth_map
Tensor
required
Dense depth map, shape (B, H, W).
​
points
Tensor
required
Query coordinates normalised to [0, 1], shape (B, N, 2).
Returns sampled depth values, shape (B, N). Internally uses torch.nn.functional.grid_sample.

​
Bounding box utilities

​
bbox_iou_single_pair()

Computes the Intersection-over-Union (IoU) for a single pair of axis-aligned bounding boxes.
​
bbox1
Tensor
required
First bounding box as [x1, y1, x2, y2], shape (4,).
​
bbox2
Tensor
required
Second bounding box as [x1, y1, x2, y2], shape (4,).
Returns a scalar Tensor in [0, 1].

​
remove_bboxes_with_area_less_than()

Filters out degenerate or tiny bounding boxes by Shapely polygon area.
​
bboxes_as_np_array
np.ndarray
required
Array of bounding boxes, shape (N, 4), format [x1, y1, x2, y2].
​
threshold
float
required
Minimum area threshold. Boxes whose Shapely polygon area is strictly less than this value are removed.
Returns a filtered np.ndarray of shape (M, 4) where M ≤ N.

​
suppress_overlapping_bboxes()

Greedy non-maximum suppression (NMS): iteratively keeps the highest-scoring box and removes any remaining boxes that overlap it above iou_threshold.
​
bboxes
Tensor | np.ndarray
required
Candidate bounding boxes, shape (N, 4).
​
scores
Tensor | np.ndarray
required
Confidence score for each box, shape (N,).
​
iou_threshold
float
default:"0.2"
IoU threshold above which a lower-scoring box is suppressed.
Returns (kept_bboxes, kept_scores) in the same type (torch or numpy) as the inputs.
The default iou_threshold of 0.2 is intentionally low. Because changed regions in the same scene are rarely identical duplicates, aggressive suppression is preferred over the more lenient thresholds common in object detection.

​
keep_matching_bboxes()

Cross-view consistency filter: for each predicted bbox in the left image, projects its centre into the right image and keeps the pair only if the projected centre falls inside a predicted bbox on the right side (and vice versa).
​
batch
dict
required
Must contain batch["transform_points_1_to_2"] and batch["transform_points_2_to_1"] callables populated by FeatureRegisterationModule.forward().
​
image_index
int
required
Index into the batch for which to perform matching.
​
left_predictions
Tensor | np.ndarray
required
Predicted bounding boxes for the left/first image, shape (N, 4).
​
right_predictions
Tensor | np.ndarray
required
Predicted bounding boxes for the right/second image, shape (M, 4).
​
left_scores
Tensor | np.ndarray
required
Confidence scores for left predictions, shape (N,).
​
right_scores
Tensor | np.ndarray
required
Confidence scores for right predictions, shape (M,).
​
confidence_threshold
float
default:"0.2"
Boxes with scores below this value are ignored before matching.
Returns (left_bboxes, right_bboxes) as np.ndarray objects containing only the geometrically consistent pairs.

​
bboxes_to_masks()

Rasterises a batch of bounding box lists into binary spatial masks.
​
batch_of_boxes
list[Tensor]
required
A Python list of length B, where each element is a Tensor of shape (N_i, 4) containing boxes for that sample.
​
image_hw
tuple[int, int]
required
(height, width) of the output mask.
Returns a binary Tensor of shape (B, 1, H, W) where pixels inside any box are 1 and all others are 0.

​
remove_invalid_bboxes()

Clamps boxes to lie within the image boundary and discards any that become degenerate after clamping.
​
bboxes_as_tensor
Tensor
required
Bounding boxes, shape (N, 4), format [x1, y1, x2, y2].
​
image_side
int
required
The image side length used as the upper clamp bound (assumes a square image or that the same value applies to both axes).
​
add_dummuy_if_empty
bool
default:"True"
If True and all boxes are removed, a single dummy zero-area box is inserted so that downstream code never receives an empty tensor.
Returns a filtered Tensor of valid boxes.
The add_dummuy_if_empty flag exists to prevent shape errors in batched inference. If you need to distinguish between “no valid boxes” and “one box detected”, check the returned box coordinates explicitly — a dummy box has all coordinates equal to 0.

​
Typical post-processing flow

The bbox utilities compose naturally into a filtering pipeline. After the decoder produces raw predicted boxes and scores for both images: 
from geometry import (
    remove_bboxes_with_area_less_than,
    suppress_overlapping_bboxes,
    keep_matching_bboxes,
)

# 1. Drop tiny / degenerate boxes
left_boxes = remove_bboxes_with_area_less_than(left_boxes_raw, threshold=100)
right_boxes = remove_bboxes_with_area_less_than(right_boxes_raw, threshold=100)

# 2. Greedy NMS within each view
left_boxes, left_scores = suppress_overlapping_bboxes(
    left_boxes, left_scores, iou_threshold=0.2
)
right_boxes, right_scores = suppress_overlapping_bboxes(
    right_boxes, right_scores, iou_threshold=0.2
)

# 3. Keep only cross-view consistent pairs
# batch must contain transform_points_1_to_2 / transform_points_2_to_1
left_final, right_final = keep_matching_bboxes(
    batch,
    image_index=0,
    left_predictions=left_boxes,
    right_predictions=right_boxes,
    left_scores=left_scores,
    right_scores=right_scores,
    confidence_threshold=0.2,
)
Run remove_bboxes_with_area_less_than before NMS — removing tiny boxes first reduces the number of IoU comparisons and avoids suppressing legitimate detections that happen to overlap with dust-speck predictions.

Build docs developers (and LLMs) love

Get started for freeTalk to us