How it works
SAM2 runs off the main thread in a Web Worker so the UI stays responsive during inference. The encoder runs with WebGPU acceleration (falling back to WASM if WebGPU is unavailable in the worker context). Each point you place triggers an immediate decode pass, updating the mask in real time. Per-frame masks are stored using run-length encoding (RLE compression) to keep memory use manageable across long clips.Segmentation workflow
Open the SAM2 panel
Click the AI Segment tab in the dock panels, or go to View → AI Segment.If the model has not been downloaded yet, the panel shows a download prompt. Click Download and wait for the progress bar to complete (~103 MB).
Select a clip
Click a video clip in the timeline. SAM2 works on the frame at the current playhead position.
Activate segmentation mode
Click the Activate button in the panel. A crosshair cursor and a semi-transparent overlay appear on the preview canvas.
Click to place points
- Left-click on an object to add a foreground point (green) — includes that region in the mask.
- Right-click to add a background point (red) — excludes that region from the mask.
Adjust the mask display
Use the display settings in the panel to refine how the mask looks:
| Setting | Range | Description |
|---|---|---|
| Opacity | 0–100% | Transparency of the mask overlay |
| Feather | 0–50 px | Edge softness |
| Invert mask | On/Off | Swap foreground and background regions |
Propagate across frames
Click Forward to propagate the mask through up to 150 subsequent frames (~5 seconds at 30 fps). A progress bar tracks propagation. Click Stop to cancel at any time.SAM2 uses a memory bank mechanism to track the object across frames.
Model details
| Detail | Value |
|---|---|
| Model | SAM 2 Hiera Small (fp16 encoder + ONNX decoder) |
| Total download size | ~103 MB (encoder ~82 MB, decoder ~21 MB) |
| Storage location | Browser Origin Private File System (OPFS) |
| Compute | WebGPU (encoder), WASM (decoder) |
| Thread model | Web Worker — UI stays responsive during inference |
| API key required | No |
| Data uploaded | None |
Model lifecycle
| Status | What you see |
|---|---|
| Not downloaded | Download prompt in the panel |
| Downloading | Progress bar with percentage |
| Loading | ONNX inference sessions initializing |
| Ready | Green status dot — ready for segmentation |
| Error | Red status dot with error message and retry button |
Performance notes
- The first run downloads ~103 MB. On a typical broadband connection this takes 10–30 seconds.
- After the initial download, the model loads from the OPFS cache in a few seconds.
- WebGPU acceleration significantly speeds up the encoder. On hardware without WebGPU support in the worker context, encoding falls back to WASM and will be slower.
- Propagation processes up to 150 frames per run. For very long clips, run multiple propagation passes or use Stop to apply a partial result.