Skip to main content

​
Overview

The run_batch module executes both audio-based (GCC-PHAT) and visual-based (motion correlation) synchronization on all synthetic test cases, recording accuracy, confidence, runtime, and resource usage metrics.

​
Functions

​
run_batch

run_batch(
    metadata_csv: str = METADATA_CSV,
    originals_dir: str = ORIGINALS_DIR,
    results_dir: str = RESULTS_DIR,
    diagnostics_dir: str = DIAGNOSTICS_DIR,
) -> str
Run audio and visual synchronization on every synthetic test case.
​
metadata_csv
str
default:"evaluation/metadata/synthetic_metadata.csv"
Path to the synthetic metadata CSV generated by offset_generation.
​
originals_dir
str
default:"evaluation/originals"
Directory containing original source videos.
​
results_dir
str
default:"evaluation/results"
Output directory for the results CSV.
​
diagnostics_dir
str
default:"evaluation/diagnostics"
Output directory for motion signal diagnostics (NPZ files for visualization).
Returns: Path to the output results CSV file. Output Files:
  • results_dir/results.csv with columns:
    • video_id: Video identifier
    • true_offset_ms: Ground truth offset
    • method_type: “audio” or “visual”
    • estimated_offset_ms: Estimated offset by the method
    • absolute_error_ms: |estimated - true|
    • confidence_score: Confidence metric (0-1)
    • runtime_seconds: Execution time
    • peak_cpu_percent: Peak CPU usage during sync
    • peak_memory_mb: Peak memory usage during sync
    • video_length_sec: Video duration
    • motion_level: Motion energy level
    • audio_energy_level: Audio RMS energy level
  • diagnostics_dir/*.npz: Motion signals for before/after visualization

​
Helper Functions

​
_run_audio_sync

_run_audio_sync(original_path: str, synthetic_path: str) -> dict
Run audio-based synchronization between two videos using GCC-PHAT.
​
original_path
str
Path to the original video.
​
synthetic_path
str
Path to the synthetic (shifted) video.
Returns: Dictionary with keys:
  • estimated_offset_ms: Estimated offset in milliseconds
  • confidence_score: GCC-PHAT peak confidence
  • runtime_seconds: Execution time
  • peak_cpu_percent: Peak CPU usage
  • peak_memory_mb: Peak memory usage
Raises: RuntimeError if either video lacks an audio stream.

​
_run_visual_sync

_run_visual_sync(
    original_path: str,
    synthetic_path: str,
    diagnostics_dir: str | None = None,
    case_label: str = "",
) -> dict
Run visual-based synchronization between two videos using motion correlation.
​
original_path
str
Path to the original video.
​
synthetic_path
str
Path to the synthetic (shifted) video.
​
diagnostics_dir
str | None
default:"None"
If provided, saves motion signals as NPZ files for later visualization.
​
case_label
str
default:"''"
Label for the diagnostic NPZ file (e.g., “video1_offset+500”).
Returns: Dictionary with keys:
  • estimated_offset_ms: Estimated offset in milliseconds
  • confidence_score: Motion correlation peak confidence
  • runtime_seconds: Execution time
  • peak_cpu_percent: Peak CPU usage
  • peak_memory_mb: Peak memory usage

​
ResourceMonitor

class ResourceMonitor:
    def __init__(self, interval: float = 0.2)
Context manager that samples CPU and memory usage in a background thread.
​
interval
float
default:"0.2"
Sampling interval in seconds.
Properties:
  • peak_cpu_percent: Maximum CPU percentage observed
  • peak_memory_mb: Maximum memory usage in MB observed
Usage: 
with ResourceMonitor() as monitor:
    # ... do work ...
    pass
print(f"Peak CPU: {monitor.peak_cpu_percent}%")
print(f"Peak Memory: {monitor.peak_memory_mb} MB")

​
_read_metadata

_read_metadata(csv_path: str) -> list
Read the synthetic metadata CSV and return list of dicts.
​
csv_path
str
Path to the metadata CSV file.
Returns: List of dictionaries, one per row.

​
_find_original_video

_find_original_video(video_id: str, originals_dir: str) -> str
Find the original video file matching video_id in originals_dir.
​
video_id
str
Video identifier (filename without extension).
​
originals_dir
str
Directory to search for the original video.
Returns: Full path to the original video. Raises: FileNotFoundError if no matching video is found.

​
_has_audio_stream

_has_audio_stream(video_path: str) -> bool
Check whether the video file contains an audio stream using ffprobe.
​
video_path
str
Path to the video file.
Returns: True if audio stream exists, False otherwise.

​
Usage Example

import logging
from evaluation.run_batch import run_batch

logging.basicConfig(level=logging.INFO)

# Run batch synchronization
results_csv = run_batch(
    metadata_csv="evaluation/metadata/synthetic_metadata.csv",
    originals_dir="evaluation/originals",
    results_dir="evaluation/results",
    diagnostics_dir="evaluation/diagnostics",
)

print(f"Results written to: {results_csv}")

​
CLI Usage

python -m evaluation.run_batch
Runs with default settings and writes results to evaluation/results/results.csv.

​
Configuration

​
Default Paths

BASE_DIR = os.path.dirname(os.path.abspath(__file__))
METADATA_CSV = os.path.join(BASE_DIR, "metadata", "synthetic_metadata.csv")
ORIGINALS_DIR = os.path.join(BASE_DIR, "originals")
RESULTS_DIR = os.path.join(BASE_DIR, "results")
DIAGNOSTICS_DIR = os.path.join(BASE_DIR, "diagnostics")

​
Result CSV Fields

RESULT_FIELDNAMES = [
    "video_id",
    "true_offset_ms",
    "method_type",
    "estimated_offset_ms",
    "absolute_error_ms",
    "confidence_score",
    "runtime_seconds",
    "peak_cpu_percent",
    "peak_memory_mb",
    "video_length_sec",
    "motion_level",
    "audio_energy_level",
]

​
Notes

  • Method Types: Each test case is evaluated with both "audio" and "visual" methods.
  • Sign Convention: Estimated offsets follow the same sign convention as true offsets (positive = prepend, negative = trim).
  • Temporary Directories: Uses tempfile.mkdtemp() for isolated working directories per test case.
  • Error Handling: Logs failures for individual test cases but continues processing remaining cases.
  • Progress Logging: Outputs detailed progress including estimated offset, error, confidence, runtime, and resource usage for each test case.

Build docs developers (and LLMs) love

Get started for freeTalk to us