Skip to main content

​
Overview

Level 3 debugging in AlpaSim enables breakpoint debugging of individual components while running a full simulation. This allows you to debug complex interactions between microservices with your favorite debugger attached.
This section covers debugging simulation components (controller, runtime, driver), not vehicle behavior within simulation. For basic configuration debugging, see the Quickstart Guide.

​
Breakpoint Debugging Workflow

The basic approach to enable breakpoint debugging:
1

Generate config files without running

Use the wizard to create configuration files but skip execution:
alpasim_wizard +deploy=local \
  wizard.log_dir=$PWD/tutorial_dbg \
  wizard.run_method=NONE \
  wizard.debug_flags.use_localhost=True
2

Start the component manually

Launch the component you want to debug from your IDE or terminal with breakpoints set
3

Run the rest of the simulation

Start remaining services via docker-compose, excluding the manually-started component

​
Debugging the Controller

The controller manages ego vehicle trajectory tracking and pose estimation.

​
Terminal-Based Debugging

1

Generate configs

alpasim_wizard +deploy=local \
  wizard.log_dir=$PWD/tutorial_dbg \
  wizard.run_method=NONE \
  wizard.debug_flags.use_localhost=True
2

Note the allocated port

Check tutorial_dbg/docker-compose.yaml for the controller port. Look for components in the sim profile: controller-0, driver-0, physics-0, runtime-0, sensorsim-0.Example: controller-0 might be allocated port 6003.
3

Start controller with debugger

From src/controller/, start the controller manually:
cd src/controller/
mkdir my_controller_log_dir
# Port must match docker-compose.yaml
uv run python -m alpasim_controller.server \
  --port=6003 \
  --log_dir=my_controller_log_dir \
  --log-level=INFO
4

Launch remaining services

From the generated directory:
cd tutorial_dbg
docker compose -f docker-compose.yaml --profile sim up \
  runtime-0 driver-0 physics-0 sensorsim-0

​
VSCode Debugger Configuration

For a better debugging experience, use VSCode’s built-in debugger: 
{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Debug Controller (Level 3 Tutorial)",
      "type": "debugpy",
      "request": "launch",
      "module": "alpasim_controller.server",
      "justMyCode": false,
      "cwd": "${workspaceFolder}/src/controller",
      "args": [
        "--port=6003",
        "--log_dir=my_controller_logdir",
        "--log-level=INFO"
      ],
      "console": "integratedTerminal"
    }
  ]
}
1

Set breakpoints

Open controller source files and click in the gutter to set breakpoints
2

Launch debugger

Press F5 or go to Run and Debug → “Debug Controller”
3

Start simulation

Breakpoints will hit as the simulation runs!
Make sure the --port argument matches the port allocated in docker-compose.yaml.

​
Debugging the Runtime

Debugging the runtime requires a different approach since other services must be running before the runtime starts.

​
Key Differences

  • Start order: Other services must be up before launching the runtime
  • Speed iteration: Set runtime.endpoints.do_shutdown=False to keep containers running between runs

​
Terminal-Based Debugging

1

Generate configs with no shutdown

alpasim_wizard +deploy=local \
  wizard.log_dir=$PWD/tutorial_dbg_runtime \
  wizard.run_method=NONE \
  wizard.debug_flags.use_localhost=True \
  runtime.endpoints.do_shutdown=False
2

Start non-runtime services

cd tutorial_dbg_runtime
docker compose -f docker-compose.yaml --profile sim up \
  driver-0 controller-0 physics-0 sensorsim-0
3

Start runtime manually

From src/runtime/:
cd src/runtime/
uv run python -m alpasim_runtime.simulate \
  --usdz-glob=../../data/nre-artifacts/all-usdzs/**/*.usdz \
  --user-config=../../tutorial_dbg_runtime/generated-user-config-0.yaml \
  --network-config=../../tutorial_dbg_runtime/generated-network-config.yaml \
  --log-dir=../../tutorial_dbg_runtime \
  --log-level=INFO

​
VSCode Debugger Configuration

{
  "name": "Debug Runtime (Level 3 Tutorial)",
  "type": "debugpy",
  "request": "launch",
  "module": "alpasim_runtime.simulate",
  "justMyCode": false,
  "cwd": "${workspaceFolder}/src/runtime",
  "args": [
    "--usdz-glob=../../data/nre-artifacts/all-usdzs/**/*.usdz",
    "--user-config=../../tutorial_dbg_runtime/generated-user-config-0.yaml",
    "--network-config=../../tutorial_dbg_runtime/generated-network-config.yaml",
    "--log-dir=../../tutorial_dbg_runtime",
    "--log-level=INFO"
  ],
  "console": "integratedTerminal"
}

​
ASL Log Analysis

​
What are ASL Logs?

ASL (AlpaSim Log) files contain most messages exchanged during simulation as size-delimited protobuf messages. They’re stored in rollouts/{scene_id}/{batch_uuid}/rollout.asl. Use cases:
  • Debugging model behavior
  • Replaying stimuli on a driver instance
  • Reproducing bugs with a debugger attached
  • Creating evaluation videos

​
Reading ASL Logs

Use the async_read_pb_log utility to stream messages: 
from alpasim_grpc.utils.logs import async_read_pb_log

i = 0
async for log_entry in async_read_pb_log("path/to/rollout.asl"):
    print(log_entry)
    i += 1
    if i == 20:
        break

​
Message Types

ASL files contain three types of messages:
Metadata header for reproducibility and bookkeeping:
  • Session UUID and scene ID
  • Batch size and simulation steps
  • Timestamp and control frequency
  • Version information (runtime, driver, API)
  • Actor definitions
Location of all actors (including 'EGO') in global coordinate space:
  • Position (x, y, z)
  • Orientation (quaternion)
  • Velocity
  • Timestamp
Microservice RPC calls enable replay mode:
  • RolloutCameraImage - Camera frame requests (used to create MP4 videos)
  • Driver planning requests/responses
  • Physics update calls
  • Sensor simulation requests

​
Replaying Stimuli

The replay_logs_alpamodel.ipynb notebook shows how to:
  1. Read an ASL log
  2. Extract driver stimuli (camera frames, ego poses)
  3. Replay them on a driver instance
  4. Reproduce behavior with a debugger attached
ASL logs don’t specify the USDZ file UUID - you’ll need to match by scene_id from metadata.

​
Console Log Analysis

The docker-compose console contains logs from all microservices and is the first place to look when something goes wrong.

​
Basic Troubleshooting

1

Check for missing output

If the rollouts directory doesn’t appear, check console logs for errors
2

Find first error

Scroll to find where the first errors occurred - subsequent errors are often cascading failures
3

Service-specific logs

Additional logs may be in:
  • txt-logs/ - Per-service text logs
  • driver/ - Driver service logs
  • telemetry/ - Performance metrics
Microservices may produce additional debug logs not shown in the console. Check txt-logs/ for detailed service output.

Build docs developers (and LLMs) love

Get started for freeTalk to us