Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/commaai/openpilot/llms.txt

Use this file to discover all available pages before exploring further.

openpilot is a Linux-based robotics operating system that runs on the comma four device. Rather than a monolithic application, it is structured as a collection of independent processes that communicate through a typed pub/sub messaging layer. Each process has a narrow responsibility — capturing camera frames, running a neural network, computing actuator commands, or logging data — and all coordination happens through message passing rather than shared state.

​
Hardware

The following hardware components make up the standard openpilot setup.
ComponentRole
comma fourARM Linux SoC running AGNOS (based on Ubuntu). Hosts all software processes, cameras, IMU, and connectivity.
pandaCAN gateway between the vehicle’s bus and openpilot. Enforces safety constraints in firmware. Connected via USB or internal SPI.
Road cameraForward-facing wide-angle camera used by the driving model.
Wide road cameraAdditional wide-angle camera for increased field of view.
Driver cameraInterior-facing camera used by the driver monitoring model.
GPS / GNSSu-blox or Qualcomm GNSS receiver for position and velocity data.
IMULSM6DS3 accelerometer and gyroscope sampled at 104 Hz via I²C.

​
Software layers

openpilot organises its software into three layers. Manager — system/manager/manager.py is the init process for openpilot. It launches, monitors, and restarts all other processes. It subscribes to deviceState and pandaStates to decide which processes should be running (onroad vs offroad) and publishes a managerState heartbeat so the rest of the system can detect if a process has died. Processes — Each major function (perception, planning, control, logging, driver monitoring) lives in its own Python or native process. Processes are defined in system/manager/process_config.py and can be conditioned on the ignition state, car type, or device capabilities. See the process reference for details on each process. Messaging layer (cereal / msgq) — All inter-process communication uses cereal, a Cap’n Proto schema library backed by msgq, a shared-memory pub/sub transport. There is no direct function-call coupling between processes; every piece of data crosses a named socket. See the messaging reference for schema and API details.

​
Data flow

The diagram below shows the high-level flow from sensor input to actuator output. 
camerad  ──(VisionIPC)──►  modeld  ──►  modelV2
                                         │
sensord  ──►  accelerometer/gyroscope    ▼
                                      plannerd  ──►  longitudinalPlan
pandad   ──►  can / carState              │
                                      controlsd ──►  carControl / sendcan
selfdrived ◄──────────────────────────────┘
          └──►  selfdriveState  ──►  controlsd / UI
  1. camerad captures frames from the road and driver cameras and shares them via VisionIPC shared memory.
  2. modeld reads camera frames, runs the supercombo neural network, and publishes modelV2 (path predictions, lead vehicles, lane lines) and cameraOdometry.
  3. pandad reads raw CAN frames from the panda, publishes can, and the car interface converts these to carState.
  4. plannerd consumes modelV2, radarState, and carState to produce longitudinalPlan.
  5. controlsd runs lateral and longitudinal PID/MPC loops, consuming longitudinalPlan, modelV2, and selfdriveState to produce carControl and controlsState.
  6. selfdrived manages the engagement state machine, monitors all system health signals, and publishes selfdriveState and onroadEvents.
  7. pandad reads carControl (via sendcan) and writes the CAN commands back to the panda.

​
Operating environment

openpilot runs on AGNOS, comma’s custom Linux distribution built on Ubuntu for the Snapdragon SoC inside the comma four. Realtime processes use Linux real-time scheduling priorities (SCHED_FIFO) and are pinned to specific CPU cores via config_realtime_process. The panda enforces an independent hardware safety layer: it validates CAN messages against a compiled safety model before forwarding them to the vehicle bus, regardless of what software sends.

​
Key design principles

  • Safety-first — The panda runs safety checks in firmware independently of the host SoC. openpilot cannot command the vehicle if the panda disagrees with the requested safety model or if controlsAllowed is not set.
  • Process isolation — Each daemon runs in its own process. A crash in one process is detected by manager via the managerState message and triggers a restart. selfdrived raises a processNotRunning event if a required process stops, which soft-disables engagement.
  • Typed messaging — All data crosses named Cap’n Proto sockets. There are no ad-hoc binary protocols or shared memory structures between processes (cameras use VisionIPC, a dedicated shared-memory ring buffer).
  • Hardware-in-the-loop testing — openpilot’s CI tests run against real panda hardware to catch regressions in CAN parsing and safety models.
  • Backwards-compatible logs — cereal’s Cap’n Proto schema is designed for forward compatibility; old logs can always be read by newer software.

​
Data logging

loggerd (system/loggerd/loggerd.cc) records every message published on every cereal socket to compressed log segments stored on the device. Camera streams are encoded to H.265 by encoderd and included in the same segment. Each segment covers approximately one minute of driving. The uploader process sends completed segments to comma’s servers when a network connection is available. Logged data includes CAN frames, GPS, IMU, model outputs, and all control state.

Process reference

Detailed reference for every openpilot daemon: purpose, inputs, and outputs.

Messaging reference

cereal schemas, msgq transport, and pub/sub API examples.

Build docs developers (and LLMs) love

Get started for freeTalk to us