Documentation Index Fetch the complete documentation index at: https://mintlify.com/zeroclaw-labs/zeroclaw/llms.txt
Use this file to discover all available pages before exploring further.
Robot Kit API
Complete API reference for the six core robot tools.
Tool Inputs Outputs Hardware Latency driveaction, distance/angle confirmation Motors 100ms-10s lookaction, prompt (optional) image description Camera 2-5s listenduration (seconds) transcribed text Microphone 1-10s speaktext confirmation Speaker 1-5s senseaction, direction sensor readings LIDAR, PIR, ultrasonic 100ms-1s emoteexpression, sound (optional) confirmation LED matrix, buzzer 100-500ms
drive
Omni-directional motor control with safety integration.
Parameters
{
"action" : "forward" | "backward" | "strafe_left" | "strafe_right" | "rotate_left" | "rotate_right" | "stop" ,
"distance" : 1.0 , // meters (for linear movement)
"angle" : 90 , // degrees (for rotation)
"speed" : 0.5 // 0.0-1.0, optional, defaults to config max_speed
}
Examples
{
"action" : "forward" ,
"distance" : 1.5
}
Response
{
"success" : true ,
"output" : "Moved forward 1.5 meters"
}
Backends
The drive tool supports multiple motor control backends:
Backend Description Configuration mock Logs commands, no hardware backend = "mock"serial Arduino/motor controller backend = "serial", serial_port = "/dev/ttyACM0"ros2 ROS2 nav stack backend = "ros2", ros2_topic = "/cmd_vel"gpio Direct GPIO control backend = "gpio" (requires peripheral-rpi feature)
Safety Integration
When wrapped with SafeDrive, the tool:
Pre-movement check : Queries sensors for obstacles
Real-time monitoring : Adjusts speed based on proximity
Collision response : Reverses if bump sensor triggered
Watchdog : Auto-stops after timeout
E-stop : Immediate halt on hardware button press
Always use SafeDrive wrapper in production. Direct DriveTool bypasses safety checks.
look
Capture images and generate AI descriptions using vision models.
Parameters
{
"action" : "capture" | "describe" | "find" ,
"prompt" : "a red ball" , // optional, for "find" action
"save_path" : "/tmp/image.jpg" // optional, defaults to temp file
}
Examples
Describe Scene
Find Object
Just Capture
Response
{
"success" : true ,
"output" : "I see a couch, a TV on the wall, and some toys on the floor. The room is well-lit with natural light from a window."
}
Vision Models
Supported models (via Ollama):
Model Size Speed Accuracy Recommended For moondream 1.7B Fast Good Pi 4/5, general use llava 7B Slow Better Pi 5 only, detailed scenes bakllava 7B Slow Better Alternative to llava
Configure in robot.toml:
[ camera ]
vision_model = "moondream"
ollama_url = "http://localhost:11434"
listen
Transcribe speech to text using Whisper.cpp.
Parameters
{
"duration" : 5 , // seconds to record
"language" : "en" // optional, defaults to "en"
}
Examples
Listen 5 Seconds
Listen 10 Seconds
Response
{
"success" : true ,
"output" : "Hey Buddy, can you help me find my toy?"
}
Whisper Models
Model Size Speed Accuracy RAM Usage tiny 75 MB Fastest Fair 200 MB base 142 MB Fast Good 400 MB small 466 MB Medium Better 1 GB medium 1.5 GB Slow Best 2.5 GB
Recommendation: base for Pi 4/5
Configure in robot.toml:
[ audio ]
whisper_model = "base"
whisper_path = "/usr/local/bin/whisper-cpp"
speak
Text-to-speech using Piper TTS.
Parameters
{
"text" : "Hello! I'm ready to play!" ,
"voice" : "en_US-lessac-medium" // optional, defaults to config
}
Examples
Simple Speech
Custom Voice
{
"text" : "I found the ball under the couch!"
}
Response
{
"success" : true ,
"output" : "Spoke: Hello! I'm ready to play!"
}
Piper Voices
Available voices:
Voice Gender Style Speed en_US-lessac-medium Male Clear Medium en_US-amy-medium Female Friendly Medium en_US-ryan-high Male Energetic Fast en_GB-alan-medium Male British Medium
Download from Piper Voices
sense
Query sensors: LIDAR, motion (PIR), ultrasonic distance.
Parameters
{
"action" : "scan" | "motion" | "distance" ,
"direction" : "front" | "all" , // for scan action
"max_range" : 5.0 // meters, optional
}
Examples
LIDAR Scan
Motion Detection
Distance Check
{
"action" : "scan" ,
"direction" : "front"
}
Response
{
"success" : true ,
"output" : "Front: clear (2.3m), Left: obstacle at 0.8m, Right: clear (3.1m)"
}
Sensor Types
Sensor Range Accuracy Use Case RPLidar A1 12m, 360° ±5cm Obstacle avoidance, mapping HC-SR04 4m, narrow ±1cm Front distance check PIR Motion 7m, 120° Binary Detect people/animals
Configure in robot.toml:
[ sensors ]
lidar_port = "/dev/ttyUSB0"
lidar_type = "rplidar"
motion_pins = [ 17 , 27 ]
ultrasonic_pins = [ 23 , 24 ]
emote
Display LED expressions and play sound effects.
Parameters
{
"expression" : "happy" | "sad" | "excited" | "thinking" | "sleepy" | "surprised" ,
"sound" : "beep" | "chime" | "buzz" // optional
}
Examples
Happy Face
Excited with Sound
Thinking
{
"expression" : "happy"
}
Response
{
"success" : true ,
"output" : "Displayed: happy face"
}
LED Patterns
8x8 LED matrix patterns (WS2812B):
Happy: Sad: Excited: Thinking:
● ● ● ● ● ● ● ●
● ● ● ● ● ● ○ ●
●●●●● ●●●●● ●●●●● ●●●
Error Handling
All tools return a ToolResult:
pub struct ToolResult {
pub success : bool ,
pub output : String ,
pub error : Option < String >,
}
Common Errors
{
"success" : false ,
"output" : "" ,
"error" : "Obstacle detected at 0.2m, stopping"
}
{
"success" : false ,
"output" : "" ,
"error" : "Camera device /dev/video0 not found"
}
{
"success" : false ,
"output" : "" ,
"error" : "E-stop pressed, all movement disabled"
}
Usage Examples
Hide and Seek
use zeroclaw_robot_kit ::* ;
#[tokio :: main]
async fn main () {
let config = RobotConfig :: default ();
let tools = create_tools ( & config );
// Game sequence
speak ( "I'll count to 20. Go hide!" );
emote ( "excited" );
tokio :: time :: sleep ( Duration :: from_secs ( 20 )) . await ;
speak ( "Ready or not, here I come!" );
loop {
let obstacles = sense ( "scan" , "all" );
drive ( "forward" , 1.0 );
let scene = look ( "find" , "a child hiding" );
if scene . contains ( "child" ) {
speak ( "Found you!" );
emote ( "happy" , Some ( "chime" ));
break ;
}
}
}
Voice-Controlled Movement
loop {
speak ( "Where should I go?" );
let command = listen ( 5 ) . await ? ;
match command . as_str () {
cmd if cmd . contains ( "forward" ) => drive ( "forward" , 1.0 ),
cmd if cmd . contains ( "back" ) => drive ( "backward" , 0.5 ),
cmd if cmd . contains ( "left" ) => drive ( "rotate_left" , 90 ),
cmd if cmd . contains ( "right" ) => drive ( "rotate_right" , 90 ),
cmd if cmd . contains ( "stop" ) => {
drive ( "stop" , 0.0 );
break ;
},
_ => speak ( "I didn't understand that" ),
}
}
Autonomous Patrol
loop {
// Check for obstacles
let scan = sense ( "scan" , "front" );
if scan . min_distance < 0.5 {
speak ( "Obstacle ahead, turning" );
drive ( "rotate_right" , 90 );
continue ;
}
// Check for motion
let motion = sense ( "motion" );
if motion . detected {
speak ( "I see movement!" );
let scene = look ( "describe" );
speak ( & format! ( "I see: {}" , scene ));
}
// Continue patrol
drive ( "forward" , 2.0 );
}
Safety Monitor
The SafetyMonitor runs as an independent task:
use zeroclaw_robot_kit :: { SafetyMonitor , SafeDrive , DriveTool };
let config = RobotConfig :: default ();
// Create safety monitor
let ( safety , safety_rx ) = SafetyMonitor :: new ( config . safety . clone ());
let safety = Arc :: new ( safety );
// Start sensor feed
tokio :: spawn ( async move {
loop {
let reading = get_lidar_data () . await ;
safety . update_sensor ( reading ) . await ;
}
});
// Wrap drive with safety
let drive = Arc :: new ( DriveTool :: new ( config . clone ()));
let safe_drive = SafeDrive :: new ( drive , safety . clone ());
// Now all drive commands go through safety checks
safe_drive . execute ( json! ({ "action" : "forward" , "distance" : 2.0 })) . await ;
Safety Events
pub enum SafetyEvent {
ObstacleDetected { distance : f64 , angle : i32 },
BumpSensorTriggered { sensor : String },
EStopPressed ,
SensorTimeout ,
SpeedLimited { from : f64 , to : f64 , reason : String },
MovementBlocked { reason : String },
}
Troubleshooting
Drive not working
# Check backend
grep backend ~/.zeroclaw/robot.toml
# Serial: verify port
ls /dev/ttyACM *
# ROS2: check topic
ros2 topic list | grep cmd_vel
# Test in mock mode
sed -i 's/backend = ".*"/backend = "mock"/' ~/.zeroclaw/robot.toml
Camera not found
# List cameras
ls /dev/video *
ffmpeg -f v4l2 -list_formats all -i /dev/video0
# Check permissions
sudo usermod -aG video $USER
Ollama vision fails
# Check Ollama is running
curl http://localhost:11434/api/tags
# Test vision model
ollama run moondream "Describe this image: /tmp/test.jpg"
# Use smaller model
sed -i 's/vision_model = ".*"/vision_model = "moondream"/' ~/.zeroclaw/robot.toml
Safety prevents movement
# Check sensor readings
tail -f /tmp/zeroclaw_sensors.fifo
# Increase safety thresholds (testing only!)
sed -i 's/min_obstacle_distance = 0.3/min_obstacle_distance = 0.15/' ~/.zeroclaw/robot.toml
# Disable safety (dangerous!)
sed -i 's/predict_collisions = true/predict_collisions = false/' ~/.zeroclaw/robot.toml
Next Steps
Setup Guide Complete hardware and software setup
Overview Architecture and features
Examples Full example programs
Source Code View robot kit source