Autopilot Types
1. Deep Learning Autopilot
The most popular approach, using neural networks trained on driving data to predict steering and throttle. How it works:- Collect training data by manually driving your car
- Train a neural network model using TensorFlow or PyTorch
- Deploy the trained model for autonomous driving
- Complex tracks with varying conditions
- Racing scenarios requiring smooth control
- Users comfortable with machine learning workflows
2. Computer Vision Autopilot
Uses OpenCV and traditional computer vision techniques to follow lines or detect features. How it works:- Configure color thresholds and image processing parameters
- Use PID controllers to follow detected lines
- No training data required
- Simple line-following tasks
- Testing and prototyping
- Environments where collecting training data is difficult
- Users who want immediate results without training
3. GPS Path Following Autopilot
Records and follows GPS waypoint paths using position sensors like GPS modules or Intel RealSense T265. How it works:- Drive manually to record a path of waypoints
- Save the path with position and optionally throttle data
- Follow the recorded path using PID control and cross-track error
- Outdoor racing with GPS coverage
- Precise path following
- Repeatable autonomous missions
- Indoor odometry with wheel encoders or T265
Choosing an Autopilot Type
| Criteria | Deep Learning | Computer Vision | GPS Path Following |
|---|---|---|---|
| Training Required | Yes (20+ min driving) | No | Yes (1 lap) |
| Setup Complexity | Medium | Low | Medium-High |
| Track Adaptability | High | Low | Medium |
| Hardware Required | Camera | Camera | GPS/T265/Encoders |
| Performance | Excellent | Good | Excellent |
| Tuning Required | Minimal | Significant | Moderate |
Data Collection
For both deep learning and GPS path following, you’ll need to collect training data: Deep Learning:- Drive smoothly for 10-20 minutes
- Collect diverse examples (centerline, recovery, turns)
- Aim for 10,000-20,000 frames
- Drive one complete lap of your track
- Maintain consistent speed
- Start and end at approximately the same position
Training Workflow
Typical deep learning training workflow:- Collect Data - Drive manually to record training data
- Review Data - Clean and filter bad frames using
donkey tubclean - Train Model - Run
donkey trainto create a model - Test Model - Deploy and test in autopilot mode
- Iterate - Collect more data or adjust hyperparameters
Model Architectures
Donkeycar includes several pre-built model architectures:- Linear - Simple CNN outputting continuous steering/throttle
- Categorical - CNN with categorical outputs (discrete bins)
- LSTM - Recurrent network using temporal sequences
- 3D CNN - 3D convolutions over image sequences
- IMU - Multi-input model combining camera and IMU data
- Behavioral - Conditional model for multiple driving behaviors
Next Steps
Data Collection
Learn how to collect high-quality training data
Deep Learning
Train neural network models with TensorFlow/PyTorch
Computer Vision
Configure line-following with OpenCV
GPS Autopilot
Record and follow GPS waypoint paths