Overview
Thegui/gui.py script provides a graphical interface for controlling the robot via ROS2. It features:
- Live camera feed from robot’s perspective
- Real-time body state visualization (position, orientation)
- Gamepad/joystick control integration
- XY trajectory plotting
- User authentication system
Prerequisites
Connect Gamepad (Optional)
Connect a USB gamepad or Bluetooth controller before starting the GUI.Supported controllers:
- Xbox controllers
- PlayStation DualShock/DualSense
- Generic USB gamepads
- Android gamepad apps (via Bluetooth)
Launching the GUI
- With Simulation
- Standalone (Testing)
Launch both the simulation and GUI:Terminal 1 (Simulation):Terminal 2 (GUI):The GUI window opens with a login screen.
User Authentication
The GUI includes a role-based login system.Default Credentials
Credentials are stored ingui/users.db (SQLite database):
| Username | Password | Role |
|---|---|---|
| admin | admin | owner |
| user | user | user |
Login Process
Access Operation Tab
After successful login:
- Switches to Operation tab automatically
- Login tab becomes disabled
- Interface configured based on user role
Interface Overview
After login, the main interface has three panels:Left Panel
Camera Feed
- Live view from robot camera
- 640×480 resolution
- Updates at 10 Hz
- Scales to fit window
Center Panel
Body State Display
- Position: X, Y, Z (meters)
- Orientation: Roll, Pitch, Yaw (degrees)
- Updates at 10 Hz
- XY trajectory plot (last 50 positions)
Right Panel
Control Interface
- D-pad direction indicators
- Arrow icons (orange = active)
- Shows current input state
- Button press indicators
Gamepad Controls
The GUI polls the gamepad at 10ms intervals (gui.py:151).
Control Mapping
- D-Pad / Hat
- Analog Sticks
Directional Controls (Hat 0)
Visual feedback: Arrow icons turn orange when pressed.
| Input | Action | ROS2 Command |
|---|---|---|
| ↑ Up | Move forward | movement_command = 1 |
| ↓ Down | Move backward | movement_command = 2 |
| ← Left | Reserved | (Not implemented) |
| → Right | Reserved | (Not implemented) |
| Center | Stop | movement_command = 0 |
How It Works
Gamepad polling implementation (gui.py:394-491):
Display Features
Camera Feed
Implementation:gui.py:331-358
The camera feed updates every 100ms:
- Receives
sensor_msgs/Imagefrom/robot_cameratopic - Converts ROS Image → OpenCV array (via cv_bridge)
- Converts OpenCV array → QImage
- Converts QImage → QPixmap
- Scales to fit label while preserving aspect ratio
- Displays in
camera_labelwidget
Body State
Implementation:gui.py:360-392
Displays real-time robot pose:
XY Trajectory Plot
Implementation:gui.py:200-226, gui.py:377-391
Real-time 2D trajectory visualization:
- Buffer size: Last 50 positions
- Update rate: 10 Hz (100ms)
- Rendering: PyQtGraph scatter plot + line trail
- Style: Blue points with connecting lines
- Auto-scaling axes
- Grid overlay
- Smooth transitions as buffer fills
- Oldest points dropped when buffer full
Troubleshooting
Gamepad Not Detected
Gamepad Not Detected
Symptoms: Console shows “No joystick detected”Solutions:
- Check physical connection (USB/Bluetooth)
- Test with jstest:
- Verify pygame detects it:
- Try reconnecting the controller
- Restart the GUI after connecting
The GUI continues to work without a gamepad (keyboard control via ROS2 commands).
Arrow Icons Not Changing
Arrow Icons Not Changing
Symptoms: D-pad arrows stay brown when pressedCauses:This reveals your controller’s button mapping.Fix: Edit
- Wrong button mapping: Your controller uses different indices
- Image files missing: Check
gui/icons/directory - Polling too slow: Increase timer frequency
gui.py:410-426 to match your controller’s hat/axis configuration.Camera Feed Frozen
Camera Feed Frozen
Symptoms: Camera shows old/static imageChecks:
- Verify
sim.pyis running - Check topic rate:
Should show ~10 Hz
- Look for errors in GUI terminal
- Verify cv_bridge is installed:
Login Failed
Login Failed
Symptoms: “Usuario o contraseña incorrectos” errorSolutions:
- Try default credentials:
- Username:
admin, Password:admin - Username:
user, Password:user
- Username:
- Check database exists:
- Verify database contents:
- Recreate database if corrupted (see Database Management below)
UI Elements Missing
UI Elements Missing
Symptoms: Panels not visible or misalignedCauses:
.uifile not found: Checkgui/untitled.uiexists- PyQt5 version mismatch
- Screen resolution too low
Database Management
The user database is stored ingui/users.db.
Viewing Users
Adding Users
Changing Passwords
Recreating Database
If the database becomes corrupted:Customization
Changing Camera Update Rate
Editgui.py:161:
50= 20 Hz (smoother, more CPU)100= 10 Hz (balanced)200= 5 Hz (lower CPU)
Adjusting Trajectory Buffer Size
Editgui.py:183:
Custom Controller Mapping
If your gamepad uses different button indices:- Run GUI and press buttons
- Note console output:
Button X pressed - Edit
gui.py:458to map your restart button:
Performance Tips
Reduce CPU Usage
- Increase timer intervals (camera: 200ms, ROS: 20ms)
- Reduce trajectory buffer size (25 instead of 50)
- Lower camera resolution in
sim.py:233
Improve Responsiveness
- Decrease joystick polling interval (5ms instead of 10ms)
- Increase ROS spin frequency (5ms instead of 10ms)
- Use wired connection instead of Bluetooth
Next Steps
ROS2 Deep Dive
Learn about the underlying ROS2 architecture
Performance Testing
Compare different control strategies quantitatively