ARM Demos is a collection of ROS 2 (Jazzy) applications built around the Colman platform — a Universal Robots UR3e arm equipped with a vacuum gripper and a wrist-mounted OAK-D Pro camera. The suite is designed for lab use and rapid experimentation: every dependency is pinned and installed through Pixi, so you get a fully reproducible environment with a single command on Linux.Documentation Index
Fetch the complete documentation index at: https://mintlify.com/MRRP-lab/arm-demos/llms.txt
Use this file to discover all available pages before exploring further.
What’s included
Three demos ship with the project, each exercising a different part of the robotics stack.Hand tracking
Control the arm in real time by moving your hand in front of the OAK-D camera. MediaPipe detects 21 hand landmarks at 30 fps; MoveIt 2 translates fingertip position and palm depth into Cartesian goals for the UR3e.
Pick and place
The arm scans the workspace using AprilTag markers detected through the OAK-D camera, plans a collision-free approach with MoveIt 2, and uses the vacuum gripper to pick and deposit objects autonomously.
Eye-in-hand calibration
Calibrate the transform between the wrist-mounted OAK-D Pro camera and the robot’s end-effector so that camera detections map accurately to robot-frame coordinates.
Key technologies
| Technology | Role |
|---|---|
| ROS 2 Jazzy | Middleware and communication layer |
| MoveIt 2 | Motion planning, collision checking, trajectory execution |
| MediaPipe | Real-time hand landmark detection (hand tracking demo) |
| DepthAI / OAK-D Pro | Depth-capable camera on the robot wrist |
| YOLO (yolo11m-seg) | Object detection and segmentation (colman_perception) |
| AprilTag | Fiducial marker-based object localisation (pick-and-place) |
| Pixi | Conda/PyPI package manager for reproducible environments |
Source packages
The repository is a colcon workspace. All packages live undersrc/.
arm_demo
The hand tracking demo node. Reads OAK-D frames, runs MediaPipe, and sends MoveIt 2 pose goals in a background thread.
colman_bringup
Launch files for the full robot stack: robot state publisher, ros2_control, MoveIt 2, camera, AprilTag, and YOLO nodes.
colman_description
URDF/Xacro robot description and ros2_control configuration for the UR3e with vacuum gripper and OAK-D mount.
colman_motion
Pick-and-place and hand-eye calibration nodes. Uses MoveIt 2 for planning and the UR IO interface for vacuum control.
colman_perception
Camera publishing, AprilTag detection, and YOLO segmentation nodes. Subscribes to
/oakd_pro/rgb/image.colman_moveit_config
MoveIt 2 configuration: SRDF, OMPL pipeline, kinematics (trac-ik), and
moveit_py.yaml.gripper_description
Xacro macros for the vacuum gripper geometry and ros2_control hardware interface.
gripper_gazebo
Gazebo plugins and materials for vacuum gripper simulation.
oakd_description
URDF description for the OAK-D Pro camera, including the wrist mount transform.
oakd_gazebo
Gazebo sensor plugins for simulating OAK-D RGB and depth output.
Hardware
The Colman platform consists of:- Universal Robots UR3e — 6-DOF collaborative arm reachable at
192.168.56.101on the lab LAN - Vacuum gripper — digital output controlled via the UR IO interface (
/io_and_status_controller/set_io) - Luxonis OAK-D Pro — wrist-mounted RGB-D camera; accessed through the DepthAI SDK and published as ROS 2 topics under
/oakd_pro/
Next steps
Quickstart
Clone the repo, build with Pixi, and run your first demo in minutes.
Environment setup
Full instructions for installing Pixi and preparing your development machine.
Hardware setup
Connect and configure the UR3e, OAK-D camera, and vacuum gripper.
Simulation setup
Run demos against mock hardware without a physical robot.