Open Source Robotics

A curated collection of high-quality open source robotics projects, tools, and software to propel the robotics community forward.

Optical motion capture

  • Also called: mocap, marker-based motion capture, optical tracking
  • Category: Sensing › Exteroceptive
  • Tags: motion-capture, ground-truth, pose-estimation, cameras, human-motion

Measuring the 3D positions of markers, and from them the pose of rigid bodies or human skeletons, by triangulating their images in several calibrated, synchronized cameras.

Definition

Optical motion capture measures motion by observing markers with several calibrated, synchronized cameras placed around a capture volume. In the most common marker-based form, the cameras emit infrared light from rings around their lenses and see passive retroreflective markers as bright spots; some systems use active LED markers instead. Each camera reports the 2D centroids of the markers it sees, and the system triangulates every marker seen by at least two cameras to a 3D position. Groups of three or more markers attached to a rigid object define a rigid body whose full 6-DoF pose is tracked, and marker sets placed on a person are fitted to a skeleton model to recover joint angles.

The quality of the result depends on calibration and setup. Camera intrinsics and extrinsics are estimated by waving a calibration wand of known geometry through the volume and setting a ground plane, after which a well-calibrated system typically reaches millimeter-level position accuracy at high frame rates. Errors come from occlusion, when too few cameras see a marker; from labeling errors, when the system swaps two nearby markers; from stray reflections; and, for people, from soft-tissue artifact, where skin-mounted markers move relative to the bones underneath. Latency and clock synchronization matter when the data is used for feedback rather than recorded offline. Vicon, OptiTrack and Qualisys are examples of commercial systems.

Marker-based optical motion capture is distinct from markerless capture, which estimates human or object pose from ordinary video with computer vision, and from inertial motion capture, which uses body-worn IMUs and needs no cameras. Optical systems are the usual reference for accuracy, but they are confined to an instrumented space, need markers on every tracked object and are costly, which is why work on wearable or markerless alternatives often uses optical capture as the ground truth to evaluate against.

Why it matters

In robotics labs, optical motion capture provides the ground-truth pose against which odometry, SLAM and state estimators are judged, external pose feedback for drones and small robots, and the human motion data behind retargeting, teleoperation and learning-from-demonstration datasets.

Key applications

  • Ground truth for evaluating odometry, SLAM and state estimation
  • External pose feedback for indoor drones and small mobile robots
  • Human motion datasets for retargeting and imitation learning
  • Teleoperation and human pose tracking for robot control
  • Biomechanics and gait analysis for exoskeletons and prosthetics

In practice

A lab flying small quadrotors indoors mounts a ring of infrared cameras around a netted flight area and calibrates them with a wand. Four retroreflective markers in an asymmetric pattern are fixed to each drone, and each pattern is registered as a rigid body. The motion capture software streams poses over the network, and the team aligns the rigid-body frame with the drone's body frame and measures the end-to-end latency before feeding the poses to the flight controller. When evaluating a visual-inertial odometry system, they record the same flights and align the estimated and motion capture trajectories in time and space before computing error.

Common misconceptions

  • Motion capture data is perfect ground truth — it has calibration error, occlusion gaps, marker swaps and latency, and these need checking before it is used as a reference
  • Markers directly give the pose of the robot or bone — they give the pose of the marker pattern, and the offset to the body frame or underlying bone must be calibrated
  • Optical and markerless motion capture are the same thing — marker-based systems track physical markers with dedicated cameras, while markerless systems estimate pose from ordinary images and are usually less accurate

Who should know this

  • State estimation and SLAM researchers who need ground truth
  • Aerial robotics engineers flying in instrumented labs
  • Researchers collecting human motion for teleoperation or imitation learning
  • Biomechanics and wearable robotics engineers

Related technologies

  • Retroreflective Markers
  • Infrared Camera Arrays
  • Multi-View Triangulation
  • Markerless Pose Estimation
  • Inertial Motion Capture

Related terms

  • Extrinsic Calibration
  • Camera Intrinsics
  • Motion Retargeting
  • IMU
  • Teleoperation

Related projects

  • AprilTag: The standard visual fiducial marker system for robot localization and pose estimation
  • PyPose: PyTorch library for differentiable robotics with deep learning and Lie group optimization
  • Intrinsic Core: Intrinsic's open industrial robot stack: real-time control, planning and perception
  • MINT: Turns ordinary egocentric video into world-space hand and camera motion
  • Hand Tracking Streamer: Meta Quest app that streams live hand tracking data for robot teleoperation
  • Eidon Tracker POV: 1,274 hours of egocentric household video with seven-point arm IMU tracking

Videos

  • The insane software behind optical motion capture.: Made By Dennis

This website requires JavaScript to function properly. Please enable JavaScript to explore our collection of open source robotics projects.