Open Source Robotics

A curated collection of high-quality open source robotics projects, tools, and software to propel the robotics community forward.

HiFi-UMI-2K

2,000 hours of robot-free bimanual manipulation data at 3 mm accuracy

License

CC-BY-4.0

Overview

HiFi-UMI-2K is a 2,000-hour open dataset of robot-free bimanual manipulation demonstrations, captured with a portable head-and-hands rig rather than a teleoperated robot. It is a curated subset of a source corpus exceeding 20,000 hours and 4.32 million episodes across more than 480 scenes. Every episode carries six synchronised camera views — stereo head plus two non-parallel fisheye views per hand covering roughly 200 degrees — with calibrated bimanual end-effector trajectories, gripper states, language annotations and quality-control metadata. Released by Simple AI under CC-BY-4.0 in a LeRobot v3-style Parquet and MP4 export.

The usual trade in manipulation data is accuracy against scale: teleoperate a real robot and the trajectories are perfect but each hour costs a robot, a rig and an operator; use a handheld UMI gripper and it scales, but pose drifts, inter-gripper geometry is reconstructed after the fact and sensors are aligned in software. HiFi-UMI attacks the second half of that trade. Head-mounted offline stereo-inertial SLAM with fiducial-marker localisation gives roughly 3 mm local end-effector accuracy, both hands are localised in the same head-camera frame so inter-hand relative pose is measured rather than reconstructed, and a shared GPIO trigger holds cross-sensor offsets below 40 microseconds. Every trajectory is reconstructed and replay-validated automatically, each stage passing at about 98%.

Why it matters

This is the dataset behind the claim that robot-free capture can carry post-training on its own, not just pre-training — policies trained only on HiFi-UMI demonstrations deploy directly on a real bimanual robot. For anyone without a teleoperation rig or the target arm, that changes what is buildable. At 121,000+ downloads it is already one of the more heavily used open manipulation corpora, and the LeRobot v3 export means it drops into existing pipelines without conversion.

Problem it solves

Open manipulation data forces a choice between fidelity and scale. Robot-free handheld capture scales but has been trusted only for pre-training, because pose drift, reconstructed inter-gripper geometry and software sensor alignment leave it short of deployment-grade.

Key takeaways

  • 2,000 hours released, curated from a source corpus of over 20,000 hours and 4.32M+ episodes across 480+ scenes
  • Six synchronised views per episode: stereo head cameras plus two non-parallel fisheye cameras per hand, ~200 degrees coverage
  • Roughly 3 mm local end-effector accuracy with no external tracking infrastructure
  • Cross-sensor time offsets under 40 microseconds via a shared GPIO hardware trigger; fewer than 2 dropped frames per hour
  • Trajectory reconstruction and whole-body-control replay validation each succeed at about 98%
  • CC-BY-4.0, distributed as LeRobot v3-style Parquet tables plus MP4 video

Who it is for

  • VLA and world-action-model pre-training teams
  • Bimanual manipulation researchers
  • Imitation-learning practitioners without a teleoperation rig
  • Benchmark and data-quality researchers

Limitations

  • The released 2,000 hours are a curated subset; the full 20,000+ hour corpus and the 4.32M episodes stay in-house
  • The capture device itself is not open hardware, so the collection method cannot be reproduced directly
  • Demonstrations are human hands in a glove-style gripper, not the target robot, so an embodiment gap remains at the arm
  • Statistics differ between the released subset and the source corpus, and the card labels which is which — read carefully before quoting figures
  • Six video streams per episode at scale means substantial storage and bandwidth to work with the full release

This website requires JavaScript to function properly. Please enable JavaScript to explore our collection of open source robotics projects.