Open Source Robotics

A curated collection of high-quality open source robotics projects, tools, and software to propel the robotics community forward.

RoboCap-10k

10,000 hours of first-person paid work with metric 3D head and hand pose

License

CC BY 4.0

Data: CC BY 4.0, per the licence section of the dataset card, its Hugging Face licence tag and the technical report; the card says it may be used for any purpose, including commercial use, with credit by citing the paper. Access: gated; Grounded SI approves each request, and the form asks you to agree not to try to re-identify anyone in the dataset. People in the video: the card says workers signed informed consent forms and faces are blurred, but CC BY 4.0 does not license privacy or publicity rights, and the report leaves compliance with local labour and privacy law to the partner in each country. Related sets: BitRobot's RoboCap-LiDAR-100 is CC BY-NC 4.0, non-commercial only, while RoboCap-IKEA-500 and RoboCap-Origami-500 are CC BY 4.0. Card image: a frame from Grounded SI's own preview of a dataset episode; credit Grounded Superintelligence. Checked 2026-11-12 against the dataset card, the technical report and the BitRobot dataset cards.

Overview

RoboCap-10k is an egocentric dataset from Grounded Superintelligence: 10,000 curated hours of people doing their paid jobs, filmed from a camera hat and labelled with metric 3D head and hand motion. It holds 1,363,915 episodes across 21 industry verticals, 365 site types and 619 tasks, recorded in 11 countries, from bread production and garment stitching to engine repair, circuit-board soldering and rebar tying. Every hour was recorded on the same device, the RoboCap, a 250 g hat with six synchronised global-shutter cameras and two IMUs, and processed by the same 3D pipeline. Each frame carries a gravity-aligned 6-DoF head pose and MANO hand parameters with 21 keypoints per hand in the world frame, and each episode has a short task caption and timed sub-task captions. The data comes in the LeRobot format and as Rerun recordings, with the raw camera streams, under CC BY 4.0. It was published on Hugging Face on 7 November 2026, and access is granted on request.

RoboCap-10k is a first-person dataset of real occupational work for training robot policies and world models, released by Grounded Superintelligence on Hugging Face under CC BY 4.0. Workers wore a camera hat during their normal paid shifts of 6 to 12 hours, and nothing was scripted or staged for the camera. The curated release holds 10,000 hours in 1,363,915 episodes of 10 to 600 seconds, with a median of 17.8 seconds. It covers 21 industry verticals, 365 site types and 619 tasks, recorded in 11 countries: India, Nigeria, Colombia, the United States, the United Kingdom, Mexico, Chile, Malaysia, the Philippines, Indonesia and Spain. Tasks are rated by difficulty, with 170 easy, 277 medium and 172 hard, the hard ones being skilled work such as soldering components onto circuit boards, tying rebar cages or industrial stitching.

All of it was recorded on one device model, the RoboCap: a 250 g hat with six global-shutter cameras at 1080p and 30 Hz, fired by one hardware trigger, and two IMUs at 200 Hz. Two fisheye stereo pairs keep the hands in a metric stereo view across the reach, and two wide-angle side cameras catch reaches that leave it. Grounded SI recalibrates the cameras of each recording session from the footage itself and reports that every released device stays below 1 px of average epipolar error. Head trajectories come from multi-camera visual-inertial SLAM with offline bundle adjustment, and hands come from a multi-view tracker that fits one MANO hand model across all cameras. Against motion capture, Grounded SI reports a median head trajectory error of 3.0 mm and a hand error of 4.5 mm PA-MPJPE; these are its own measurements, described in the technical report.

Language comes at two levels: one sparse caption for each interval, and consecutive dense sub-task captions inside it. The dense captions are generated by Gemini 3.8 Flash from the two front cameras, and people reviewed the captions for 1,000 hours, 10% of the corpus, against the video.

Every episode comes in two forms. The LeRobot format is for training, with one H.265 MP4 per camera and Parquet tables of head pose, hand pose and IMU readings indexed by frame. A Rerun recording shows the six streams, the head trajectory and the hand meshes in one 3D world frame. Each episode also carries its calibration and a JSON file with taxonomy labels, captions and quality statistics, and the raw streams are released so the labels can be checked and regenerated. The video totals about 108 TB, held in a Hugging Face storage bucket that LeRobot can stream from. Access needs a request on the dataset page, which Grounded SI says it reviews within 2 to 3 business days. Without an account, the dataset preview site lets anyone browse 1,794 episodes with captions, country and work zone, and play them in a Rerun viewer.

On privacy, the card says every worker signed an informed consent form, every partner company signed a data agreement, and wearers are paid on top of their wages. No audio was recorded, faces are blurred permanently by Grounded SI's own detector, and the metadata names only the country and site type. The announcement credits BitRobot as co-author, and BitRobot has published smaller RoboCap recordings of its own, such as RoboCap-IKEA-500 and RoboCap-Origami-500.

Why it matters

Robots learn hand skills from demonstrations, and teleoperation gives few of them at high cost. Video of people working is plentiful, but a robot imitates poses, not pixels: footage only becomes training data once the head and hands are located in metres in every frame. RoboCap-10k supplies that for 10,000 hours of real work from one calibrated device, with the raw streams kept so the labels can be redone, under a licence that allows commercial use. It is the largest egocentric dataset in this catalogue, next to Eidon Tracker POV, which records household chores, and HiFi-UMI-2K, which records robot-free bimanual work on a head-and-hands rig; RoboCap-10k records skilled trades. It is material for pre-training VLA and world models, for imitation learning and for motion retargeting from human hands to robot hands, and it arrives in the LeRobot format, with Rerun recordings for inspection. Tools such as MINT estimate hand motion from ordinary video; this release ships it already measured from calibrated stereo.

Problem it solves

Egocentric video is the most scalable source of manipulation data, but most large corpora have no metric 3D labels, mix many different devices, or were recorded in labs with a few participants doing staged tasks. RoboCap-10k gives 10,000 hours of unscripted occupational work from one calibrated device model, with metric head pose, 3D hand pose and two levels of captions on every episode, in a format robot-learning tools already read and under a licence that allows commercial use.

Key takeaways

  • 10,000 curated hours in 1,363,915 episodes of real paid work, across 21 industry verticals, 365 site types and 619 tasks in 11 countries
  • All recorded on one device model, the RoboCap hat: six synchronised global-shutter cameras and two IMUs
  • Gravity-aligned metric 6-DoF head pose and MANO hand pose with 21 keypoints per hand on every frame
  • Sparse task captions and timed dense sub-task captions, with 10% of hours reviewed by people
  • LeRobot format for training and Rerun recordings for inspection, with raw streams and per-session calibration
  • CC BY 4.0, open for commercial use; access is granted on request, and the video totals about 108 TB

Who it is for

  • Researchers pre-training VLA, world and robot foundation models on human demonstration data
  • Teams retargeting human hand motion to robot hands for dexterous manipulation
  • Computer-vision researchers working on egocentric video, SLAM, hand pose and action segmentation
  • LeRobot users who want large-scale human data in a format their tools already read

Limitations

  • Human demonstrations, not robot actions: training a robot on them needs retargeting, and human arms differ from robot arms
  • All 3D labels are model outputs, not ground truth, and hand errors concentrate where fingers are hidden during tight grasps
  • The accuracy figures are Grounded SI's own, measured on its own benchmarks and published in its technical report
  • Dense captions are generated by Gemini 3.8 Flash; people reviewed only 10% of the hours
  • Collection is concentrated in a few countries, India accounts for most verticals, and household and one-handed work are underrepresented
  • No touch or force sensing: contact has to be inferred from vision alone
  • Gated: access needs an approved request, and the full video is about 108 TB, so most users will stream or download a part
  • Faces are blurred by a detector with 77.3% recall on single frames, helped by tracking across frames; the card does not claim every face is caught
  • CC BY 4.0 requires attribution and does not license the privacy or publicity rights of the workers and bystanders in the video

This website requires JavaScript to function properly. Please enable JavaScript to explore our collection of open source robotics projects.