Open Source Robotics

A curated collection of high-quality open source robotics projects, tools, and software to propel the robotics community forward.

FiftyOne now reads LeRobot datasets directly, with search across episodes

  • Source: Voxel51 and Hugging Face
  • Author: Open Source Robotics
  • Published: 2026-10-05
  • Category: research

FiftyOne, Voxel51's open-source dataset curation tool, now opens LeRobot v3 datasets without converting them or copying video. Each episode can be played back with its cameras, states and actions in step, then filtered, searched by text or similarity, and exported as a new LeRobot dataset. It reads v3 only, and the loader shipped in FiftyOne 1.22.0.

FiftyOne, the open-source dataset curation tool from Voxel51, now opens LeRobot v3 datasets as they are. Point it at a dataset folder and every episode becomes one sample that can be played back, filtered and searched, with no format conversion and no video re-encoded or copied. Engineers from Voxel51 and Hugging Face described the support on 5 October; the loader itself shipped in FiftyOne 1.22.0 on 12 September, listed in the release notes as the new LeRobotDataset type with an episode viewer.

What an episode looks like

Importing reads only the dataset's metadata files. Each sample is a pointer: which rows of the Parquet files hold the episode's states and actions, and which time window of each camera's MP4 holds its video. In the app every camera gets its own tile, and a State & Action tile shows each joint's value frame by frame, in step with the video. The robot type, task, duration and frame rate of each episode become ordinary fields, so "all SO-101 episodes longer than ten seconds" is a filter rather than a script. The FiftyOne documentation covers the import and the fields.

The worked example

The announcement comes with a 497-episode example dataset, cut from the LeRobot community dataset v3: about ten episodes from each of 50 robot types. Every episode carries a video embedding from Qwen3-VL-Embedding-2B, computed from up to 32 evenly spaced frames. That is what makes the curation side work: searching episodes by a text description, finding the episodes most similar to a chosen one, and scoring each episode for how unusual or how typical it is, so near-duplicates and outliers can be found before training.

Getting the data back out

A filtered selection reaches training in three ways: read the rows in place through the pointer, pass the chosen episode numbers to LeRobot's own LeRobotDataset loader, or export the selection as a new self-contained LeRobot v3 dataset. Only the export copies data.

What to check first

- Only LeRobot v3 is read; older datasets need converting with LeRobot's own tools first. - Samples point at the source files, so a dataset that is moved or changed has to be imported again. - Episode numbers restart at zero in every source dataset. When several datasets are loaded together, filter by dataset name before exporting or training, and export one source dataset at a time, because their schemas differ. - Loading a LeRobot dataset straight from the Hugging Face Hub needed a fix that merged on 5 October, after the latest release (1.22.1), so a local folder is the safe route until the next version.

Why it matters

Most robot-learning datasets are judged by size, because looking inside them is slow: thousands of episodes, several cameras each, and no quick way to see which are near-copies, which are broken and which show something rare. A tool that reads the LeRobot format in place, plays any episode with its joint values alongside, and ranks episodes by how typical or unusual they are makes inspection cheap enough to do before every training run. It also gives the community datasets recorded on low-cost arms the same curation tools that image datasets have had for years.

Key takeaways

  • FiftyOne 1.22.0 and later load LeRobot v3 datasets with one call; nothing is converted, re-encoded or copied on import
  • Each episode is one sample: every camera plays in its own tile, with joint states and actions shown frame by frame in step with the video
  • Robot type, task, duration and frame rate are queryable fields, so selecting episodes is a filter rather than a script
  • With video embeddings added, episodes can be searched by text, matched to similar episodes, and scored as unusual or typical
  • A selection goes back to training in place, through LeRobot's own loader, or as an exported LeRobot v3 dataset
  • Only v3 is supported, and loading from the Hugging Face Hub needs a fix merged after release 1.22.1

Open-source angle

FiftyOne is open source under Apache-2.0 and the LeRobot support is in that open-source package, not behind Voxel51's commercial FiftyOne Enterprise product. The example dataset is published under Apache-2.0 on the Hugging Face Hub and draws on recordings the LeRobot community contributed. The release notes do list loading LeRobot datasets from cloud storage separately, which is the part to check against the Enterprise feature list before planning around it.

Related technologies

  • FiftyOne
  • LeRobot
  • LeRobot dataset v3
  • Qwen3-VL-Embedding
  • Parquet
  • Hugging Face Hub

Related projects

  • LeRobot

Links

  • Source: https://huggingface.co/blog/harpreetsahota/fiftyone-now-reads-lerobot-50-embodiments-497-epis

This website requires JavaScript to function properly. Please enable JavaScript to explore our collection of open source robotics projects.