Robo-DM: Data Management For Large Robot Datasets

Authors: Kaiyuan Chen, Letian Fu, David Huang, Yanxiang Zhang, Lawrence Yunliang Chen, Huang Huang, Kush Hari, Ashwin Balakrishna, Ted Xiao, Pannag R Sanketi, John Kubiatowicz, Ken Goldberg · ICRA 2025

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

Robo-DM stores each robot episode's video, language and action streams in one time-stamped EBML file, cutting size up to 70x versus RLDS and loading faster than LeRobot.

Why it matters

Dataset size sets both the host's download bill and how long a training job waits for data, so smaller files that load quickly lower the cost of sharing and using large robot datasets. In the Octo and ICRT experiments, lossy video compression at these ratios changed validation loss by 2.6% and did not stop a policy from completing all 15 physical pick-and-place trials.

Problem

Large teleoperated robot datasets such as Open X-Embodiment are costly to share and slow to load: storing 8.9 TB on Google Cloud costs 172 USD per month, but each full download costs the host between 172 and 1,540 USD. Existing formats either store images as matrices (RLDS, HDF5), which is lossless but not space efficient, or scatter streams across several files with framework-specific alignment assumptions (LeRobot), which complicates loading and slows decoding.

Proposed solution

Robo-DM extends the Extensible Binary Meta Language (EBML), the container behind MKV video, to store all vision, language and action streams of an episode in one self-contained file, each packet tagged with a timestamp relative to the episode start. Data is recorded raw and transcoded after collection with lossy (for example AV1) or lossless (FFV1) codecs; at training time a load balancer chooses between decoding video and reading memory-mapped caches of decoded frames. It works with ROS 2 (rosbag, rviz, Foxglove), TensorFlow and PyTorch, and exports to HDF5 and RLDS.

Key insights

Most of the size reduction comes from video compression of the camera streams, as the authors state.
Keeping every stream in one time-stamped container removes the need for heuristics to align streams recorded at different rates.
Deferring compression until after collection keeps transcoding from interfering with data collection.
Small episodes gain less: RT-1 averages 1.51 MB per trajectory in RLDS, and Robo-DM's seek and decode metadata limits the reduction to 4.39x.

Main results

Lossy Robo-DM against the original RLDS files (Table I): Bridge 387.49 GB to 5.31 GB (73x), Cable Routing 4.67 GB to 0.26 GB (18x), Door Opening 7.12 GB to 0.10 GB (71x), AutoLab UR5 76.39 GB to 3.26 GB (23x). The text gives these as 18x, 73x, 23x and 73x; the abstract states up to 70x lossy and 3.5x lossless.
Loading throughput on three OXE datasets (Fig. 4): faster than LeRobot by 33x, 20x and 5x, with the lossless version close to the lossy one; the abstract states up to 50x faster than LeRobot when decoding sequentially. HDF5 was faster than Robo-DM because its data is uncompressed.
Octo-small fine-tuning on RT-1 (111.06 GB to 36.50 GB, 4.39x): 0.02 s data loading and 0.10 s training per iteration at batch size 64, the same as the TensorFlow loader; validation MSE 1.86 with lossless data and 1.91 with lossy data, a 2.6% increase.
The introduction reports that for Octo fine-tuning Robo-DM loads 3.0x faster at small batch size and adds no slowdown at large batch size.
ICRT on a Franka arm, trained on 335 demonstrations stored as 77 MB lossy (75.3x from 5.8 GB of gzip HDF5) or 1.7 GB lossless (3.41x): 15 of 15 pick-and-place trials succeeded with the lossy-trained model.

Limitations

Using RAM as a decoding cache raises memory use and can lower throughput when per-episode data is large; on Bridge, throughput fell as batch size grew, and sampling a single frame per episode may be slower (stated by the authors).
Decoding used PyAV on the CPU without GPU acceleration; the authors name Decord and NVIDIA NVDEC as faster options for future work.
Small episodes compress less because of per-file seek and decode metadata (4.39x on RT-1).
The physical check is one pick-and-place task with 15 trials of a model trained on lossy data, with no matching model trained on uncompressed data.
LeRobot's conversion drops the depth stream and some action streams, so there is no LeRobot size figure for AutoLab UR5.

Field impact

Won the IEEE ICRA 2025 Best Paper Award on Robot Learning; 5 citations. The authors release Robo-DM as open source, with code and evaluation scripts at github.com/BerkeleyAutomation/fog_x.

Abstract

Recent results suggest that very large datasets of teleoperated robot demonstrations can be used to train transformer-based models that have the potential to generalize to new scenes, robots, and tasks. However, curating, distributing, and loading large datasets of robot trajectories, which typically consist of video, textual, and numerical modalities - including streams from multiple cameras - remains challenging. We propose Robo-DM, an efficient open-source cloud-based data management toolkit for collecting, sharing, and learning with robot data. With Robo-DM, robot datasets are stored in a self-contained format with Extensible Binary Meta Language (EBML). Robo-DM can significantly reduce the size of robot trajectory data, transfer costs, and data load time during training. Compared to the RLDS format used in OXE datasets, Robo-DM’s compression saves space by up to 70x (lossy) and 3.5x (lossless). Robo-DM also accelerates data retrieval by load-balancing video decoding with memory-mapped decoding caches. Compared to LeRobot, a framework that also uses lossy video compression, Robo-DM is up to 50x faster when decoding sequentially. We physically evaluate a model trained by Robo-DM with lossy compression, a pick-and-place task, and In-Context Robot Transformer. Robo-DM uses 75x compression of the original dataset and does not suffer reduction in downstream task accuracy. Code and evaluation scripts can be found on website https://github.com/BerkeleyAutomation/fog_x.

Open Source Robotics