MimicPlay: Long-Horizon Imitation Learning by Watching Human Play

Authors: Chen Wang, Linxi Fan, Jiankai Sun, Ruohan Zhang, Li Fei-Fei, Danfei Xu, Yuke Zhu, Anima Anandkumar · CoRL 2023

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

MimicPlay learns a latent planner from 10 minutes of human hand play video and a plan-guided transformer controller from a few teleoperated demos, raising long-horizon success on a Franka arm.

Why it matters

Human play data needs no robot, task labels or environment resets: a human finishes in about 5 seconds what takes 90 seconds of robot teleoperation, and 10 minutes of play is roughly equivalent to a 3-hour teleoperation dataset. Because human motion and robot skills share one latent plan space, a human video can also be used directly as the goal prompt for the robot.

Problem

Imitation learning has mostly been limited to short-horizon primitives because long-horizon demonstrations are slow and labour-intensive to collect. Learning from robot play data helps but still needs hours of teleoperation, for example 4.5 hours for C-BeT and 6 hours for TACO-RL.

Proposed solution

MimicPlay splits learning by data source. A goal-conditioned latent planner is trained on 10 minutes of human play video per scene to predict future 3D hand trajectories, tracked with two calibrated cameras, through a Gaussian Mixture Model head, while a KL loss aligns human and robot visual features. A GPT-style transformer policy, trained on 20–40 teleoperated demonstrations per task, then maps the frozen planner's latent plans plus wrist-camera and proprioceptive inputs to end-effector and gripper commands.

Key insights

High-level plans can be learned from cheap human play data, while low-level control comes from teleoperated demonstrations that have no embodiment gap.
A GMM head is needed to capture the multimodal human hand trajectories; without it, the planner generalized worse than a variant trained with no human data.
A KL loss on encoder feature distributions narrows the human-robot visual gap without paired data, giving a 23% larger overlap between human and robot feature distributions in t-SNE.
Using all 10 minutes of human play rather than half improves generalization to unseen subgoal compositions.
Pretrained latent plans reduce the loss from single-task to multi-task training compared with end-to-end methods such as LMP and R3M-BC.

Main results

Kitchen, long-horizon, all three tasks (Table 1): 0.47 success with 20 demos and 0.70 with 40 demos, against at most 0.07 and 0.17 for the baselines and 0.23 / 0.40 for the variant without human data.
Study Desk (Table 2, 20 demos): 0.55 on trained tasks and 0.47 on unseen task compositions, against 0.00 for GC-BC (BC-trans) and LMP and 0.20 / 0.10 for the variant without human data.
Multi-task across Flower, Whiteboard, Sandwich and Cloth (Table 3): 0.55 with one shared model against 0.58 with one model per task; LMP drops from 0.13 to 0.05 and R3M from 0.25 to 0.15.
Without the KL loss, trained-task success falls to 0.38 (17% lower than 0.55); without the GMM, it is 0.28 on trained and 0.07 on unseen tasks.
In LIBERO simulation, where only robot play data is available, the hierarchical variant without human data reaches 0.29–0.67 on long-horizon tasks while the GC-BC baselines score 0.00 (5 seeds, 100 trials).
The whole system runs at 17 Hz, fast enough to re-plan after human disturbances such as unfolding a folded towel; Section 4 states a 20 Hz control frequency and the appendix gives 17–20 Hz.

Limitations

Latent plans are learned from scene-specific human play data; training on Internet-scale data could improve scalability (stated by the authors).
Tasks are limited to tabletop settings; mobile manipulation is not covered (stated).
Cross-embodiment representation learning has room to improve, for example with temporal contrastive or cycle-consistency learning (stated).
In multi-task training, whiteboard success dropped from 0.5 to 0.2, which the authors attribute to shorter whiteboard demonstrations unbalancing the dataset.
3D hand tracking needs a calibrated two-camera setup, and the human-play component could not be tested in simulation.

Field impact

Finalist for the CoRL 2023 Best Systems Paper Award and on the CoRL 2023 Best Paper / Best Student Paper combined finalist list, with 415 citations. The implementation code is released at github.com/j96w/MimicPlay, with videos on the project page.

Abstract

Imitation learning from human demonstrations is a promising paradigm for teaching robots manipulation skills in the real world. However, learning complex long-horizon tasks often requires an unattainable amount of demonstrations. To reduce the high data requirement, we resort to human play data—video sequences of people freely interacting with the environment using their hands. Even with different morphologies, we hypothesize that human play data contain rich and salient information about physical interactions that can readily facilitate robot policy learning. Motivated by this, we introduce a hierarchical learning framework named MimicPlay that learns latent plans from human play data to guide low-level visuomotor control trained on a small number of teleoperated demonstrations. With systematic evaluations of 14 long-horizon manipulation tasks in the real world, we show that MimicPlay outperforms state-of-the-art imitation learning methods in task success rate, generalization ability, and robustness to disturbances. Code and videos are available at mimic-play.github.io.

Open Source Robotics