HumanPlus: Humanoid Shadowing and Imitation from Humans

Authors: Zipeng Fu, Qingqing Zhao, Qi Wu, Gordon Wetzstein, Chelsea Finn · CoRL 2024

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

An RL policy trained on 40 h of human motion lets a 33-DoF humanoid copy an operator seen by one RGB camera; that teleop data then trains vision skill policies with 60–100% success.

Why it matters

The teleoperation setup needs one operator and one RGB camera (listed at $50 in the paper's comparison) and controls body and hands together, including squatting and walking. Teams working with humanoids get a path from low-cost whole-body demonstrations to autonomous skills without building a task-specific simulation for each skill.

Problem

Humanoids could in principle learn from large amounts of human motion data, but differences in morphology and actuation, hard whole-body perception and control, and the lack of an accessible whole-body data pipeline have blocked this. Prior humanoid teleoperation relied on motion-capture suits, exoskeletons or VR headsets, which the authors describe as expensive and restricted in where they can be used.

Proposed solution

A decoder-only transformer low-level policy (Humanoid Shadowing Transformer) is trained with PPO in simulation on AMASS human poses retargeted to the robot, then deployed zero-shot at 50 Hz with a 1000 Hz PD controller. Body pose (WHAM) and hand pose (HaMeR) estimated from a single RGB camera are retargeted in real time, so one operator drives the whole robot while two head cameras record egocentric data. A decoder-only Humanoid Imitation Transformer, adapted from ACT, is trained on these demonstrations and adds an L2 loss on predicted future image features to keep the policy from ignoring vision.

Key insights

One task-agnostic low-level policy conditioned on target poses, trained on retargeted human motion, replaces per-skill reward engineering and transfers zero-shot to hardware.
Shadowing from one RGB camera lets a single operator control body and hands at once; the kinesthetic, ALOHA-style and Meta Quest baselines needed two or more operators and did not support whole-body control.
Predicting future image-feature tokens alongside actions stops a vision policy from overfitting to proprioception; ACT without it got stuck repeating sub-tasks it had already completed.
Binocular input matters when depth matters: the monocular variant failed Wear a Shoe and Walk completely and handled the table roughly in Fold Clothes.

Main results

Whole-task success of the imitation policy: Fold Clothes 100%, Rearrange Objects 90%, Type "AI" 80%, Two-Robot Greeting 90%, Warehouse 90%, Wear a Shoe and Walk 60%, using 25–40 demonstrations per task.
Wear a Shoe and Walk (40 demos): 60% for HIT versus 0% for the monocular variant, ACT and open-loop replay; ACT also scored 0% on Type "AI" but matched HIT on Fold Clothes, Warehouse and Two-Robot Greeting.
Six-participant user study: whole-task time on Rearrange Objects was 5.20 s with shadowing versus 6.60 s kinesthetic, 7.15 s ALOHA-style and 8.87 s Meta Quest; only shadowing could do the squatting Rearrange Lower Objects task (15.34 s).
Push tests at the pelvis: the learned policy withstood 32/44/70/100 N (forward/backward/leftward/rightward) versus 24/36/40/40 N for the H1 default controller, and recovered in 1.2 s versus 15 s.
The learned policy squats to 0.44 m (default controller 0.85 m), jumps 0.35 m (default 0 m) and stands up from a chair, which the default controller cannot.
Body pose estimation and retargeting runs at 25 fps and hand pose at 10 fps on an RTX 4090; the imitation policy runs onboard at 25 Hz, feeding the 50 Hz low-level policy.

Limitations

The hardware has fewer DoF than a human: 1-DoF ankles prevent movements such as lifting and shaking one leg, and 5-DoF arms (including the wrist) limit 6-DoF operational-space control and leave unreachable regions during shadowing.
The egocentric cameras are fixed on the head and not active, so hands and interactions can fall out of view.
A fixed retargeting map drops human joints the robot does not have, which may restrict learning to a subset of human motions.
Pose estimation fails under large occlusions, limiting where the operator can stand; the operator also relies on line of sight to the robot.
Long-horizon navigation is not addressed; the authors state it would need many more demonstrations and accurate real-world velocity tracking.

Field impact

Finalist for the CoRL 2024 Outstanding Paper Award, with 442 citations. The official project page links the code at github.com/MarkFzp/humanplus.

Abstract

One of the key arguments for building robots that have similar form factors to human beings is that we can leverage the massive human data for training. Yet, doing so has remained challenging in practice due to the complexities in humanoid perception and control, lingering physical gaps between humanoids and humans in morphologies and actuation, and lack of a data pipeline for humanoids to learn autonomous skills from egocentric vision. In this paper, we introduce a full-stack system for humanoids to learn motion and autonomous skills from human data. We first train a low-level policy in simulation via reinforcement learning using existing 40-hour human motion datasets. This policy transfers to the real world and allows humanoid robots to follow human body and hand motion in real time using only a RGB camera, i.e. shadowing. Through shadowing, human operators can teleoperate humanoids to collect whole-body data for learning different tasks in the real world. Using the data collected, we then perform supervised behavior cloning to train skill policies using egocentric vision, allowing humanoids to complete different tasks autonomously by imitating human skills. We demonstrate the system on our customized 33-DoF 180cm humanoid, autonomously completing tasks such as wearing a shoe to stand up and walk, unloading objects from warehouse racks, folding a sweatshirt, rearranging objects, typing, and greeting another robot with 60-100% success rates using up to 40 demonstrations. Project website: https://humanoid-ai.github.io/

Open Source Robotics