Legged Locomotion in Challenging Terrains using Egocentric Vision

Authors: Ananye Agarwal, Ashish Kumar, Jitendra Malik, Deepak Pathak · CoRL 2022

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

RL on cheap terrain samples, then distillation to depth, gives an A1 quadruped a depth-to-joint-angle policy that climbs stairs and curbs and crosses gaps and stepping stones.

Why it matters

The deployed policy uses one depth camera and needs no elevation map, localization or MPC, so it runs on the robot's onboard UP board and Jetson NX. Gaits such as large hip abduction on stairs emerge from training, which lets a robot with a 28 cm hip height climb steps close to its own height.

Problem

Vision-based legged locomotion is usually split into elevation mapping and foothold planning, and elevation maps need several depth images fused with pose estimates from odometry. Pose drift and sensor noise make these maps unreliable on gaps and stepping stones, and the extra cameras, lidars and compute are not available on a small, low-cost robot. A small robot also cannot rely on predefined gait priors to clear obstacles close to its own height.

Proposed solution

Phase 1 trains a policy in simulation with PPO using scandots (terrain heights sampled at fixed points under the robot) and privileged information, with energy-based rewards and no gait priors. Phase 2 distils it with DAgger into a recurrent policy, either a monolithic GRU or RMA-style estimators feeding the frozen base policy, that maps egocentric depth and proprioception directly to target joint angles at 50 Hz and uses memory to infer the terrain under the hind feet.

Key insights

Rendering depth slows simulation by an order of magnitude; running RL on cheap scandots and then distilling to depth with supervised learning lets the whole system train on a single GPU in a few days.
A recurrent policy can remember terrain seen earlier by the front camera to place the hind feet, replacing an explicit elevation map.
With no gait priors, behaviours suited to a small robot, such as large hip abduction to clear stair edges, emerge from energy-minimising rewards.
If scandots and camera field of view are chosen so the phase 2 imitation loss stays low, the phase 2 policy's return is provably close to optimal (Theorem 2.1).
Elevation-map pipelines need large training noise to cope with pose drift; in simulation this 'noisy' baseline barely moves on stepping stones.

Main results

Real world: 100% success going up and down stairs (17 cm high, 30 cm deep, 13 stairs), 100% on 26 cm gaps, and 94% on stepping stones (30 cm wide, 15 cm apart); the blind baseline scores 0% upstairs, on gaps and on stepping stones.
Simulation totals: average forward displacement 134.18 m (RMA) and 135.85 m (monolithic) versus 73.05 m (noisy) and 84.79 m (blind); mean time to fall 278.45 s and 275.01 s versus 147.83 s and 175.06 s.
Simulated stepping stones: the RMA and monolithic policies travel 18.83 m and 20.72 m, while the noisy and blind baselines travel 1.09 m and 1.02 m.
On simulated slopes the vision policies travel farther than the baselines: up to 27% according to the Results text, up to 25% according to the Table 1 caption.
Stair height in urban tests: up to 24 cm high (28 cm lowest width) according to the Urban Environments section, and up to 25 cm, 89% of the hip height, according to the Introduction; curbs up to 26 cm.
The blind baseline completes the downstairs task every time, but with a high-impact falling gait that detached the robot's rear right hip during the trials.

Limitations

Failures still happen when vision or terrain differ between simulation and the real world; the only fix in this approach is to recreate the case in simulation and retrain, which the authors call a fundamental limitation.
The robot only moves through the environment and does not interact with it; adding an arm is future work.
Because terrain under the body comes from visual memory, the robot sometimes misses a step on stairs (it recovers), and outdoors it sometimes gets stuck in crevices.
The real-world comparisons used separate phase 2 policies: the built-in front camera for stairs and an externally mounted top camera for gaps and stepping stones.
On hardware the method is compared only with the blind baseline; the elevation-map (noisy) baseline is tested in simulation only.

Field impact

Winner of the CoRL 2022 Best Systems Paper Award, with 373 citations. The paper publishes videos on its project page; it does not state a code release.

Abstract

Animals are capable of precise and agile locomotion using vision. Replicating this ability has been a long-standing goal in robotics. The traditional approach has been to decompose this problem into elevation mapping and foothold planning phases. The elevation mapping, however, is susceptible to failure and large noise artifacts, requires specialized hardware, and is biologically implausible. In this paper, we present the first end-to-end locomotion system capable of traversing stairs, curbs, stepping stones, and gaps. We show this result on a medium-sized quadruped robot using a single front-facing depth camera. The small size of the robot necessitates discovering specialized gait patterns not seen elsewhere. The egocentric camera requires the policy to remember past information to estimate the terrain under its hind feet. We train our policy in simulation. Training has two phases - first, we train a policy using reinforcement learning with a cheap-to-compute variant of depth image and then in phase 2 distill it into the final policy that uses depth using supervised learning. The resulting policy transfers to the real world and is able to run in real-time on the limited compute of the robot. It can traverse a large variety of terrain while being robust to perturbations like pushes, slippery surfaces, and rocky terrain. Videos are at https://vision-locomotion.github.io

Open Source Robotics