Perceptive pedipulation with local obstacle avoidance

Authors: Jonas Stolle, Philip Arm, Mayank Mittal, Marco Hutter · Humanoids 2024

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

End-to-end RL policy for ANYmal that tracks a foot-position command while avoiding obstacles with its whole body, using a 2.5D height scan and a switch that allows deliberate foot contact.

Why it matters

The control interface stays a single foot-position command, so a human operator or high-level planner can use it near obstacles without planning collision-free motion for the whole body. It gives a quadruped manipulation skills without a dedicated arm, which the authors note adds mass and complexity and reduces payload capacity.

Problem

Earlier pedipulation policies track a foot-position command using only proprioception, so they cannot avoid obstacles and collide with anything in the way. Putting an obstacle-aware planner above a blind policy still leaves the rest of the body free to collide, and the large safety margins needed to prevent that rule out work in confined spaces.

Proposed solution

A single RL policy, trained in NVIDIA Isaac Lab, takes proprioception, a 2.4 m × 1.6 m height scan at 0.1 m resolution and a foot-position command, and outputs joint position offsets. Training runs in three stages: obstacle-free pedipulation with a command-space curriculum, then movable cuboid obstacles in five scenarios with penalties on both contact events and contact forces, then added elevation-map noise for sim-to-real transfer. A boolean contact switch decides whether contact with the right front foot is penalised, so the same policy can either avoid an obstacle or push it.

Key insights

Penalising both contact events and contact force magnitudes blocks two exploits: pushing an obstacle with minimal force, and shoving it away with a short high-force push.
Five static training scenarios with cuboids were enough for the policy to avoid unseen shapes and moving obstacles; the authors infer it treats each occupied height-scan cell as a column of space to avoid rather than recognising object shapes.
Movable rather than static-terrain obstacles cost training time but let the policy learn to push objects when the contact switch allows it.
Training obstacle-free pedipulation first and adding obstacles afterwards speeds up training.
The policy tracks points, not trajectories, so reaching behind its back can leave it in a pose that is poor for following the rest of a path.

Main results

Free-space tracking over 1024 sampled commands: 0.057 m average error with the contact switch at 0 (avoid) and 0.047 m with it at 1 (contact allowed).
Reaching around a corner in simulation: the earlier blind policy hit the corner and fell; the perceptive policy took a longer path and avoided collisions.
Single-obstacle simulation tests: no self-collisions or environment collisions in all four tests; with a lowered obstacle the foot went over it instead of around.
Four obstacles: at a 1.3 m gap the policy stopped turning its base and reached behind its back, with some low-force collisions on the back of the base; clockwise command paths degraded tracking.
On ANYmal hardware the foot went around a cardboard box while the command passed through it (repeated five times), and around a round metal bin not seen in training.
Contact switch on hardware: with the switch set to 1 the robot pushed the obstacle over to reach the command, ending with 0.073 m tracking error.

Limitations

The authors state the controller handles some corner cases poorly, including obstacles directly under the base and confined spaces not seen in training.
The 2.5D height scan only represents structures that fit a 2.5D map and are larger than the grid step; the authors call for a 3D representation.
Side-on dynamic obstacles were only avoided on the robot's right side, and reaction speed is limited by the elevation-map update rate.
In simulation the policy sometimes got stuck because of its limited perceptive field of view or lack of space to rotate the base.
Avoiding contact with the pedipulating foot slightly reduces free-space tracking accuracy (0.057 m vs 0.047 m).

Field impact

Finalist for the Humanoids 2024 Best Interactive Presentation Award, with 12 citations. The paper does not link code, trained policies or data.

Abstract

Pedipulation leverages the feet of legged robots for mobile manipulation, eliminating the need for dedicated robotic arms. While previous works have showcased blind and task-specific pedipulation skills, they fail to account for static and dynamic obstacles in the environment. To address this limitation, we introduce a reinforcement learning-based approach to train a whole-body obstacle-aware policy that tracks foot position commands while simultaneously avoiding obstacles. Despite training the policy in only five different static scenarios in simulation, we show that it generalizes to unknown environments with different numbers and types of obstacles. We analyze the performance of our method through a set of simulation experiments and successfully deploy the learned policy on the ANYmal quadruped, demonstrating its capability to follow foot commands while navigating around static and dynamic obstacles.

Open Source Robotics