Learned Perceptive Forward Dynamics Model for Safe and Platform-aware Robotic Navigation

Authors: Pascal Roth, Jonas Frey, Cesar Cadena, Marco Hutter · RSS 2025

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

A learned model predicts ANYmal's future poses and failure risk from a height scan and proprioceptive history, so an MPPI planner can plan safely with only a goal term and a risk term.

Why it matters

Because collisions and untraversable terrain are learned into the model, the planner reward reduces to two terms and needs no hand-built traversability map. The model is trained for a specific robot and locomotion policy, so its predictions reflect that platform, and it runs onboard a Jetson Orin AGX with 2048 sampled trajectories at 7 Hz.

Problem

Safe navigation in rough terrain needs real-time traversability assessment relative to what the specific robot and its locomotion policy can do. Planners built on simplified dynamics need hand-designed, environment-specific cost functions, and the earlier learned forward dynamics model for quadrupeds used only a 2D LiDAR scan, worked in flat scenes with 2D obstacles, and was not transferred to a real robot.

Proposed solution

The perceptive Forward Dynamics Model encodes a history of past states and proprioceptive measurements with a GRU and the current height scan with a CNN; a second GRU then takes a sequence of velocity commands and predicts, for ten 0.5 s steps, corrections to the commanded velocities (integrated into SE(2) poses) and a failure probability. It is pre-trained on simulated data from Isaac Lab, including collisions and falls, and fine-tuned on real ANYmal data. An MPPI planner samples action sequences, rolls them through the model, and selects them with a goal-distance reward and a failure-risk penalty.

Key insights

Simulation provides dangerous maneuvers such as falls and collisions that cannot be collected on hardware, while real-world data covers effects outside rigid-body simulation, such as slipping on snow and entanglement in vegetation.
Predicting velocity corrections and integrating them, rather than predicting poses directly, follows a residual formulation over the commanded actions.
The height scan is critical: removing it raised position error by 110% on average and lowered F1 by 0.13 (Appendix D).
Keeping the risk term in the planner raised success by 2.5% in 2D and 4.6% in 3D environments, at the cost of slower, more cautious paths.
A 2D-LiDAR baseline cannot tell stairs from walls, so it predicts collisions too often: higher recall but lower precision and F1.

Main results

Final-step position error 41.28% lower than the perceptive baseline of Kim et al. and 70.57% lower than a constant-velocity model, averaged over all environments (the abstract rounds this to 41%).
Failure prediction F1 of 0.90 in 2D and 0.85 in 2D-3D and 3D environments (Table II). Section VI states an accuracy of at least 89% and the conclusion at least 89.20%, while Table II's lowest accuracy is 89.13% (2D).
Planning success (Table III): 88.33% in 2D vs 78.33% (Kim et al.) and 82.50% (heuristic traversability); 73.75% in 3D vs 48.75% and 33.13%. The abstract reports this as a 27% higher success rate in rough simulation environments, and the conclusion as 81% average goal success.
Real-world fine-tuning reduced mean position error by 34.38% in forest, 30.55% in snow and 30.30% on pavement.
Onboard Jetson Orin AGX: 40.6 ms model inference per iteration with 2048 trajectories, planner at 7 Hz; the model has 1.16 million parameters and trains in about eight hours on one RTX 4090.
Separate models trained for ANYmal, Barry and ANYmal-on-Wheels in simulation gave different, platform-specific predictions for the same action sequences.

Limitations

The model stays in a primarily geometric domain and is bounded by the locomotion policy; it may fail on geometry not seen in training such as spiral staircases, caves or tunnels, or in extreme conditions such as ice or deep mud.
Simulated failure states do not translate perfectly to real failures, and real-world data contains no collisions because of the risk of hardware damage.
MPPI still needs tuning of the action distribution, command ranges and time-correlation factors, and the simulation and learning setup adds its own design choices.
The authors do not expect generalization beyond the training domain, and the method does not consider social norms or fast-moving and multiple dynamic agents.
In planar environments failures are rare (0.042%) and likely caused by simulation instabilities, so failure-prediction F1 there is low (0.10).

Field impact

The paper was a finalist for the RSS 2025 Outstanding Systems Paper Award and has 35 citations. Code and models are released at github.com/leggedrobotics/fdm.

Abstract

Ensuring safe navigation in complex environments requires accurate real-time traversability assessment and understanding of environmental interactions relative to the robot’s capabilities. Traditional methods, which assume simplified dynamics, often require designing and tuning cost functions to safely guide paths or actions toward the goal. This process is tedious, environment-dependent, and not generalizable. To overcome these issues, we propose a novel learned perceptive Forward Dynamics Model (FDM) that predicts the robot’s future state conditioned on the surrounding geometry and history of proprioceptive measurements, proposing a more scalable, safer, and heuristic-free solution. The FDM is trained on multiple years of simulated navigation experience, including high-risk maneuvers, and real-world interactions to incorporate the full system dynamics beyond rigid body simulation. We integrate our perceptive FDM into a zero-shot Model Predictive Path Integral (MPPI) planning framework, leveraging the learned mapping between actions, future states, and failure probability. This allows for optimizing a simplified cost function, eliminating the need for extensive cost-tuning to ensure safety. On the legged robot ANYmal, the proposed perceptive FDM improves the position estimation by on average 41% over competitive baselines, which translates into a 27% higher navigation success rate in rough simulation environments. Moreover, we demonstrate effective sim-to-real transfer and showcase the benefit of training on synthetic and real data. Code and models are made publicly available under https://github.com/leggedrobotics/fdm.

Open Source Robotics