Differentiable Simulation

• Also called: differentiable physics, differentiable simulator, differentiable dynamics
• Category: Software & tooling › Simulation & twins
• Tags: simulation, automatic-differentiation, system-identification, trajectory-optimization, sim-to-real, physics-engine

Physics simulation that can compute gradients of its outputs with respect to its inputs and parameters, so robot controllers, designs and models can be optimized directly by gradient descent.

Definition

Differentiable simulation is physics simulation written so that derivatives can be computed through it. A conventional simulator takes a state, control inputs and physical parameters and returns the next state. A differentiable simulator also returns how that next state, or any quantity computed from a whole simulated trajectory, changes with respect to those inputs. The gradients are usually obtained with automatic differentiation in frameworks such as JAX, PyTorch or NVIDIA Warp, or from analytic derivatives of the dynamics equations.

Gradients through physics open up several uses. In trajectory optimization, a sequence of controls can be improved by following the gradient of a cost through the simulated rollout. In policy learning, analytic policy gradients backpropagate a task loss through the simulator into the policy network, which can need far fewer samples than reinforcement learning methods that estimate gradients from rewards alone. In system identification, parameters such as mass, friction or stiffness are fitted to real measurements, and in co-design a robot's dimensions are optimized together with its controller.

Robotics is a demanding setting for differentiable physics because of contact. Impacts and friction create discontinuities and stiff dynamics, so the true gradient may be zero, undefined or misleading. Simulators handle this with smoothed or soft contact models, compliant penalty forces, or by differentiating through the optimization problems used to resolve contact. Each choice trades physical accuracy against gradient quality. Long rollouts add a second problem: gradients can explode or vanish over many time steps, and the optimization landscape can be rugged even when gradients exist.

Differentiable simulation is often combined with learned components. A neural network can model the parts of the dynamics that are hard to derive, such as actuator behavior or soft-body deformation, while the rest of the simulator stays physics-based, and the whole system is trained end to end. Differentiable rendering lets gradients flow from camera images back to scene parameters. Available tools include MuJoCo MJX, Brax and NVIDIA Warp.

Why it matters

Gradients let optimization use the structure of physics instead of treating the simulator as a black box, which can make learning, calibration and design much more data-efficient. Differentiable simulation is also a practical way to fit simulators to real hardware, narrowing the gap between simulated and real behavior.

Key applications

• System identification — fitting physical parameters of robots and objects to measured data
• Trajectory optimization — refining control sequences by gradient descent through simulated dynamics
• Policy learning — training controllers with analytic gradients through the simulator
• Soft robot modeling — calibrating deformable-body simulations against real hardware
• Robot co-design — optimizing mechanical parameters together with the controller

In practice

A team using differentiable simulation might first record motion and joint torque data from their robot, then fit friction, damping and mass parameters by minimizing the difference between simulated and measured trajectories with gradient descent. With the calibrated model they can optimize a trajectory or train a policy by backpropagating a task loss through short simulated rollouts, often truncating the horizon or mixing analytic gradients with reinforcement learning to keep gradients stable. They check gradient quality carefully, for example against finite differences, and pay attention to contact settings, since soft contact parameters chosen for smooth gradients can make the simulation less faithful to the real robot.

Common misconceptions

• Differentiable simulation always gives better gradients than reinforcement learning — contact discontinuities and long horizons can make analytic gradients noisy or misleading
• Any simulator becomes differentiable by wrapping it in autodiff — non-smooth contact handling, branching code and iterative solvers need special treatment to yield useful gradients
• Differentiable means more accurate — differentiability is about computing derivatives; smoothing for good gradients can reduce physical accuracy

Who should know this

• Robot learning researchers working on model-based control and learning
• Simulation engineers building physics engines
• Soft robotics researchers calibrating deformable models
• Control engineers using trajectory optimization

Related technologies

• Automatic Differentiation
• JAX
• MuJoCo MJX
• NVIDIA Warp
• Trajectory Optimization
• System Identification

Related terms

• Digital Twin
• Sim2Real
• Reinforcement Learning
• System Identification

Related projects

• GENESIS: World's fastest physics engine: 43M FPS for embodied AI learning
• LeRobot: HuggingFace ML library for real-world robotics in PyTorch
• MuJoCo: DeepMind physics engine for robotics and ML research
• PyBullet: Python physics simulator for robotics and RL research with fast CPU-based simulation
• RLinf: Scalable infrastructure for post-training foundation models via RL
• Webots: Cross-platform robot simulator with built-in models, sensors, and development environment

Videos

• Finally, Differentiable Physics is Here!: Two Minute Papers
• Learning on the Fly: Rapid Policy Adaptation via Differentiable Simulation (RA-L 2026): UZH Robotics and Perception Group