ActivePusher: Active Learning and Planning with Residual Physics for Nonprehensile Manipulation

Authors: Zhuoyun Zhong, Seyedali Golestaneh, Constantinos Chamzas · ICRA 2026

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

A residual-physics push model with NTK-based uncertainty picks the most informative pushes to practise and the most reliable pushes to plan with, improving data efficiency and push-to-grasp success.

Why it matters

The approach needs no large offline dataset, high-fidelity simulator or human demonstrations, and the robot resets the object itself, so learning runs without human intervention. The same uncertainty estimate serves both data collection and planning, which links skill learning directly to task success.

Problem

Analytical pushing models rest on simplified friction, contact and mass assumptions, while learned models need much costly real interaction data and are unreliable in parts of the action space they have seen little of. Random data collection does not target the pushes that would improve the model most.

Proposed solution

A push is parameterised by side, lateral offset and distance; an MLP learns the residual between a quasi-static analytical pushing model and the observed SE(2) outcome. Epistemic uncertainty comes from treating the trained network as a Gaussian process with its Neural Tangent Kernel. During learning, a modified BAIT acquisition picks batches of pushes that most reduce posterior variance; during planning, the SST kinodynamic planner samples candidate pushes and keeps the one with lowest uncertainty.

Key insights

One uncertainty estimate serves two opposite uses: seek high uncertainty when choosing what to practise, avoid it when choosing what to execute
Adding a coarse analytical push model as a residual prior helps most when little data is available
Biasing SST's control sampling toward low-uncertainty pushes keeps the planner asymptotically optimal while avoiding poorly modelled actions
With active planning the planner tends to choose longer paths, which the model deems safer

Main results

Active planning with the best trained model raised Push-to-Grasp success up to 95% in simulation and 85% in the real world, and reduced tracking error compared with random control sampling
Residual model with active learning gave lower validation SE(2) RMSE than random data collection on four simulated YCB objects (cracker box, mug, mustard bottle, banana) and a real cracker box, with a batch size of 20
In simulation over 100 planning problems, residual active learning gave higher task success and lower tracking error than MLP and random-sampling baselines at every dataset size
Setup: pool of 1,000 candidate push parameters, 800 validation pushes, batches of 20 (simulation) or 40 (real), 1000 parallel Genesis environments for data collection

Limitations

Only epistemic uncertainty is modelled; aleatoric noise is ignored, so the planner may avoid low-noise regions or over-commit to inherently noisy pushes
Tasks and objects are limited to planar pushes in SE(2); non-convex, articulated objects, 3D pushing, rolling and throwing are not covered
Plans are executed open-loop without feedback or replanning
The analytical model approximates every object by a bounding rectangle, fixes the friction ratio at c = 0.05 for all objects and assumes sticking contact
Results are reported as curves in figures; apart from the 95% and 85% peak success rates, few exact numbers are given

Field impact

Finalist for the ICRA 2026 Best Paper Award on Planning and Control, with 9 citations. The paper does not name a code or data release.

Abstract

Planning with learned dynamics models offers a promising approach toward real-world, long-horizon manipulation, particularly in nonprehensile settings such as pushing or rolling, where accurate analytical models are difficult to obtain. Although learning-based methods hold promise, collecting training data can be costly and inefficient, as it often relies on randomly sampled interactions that are not necessarily the most informative. To address this challenge, we propose ActivePusher, a novel framework that combines residual-physics modeling with kernel-based uncertainty-driven active learning to focus data acquisition on the most informative skill parameters. Additionally, ActivePusher seamlessly integrates with model-based kinodynamic planners, leveraging uncertainty estimates to bias control sampling toward more reliable actions. We evaluate our approach in both simulation and real-world environments and demonstrate that it improves data efficiency and planning success rates compared to baseline methods.

Open Source Robotics