Learning a Unified Policy for Position and Force Control in Legged Loco-Manipulation

Authors: Peiyuan Zhi, Peiyang Li, Jianqin Yin, Baoxiong Jia, Siyuan Huang · CoRL 2025

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

One RL policy for legged robots accepts position and force commands and estimates external force from joint history, giving compliant control and force-aware imitation without force sensors.

Why it matters

Position tracking, force control, force tracking and impedance control come from one policy by choosing the commands, with no per-mode controller or retraining. Because force is estimated from proprioception, contact-aware demonstrations can be collected on legged robots that carry no force sensor.

Problem

Contact-rich loco-manipulation needs contact force and robot position to be modeled together, but legged manipulators usually have no force-sensing hardware. RL whole-body controllers mostly rely on precise position control, and large imitation datasets record only trajectories; the authors show that such trajectory-only data is not enough even for a basic contact task such as wiping a blackboard.

Proposed solution

The authors model end-effector behavior with a simplified impedance law, x_target = x_cmd + (F_ext + F_cmd − F_react)/K, and base behavior with the equivalent velocity form, then train one PPO policy in Isaac Gym on randomly sampled position, velocity and force commands plus external disturbance forces up to 60 N. The policy encodes 32 steps of observation history, uses a state estimator to predict external force, end-effector position and base velocity, and outputs joint position targets for a PD controller. During teleoperation the estimated force is recorded with RGB images and robot state, and a diffusion-based imitation policy learns to output both position and force commands for the low-level policy.

Key insights

Position control, force control, impedance control, force tracking and hybrid position/force control all follow from one equation by setting the position and force commands; no extra training is needed per mode.
A two-stage curriculum (whole-body reaching and locomotion first, random force commands and disturbances second) trained more stably than a single stage.
PD gains lower than in pure position-based whole-body control improved force estimation, but gains that were too low reduced position-tracking accuracy.
At the base, external force is compensated through velocity: a G1 humanoid pushed against its velocity command halts and leans, and the B2 walks forward when kicked with zero commands.
Estimated force helps most where vision is uninformative, such as a push-to-open cabinet with a 3 mm stroke or a drawer front that becomes occluded during the task.

Main results

Force-aware imitation vs the same pipeline without force (50 trials per task, Table A.3): wipe-blackboard 0.58 vs 0.22, open-cabinet 0.70 vs 0.36, close-cabinet 0.72 vs 0.30, open-drawer-occlusion 0.76 vs 0.30, about 39.5% higher success on average.
The abstract and Section 4.2 attribute the 39.5% gain to four tasks; the introduction's contribution list says three tasks.
Simulation: end-effector tracking error stays mostly within 0.1 m with or without matching force commands, and end-effector position estimation error stays within 0.05 m on all axes.
Real robot force control from 0 N to 60 N, measured with a dynamometer at five end-effector positions: average error within 10 N; force estimation across six levels: 5–10 N error. Y and Z axes were tested only to 40 N because of Unitree Z1 hardware limits.
With a 2.5 kg dumbbell on the end-effector, a 25 N upward force command holds it in place; without the command the end-effector drops.
Demonstrations: 50 trajectories for wipe-blackboard (30k training steps) and 30 episodes for each cabinet and drawer task (20k steps).

Limitations

Force-estimation accuracy degrades in high-frequency interactions and at the edges of the robot's workspace.
A sim-to-real gap remains in force accuracy along different axes, especially Y, which the authors attribute to mismatches in actuator dynamics and contact modeling.
Force is estimated at a single interaction point, and end-effector and base are treated independently; multi-point and whole-body force interaction is left to future work.
The impedance model omits damping and inertia terms, which suits quasi-static tasks such as wiping or tug-of-war; dynamic skills would need higher control rates and velocity and acceleration terms.
Teleoperated data collection with the B2 wireless controller was slow, about 20 s per wiping trial and 10 s per cabinet trial.

Field impact

The paper received the CoRL 2025 Best Paper Award and has 35 citations. It points to a project website, unified-force.github.io; the paper text does not state a code release.

Abstract

Robotic loco-manipulation often involves contact-rich interactions with the environment, requiring the joint modeling of contact force and robot position. However, recent visuomotor policies often focus solely on learning position or force control, overlooking their co-learning. We propose the first unified policy for legged robots that jointly models force and position control learned without relying on force sensors. By simulating diverse combinations of position and force commands alongside external disturbance forces, we use reinforcement learning to learn a policy that estimates forces from historical robot states and compensates for them through position and velocity adjustments. This policy enables a wide range of manipulation behaviors under varying force and position inputs, including position tracking, force application, force tracking, and compliant interactions. Moreover, we demonstrate that the learned policy enhances trajectory-based imitation learning pipelines by incorporating essential contact information through its force estimation module, achieving approximately ∼39.5% higher success rates in four challenging contact-rich manipulation tasks over position-control policies. Experiments on both a quadrupedal manipulator and a humanoid robot validate the versatility and robustness of the proposed policy in diverse scenarios.

Open Source Robotics