NeuralActuator: Neural Actuation Modeling for Robot Dynamics and External Force Perception

Authors: Zhiyang Dou, John U. Onyemelukwe, Hangxing Zhang, Heng Zhang, Minghao Guo, Yunsheng Tian, Michal Piotr Lipiec, Joshua Jacob, Chao Liu, Peter Yichen Chen, Yuri Ivanov, Wojciech Matusik · RSS 2026

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

A Transformer actuator model for low-cost servo arms, trained through differentiable MuJoCo from pose data, estimates end-effector force without a force sensor and improves force-aware imitation.

Why it matters

It gives low-cost arms a usable force signal without a force/torque sensor at inference time, with sub-millisecond GPU inference above a 60 Hz control rate. That signal raised behaviour-cloning success on payload tasks, and the model can be recalibrated on a new unit with 12 trajectories in about six minutes.

Problem

Low-cost servo arms use geared, brushed DC actuators where friction, hysteresis, backlash and thermal drift break the linear current-to-torque assumption. This corrupts simulation, effort tracking and any force estimate built on motor current. Prior learned actuator models need torque labels from joint-torque sensors or calibrated current, which these platforms do not provide reliably.

Proposed solution

NeuralActuator is a four-layer Transformer that reads nine frames of commands, joint states and actuator telemetry (current or load, voltage, temperature) and outputs a torque surrogate, a raw 3D end-effector force, a contact-probability gate and a per-motor condition score. The torque head is trained by rolling it through a differentiable MuJoCo/MJX simulator and matching measured poses, so no torque labels are needed; the force, gate and condition heads are trained directly on labels from the new Neural Actuation Dataset, recorded with a leader-follower pair of OpenManipulator-X arms and a six-axis force/torque sensor.

Key insights

On geared low-cost servos the linear current-to-torque map can fail outright: fitting Kt per joint on OpenManipulator-X gave negative slopes on two joints before clamping
Pose trajectories rolled through a differentiable simulator are enough to supervise a torque surrogate when no torque labels exist
Separating force regression from a contact-probability gate keeps predicted force near zero when there is no contact, where classical observers raise false contacts
Fixed parametric friction models (Stribeck, backlash) add little over the linear model on these servos; a history-conditioned network captures the dominant nonlinearities
The predicted force, used as an extra input, improves imitation-learning policies on payload manipulation

Main results

Payload benchmark at 100 steps: force MAE 0.12 N vs 0.66 N for the best classical baseline (generalized momentum observer), 1.23 N for ID-Friction and 1.41 N for ID-Linear; classical methods showed 33-59% false-positive contact in the no-contact regime
Six-axis sensor benchmark with forces up to 9 N: force MAE 0.36-0.39 N at 100 steps and 0.38-0.57 N at 500 steps; 0.00-0.02 N predicted on no-contact references
Behaviour cloning over 40 trials: pick-and-place success rose from 80% to 92.5% and go-up-and-stay from 85% to 95% with NeuralActuator force input
Motor-condition detection (rubber-band restricted Joint 3): 91.0% accuracy, 96.2% recall, 0.95 AUC-ROC vs 67.1% accuracy and 0.72 AUC-ROC for the best handcrafted baseline (Random Forest)
Runtime: 1.44M parameters, 0.25 ms mean GPU inference, 4,019 Hz throughput at batch size 1
Cross-platform: Franka Panda offline force MAE 0.26-0.42 N over 200-600 g payloads; SO-101 force MAE 0.47-0.73 N at 500 steps; online adaptation on 12 trajectories (356 s) reduced tracking error by about 44%

Limitations

Long-horizon rollouts conditioned on recorded telemetry accumulate error and eventually degrade tracking accuracy (the authors mitigate this with few-trajectory online adaptation)
The torque surrogate is simulator-equivalent, not a measured actuator torque: when contacts or payloads are missing from the forward model it can absorb their effects and other model mismatch
Online inference needs live actuator telemetry, and the model cannot roll out unexecuted candidate commands, so it cannot be used for counterfactual planning such as MPC without an extra effort-response model
The force head estimates a single 3D resultant force at the end effector: no multipoint contact localisation, no full wrench, and it needs explicit force labels for training
Payloads at or below about 50 g (about 0.5 N) sit near the noise floor of the low-cost platform; the Franka results are an offline benchmark that uses a future-state proxy for the command signal

Field impact

Winner of the RSS 2026 Outstanding Systems Paper in Memory of Seth Teller Award and a finalist for the RSS 2026 Outstanding Paper Award; 5 citations so far. The authors release the Neural Actuation Dataset and list a project page and code repository.

Abstract

Differentiable simulators have advanced policy learning and model-based control, yet actuator dynamics remain an important source of sim-to-real error. This is particularly acute on low-cost platforms, where the linear current-to-torque relation τ = K_t·I becomes unreliable during commanded-target tracking because of friction, hysteresis, backlash, and thermal effects. We present NeuralActuator, a neural actuator model that jointly predicts (i) a simulator-equivalent generalized-effort surrogate for trajectory propagation on low-cost servo platforms, (ii) external force with a contact-probability gate for sensorless force perception, and (iii) a motor-condition score for the supervised joint. We also introduce the Neural Actuation Dataset (NAD), collected with a twin-arm teleoperation system that records robot states and actuator telemetry together with external-force labels. The torque-surrogate head is trained through differentiable simulation from pose trajectories without direct generalized-effort labels, while the force, gate, and motor-condition heads receive direct supervision. A Transformer captures temporal dependencies while supporting real-time inference. We evaluate NeuralActuator on a 5-DoF OpenManipulator-X, a 6-DoF SO-101, and a 7-DoF Franka Emika Panda, spanning three actuator families and platforms costing approximately USD 500 to over USD 30,000. The low-cost platforms support dynamics and force evaluation, while the offline Franka experiment provides an additional payload-force-estimation benchmark. Experiments further demonstrate its application for motor condition estimation on OpenManipulator-X and improved behavior-cloning performance when NeuralActuator is used as a pretrained module.

Open Source Robotics