RLDX-1 Technical Report

Authors: Dongyoung Kim, Huiwon Jang, Myungkyu Koo, Suhyeok Jang, Taeyoung Kim, Beomjun Kim, Byungjun Yoon, Changsung Jang, Daewon Choi, Dongsu Han, Donguk Lee, Heeseung Kwon, Hojin Jeon, Jaehyun Kang, Jaekyoung Bae, Jihyuk Lee, Jimin Lee, John Won, Joonwoo Ahn, Junhyeong Park, Junyoung Sung, Kyungmin Lee, Minseong Han, Minsung Yoon, Sejune Joo, Seonil Son, Seungcheol Park, Seunggeun Cho, Seungjun Moon, Seungku Kim, Yonghoon Dong, Yongjin Cho, Youngchan Kim, Jinwoo Shin · arXiv 2026

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

RLDX-1 adds motion, memory and torque/tactile streams to a Qwen3-VL-based flow-matching VLA and beats π0.5 and GR00T N1.6 in simulation and on ALLEX, OpenArm and Franka robots.

Why it matters

The report tests these capabilities on real hardware with tasks where vision alone is insufficient, and shows how the baseline VLAs fail (fixed-speed conveyor motions, random or repeated box choices). It also documents an inference pipeline that runs an 8B-backbone VLA at 43.7 ms per step on one desktop GPU.

Problem

VLAs handle scene understanding and instruction following, but many real tasks also need motion awareness (picking from a moving conveyor), memory of earlier events (a shell game) and physical sensing when vision is occluded or uninformative (plug insertion, pouring to a target weight). Existing VLAs mostly act on static current observations.

Proposed solution

RLDX-1 fine-tunes Qwen3-VL 8B on robot VQA, adds a space-time self-similarity motion module with multi-frame compression, a memory queue of past cognition features, and a Multi-Stream Action Transformer, an MM-DiT-style flow-matching action model with separate cognition, action and physics streams joined by self-attention; the physics stream takes torque and tactile signals and predicts their future values. Training runs in three stages (multi-embodiment pre-training, embodiment mid-training on ALLEX and Franka Research 3 with video-generated synthetic data, task post-training with optional RECAP-style RL), and static CUDA-graph capture plus hand-written fused kernels cut inference latency.

Key insights

Separate modality streams let torque and tactile inputs be masked out when absent, so one architecture serves robots with and without physical sensors.
Physical sensing changes behaviour at contact-critical steps: with torque feedback RLDX-1 notices the cup's weight change when pouring, and with tactile input it pushes down once the plug reaches the socket.
Video-generated synthetic data helps: GR-1 Tabletop success rose from 41.0% with real data only to 50.1% with the full synthetic set, increasing with synthetic share.
Taking VLM features from an intermediate layer (18, 60.9%) beats an early (8, 51.1%) or late (28, 56.3%) layer on RoboCasa Kitchen; robot-VQA training adds 3.4 points.
Torch Compile alone gives only about 1.18× over eager execution because the graph fragments; static graph conversion plus fused kernels reach 1.63×.

Main results

ALLEX humanoid (four tasks): average 86.8 versus 39.1 for π0.5 and 44.8 for GR00T N1.6; Object-in-Box Selection 91.7% vs 33.3% and 29.2%; Conveyor Pick-and-Place 87.5% vs 29.2% and 50.0%.
Simulation: LIBERO 97.8%, LIBERO-Plus 86.7%, SIMPLER Google-VM 81.5%, Google-VA 77.4%, WidowX 71.9%, RoboCasa Kitchen 70.6%, GR-1 Tabletop 58.7% (GR00T N1.6: 47.6%), RoboCasa365 32.1%.
Franka Research 3: Spin Tracking 97.9% (π0.5 32.3%), Shell Game 91.7% (baselines about 50%), Egg Pick-and-Place 61.1% (π0.5 45.8%), Plug Insertion 33.3% (π0.5 20.8%).
OpenArm humanoid: Object Grounding 87.5% vs 33.3% for GR00T N1.6; Unseen Object 54.2% vs 37.5% for π0.5.
Three rounds of RECAP on ALLEX light-bulb twisting: 353 ± 22 frames and 4.1 ± 0.3 attempts, versus 1056 ± 326 frames and 12.7 ± 3.0 attempts with behaviour cloning.
Latency on an RTX 5090: 71.2 ms (PyTorch eager) → 43.7 ms (1.63×) for the all-modality model.

Limitations

The report has no limitations section; the points below come from its own reported results.
Several functional tasks remain far from solved: Plug Insertion 33.3% and Cup Swapping 45.8% on Franka Research 3; RoboCasa365 composite tasks 19.0% (seen) and 5.6% (unseen).
Tactile and torque supervision comes only from small in-house datasets on two platforms (ALLEX and Franka Research 3).
RL refinement is shown on a single task (light-bulb twisting).
The ALLEX average mixes success rates with progress scores (Card Slide-and-Pick, Pot-to-Cup Pouring); the abstract gives 86.8% while the conclusion says approximately 90%.

Field impact

On Hugging Face daily papers it drew 106 upvotes, in the top 5% of robotics papers there over the last six months. The report links a GitHub repository (github.com/RLWRLD/RLDX-1), and its contributor list records release planning and licensing of data, code and model weights for public distribution.

Abstract

While Vision-Language-Action models (VLAs) have shown remarkable progress toward human-like generalist robotic policies through the versatile intelligence (i.e., broad scene understanding and language-conditioned generalization) inherited from pre-trained Vision-Language Models, they still struggle with complex real-world tasks requiring broader functional capabilities (e.g., motion awareness, memory-aware decision making, and physical sensing). To address this, we introduce RLDX-1, a general-purpose robotic policy for dexterous manipulation built on the Multi-Stream Action Transformer (MSAT), an architecture that unifies these capabilities by integrating heterogeneous modalities through modality-specific streams with cross-modal joint self-attention. RLDX-1 further combines this architecture with system-level design choices, including synthesizing training data for rare manipulation scenarios, learning procedures specialized for human-like manipulation, and inference optimizations for real-time deployment. Through empirical evaluation, we show that RLDX-1 consistently outperforms recent frontier VLAs (e.g., π0.5 and GR00T N1.6) across both simulation benchmarks and real-world tasks that require broad functional capabilities beyond general versatility. In particular, RLDX-1 shows superiority in ALLEX humanoid tasks by achieving success rates of 86.8% while π0.5 and GR00T N1.6 achieve around 40%, highlighting the ability of RLDX-1 to control a high-DoF humanoid robot under diverse functional demands. Together, these results position RLDX-1 as a promising step toward reliable VLAs for complex, contact-rich, and dynamic real-world dexterous manipulation.

Open Source Robotics