RLinf

Scalable infrastructure for post-training foundation models via RL

License

Apache-2.0

Overview

RLinf is a flexible and scalable open-source infrastructure for post-training foundation models (LLMs, VLMs, VLAs) via reinforcement learning. It supports embodied intelligence with multiple robot simulators (ManiSkill, LIBERO, IsaacLab, CALVIN, Metaworld) and real-world platforms (Franka Arm). Key features include support for diverse RL algorithms (PPO, GRPO, DAPO, Reinforce++), distributed training across multiple GPU nodes, 100%+ throughput improvement over baselines, and integration with popular VLA models like pi0, OpenVLA, and GR00T.

RLinf is a flexible and scalable open-source infrastructure designed for post-training foundation models, including LLMs, VLMs, and VLAs, via reinforcement learning for embodied intelligence applications. It integrates with five major robot simulators (ManiSkill, LIBERO, IsaacLab, CALVIN, and Metaworld) as well as real-world platforms like the Franka Arm, while supporting diverse RL algorithms including PPO, GRPO, DAPO, and Reinforce++. The infrastructure enables distributed training across multiple GPU nodes and achieves over 100% throughput improvement over baseline implementations. By providing a unified pipeline connecting popular VLA models such as pi0, OpenVLA, and GR00T with simulation and hardware environments, RLinf eliminates the fragmentation that previously required custom engineering for each combination of model, algorithm, and simulator.

Why it matters

RLinf addresses a critical gap — the infrastructure needed to train foundation models for robotics using reinforcement learning. It supports the full pipeline from LLMs to VLAs to embodied agents, making it the go-to infrastructure for post-training robot foundation models.

Problem it solves

Post-training robot foundation models with RL requires specialized distributed training infrastructure that combines language model training with robot simulation. RLinf provides this infrastructure as an open-source package.

Key takeaways

• Flexible infrastructure for post-training foundation models via RL
• Supports LLMs, VLMs, and VLAs for robotic applications
• Multiple robot simulator integrations for embodied training
• Scalable architecture for distributed training
• Open-source implementation of state-of-the-art RL training pipelines

Who it is for

• Researchers fine-tuning foundation models for robotics
• Teams training VLA models with reinforcement learning
• Developers building RL infrastructure for embodied AI
• Companies scaling robot model training

Limitations

• Requires significant compute resources for training
• Complex setup with multiple simulator dependencies
• Rapidly evolving field means the API changes frequently
• Knowledge of both RL and language model training needed