RAI

Vendor-agnostic agent framework for ROS 2 robots and physical AI

License

Apache-2.0

Overview

RAI is a vendor-agnostic agentic framework for physical AI, built by Robotec.ai on ROS 2 tooling and maintained since mid-2024. Rather than being one agent, it is a set of packages you compose: rai core for the multi-agent system and human-robot interaction, rai_whoami to synthesise a robot's self-description from its documentation, images and URDFs, rai_asr and rai_tts for speech in and out, rai_perception for open-set object detection, rai_sim for connecting to simulators, rai_bench for benchmarking agents and models, and rai_nomad for navigation. The demos run on real platforms — an autonomous orchard tractor, a Franka Panda doing language-driven manipulation, and a Husarion ROSbot XL navigating.

The design choice that distinguishes it is in the name: vendor-agnostic. RAI does not assume a particular model provider, and it does not assume a particular robot — it assumes ROS 2 and builds upward. That makes it the most conservative of the current crop of agentic robotics frameworks, and conservative is the right word: it has been maintained since June 2024, which predates the wave of similar projects by well over a year, and it shows in the structure. Speech, perception, simulation and benchmarking are separate packages with their own boundaries rather than features of one monolith.

rai_whoami is the piece worth singling out, because it addresses a problem the others mostly ignore: an agent driving a robot needs to know what the robot is. It extracts and synthesises embodiment information from a structured directory of documentation, images and URDFs, which is a concrete answer to a question usually left to prompt engineering. rai_bench matters for the same reason — a framework that ships its own benchmarking suite is one that expects to be measured. Of the listed packages only rai_finetune, for fine-tuning language models on embodied data, is still unchecked on the roadmap.

Why it matters

Agentic frameworks for robots arrived all at once in 2026 and most of them are months old. RAI has been going since before the wave, from a robotics software house rather than a lab, and it is built on ROS 2 rather than around a particular model. For a reader deciding what to build on rather than what to read about, longevity and vendor-neutrality are the two properties that actually matter, and this is the entry in the category that has both.

Problem it solves

Connecting a language model to a robot usually means writing the glue yourself: speech, perception, the description of what the robot even is, and some way to tell whether the result works. RAI supplies those as separate ROS 2 packages that can be adopted individually, without committing to a model vendor or a robot vendor.

Key takeaways

• Vendor-agnostic by design: no assumed model provider, no assumed robot, built on ROS 2 tooling
• Composed of separate packages — core, whoami, asr, tts, perception, sim, bench, nomad — rather than one monolith
• rai_whoami synthesises a robot's own embodiment description from its documentation, images and URDFs, which is the piece most agent frameworks leave to prompt engineering
• rai_bench ships a benchmarking suite for agents, models, tools and simulators alongside the framework itself
• Demos run on real platforms: an autonomous orchard tractor, a Franka Panda with Grounded SAM 2, and a Husarion ROSbot XL
• Maintained since June 2024, which predates the 2026 wave of comparable frameworks by more than a year
• Multi-modal by construction — speech in and out, vision, and text interfaces are first-class

Who it is for

• ROS 2 teams adding natural-language interaction to an existing robot stack
• Anyone building multi-agent or multi-modal robot behaviour who does not want to be locked to one model provider
• Researchers who need a benchmarking suite for agent behaviour rather than just a runtime
• Integrators working on mobile robots, manipulators or field machinery on ROS 2

Limitations

• ROS 2 is assumed throughout — there is no path here for a robot that is not already on it
• Breadth has a cost: eight packages with different maturity levels, and rai_finetune is still unchecked on the roadmap
• The demos are simulation-first; the orchard tractor and ROSbot XL scenarios show the framework in a simulator rather than footage of deployed machines
• Vendor-agnostic means you supply the model and the keys, so running it is not free and the quality depends on choices the framework does not make for you
• Documentation lives on a separate site rather than in the repository, so the README alone does not tell you how to start
• A framework of this scope is a commitment; adopting one package is easy, adopting the architecture is not