SenseNova-RoboRSI

Open runtime that runs, scores and improves robot agents driven by language models

License

• code and the SenseNova-RoboRSI-27B-Preview weights: Apache 2.0
• run logs dataset: MIT

Split licensing. RoboRSI's code and the SenseNova-RoboRSI-27B-Preview weights are Apache-2.0, and the run logs dataset is MIT. The experience packages dataset declares component-specific licences: DexJoCo material derived from GeoRT robot assets is CC BY-NC (non-commercial), the BEHAVIOR-1K packages come from a source with no repository licence, and Bench2Dex redistribution permission is recorded as pending. The benchmarks, simulators and models it connects to keep their own licences, and using GPT-6 Astra needs a paid OpenAI API account. Checked 2026-10-27.

Overview

SenseNova-RoboRSI is an open-source runtime from SenseNova for building, evaluating and improving embodied agents: a language model that sees camera images and controls a robot through tools. The repository connects a model, its prompts, skills and tools to a robot, runs it inside a benchmark's own environment with that benchmark's own scoring, and records every step. On top of that runtime, 'recursive self-improvement' (RSI) uses the recorded runs and task feedback to improve the agent's harness, its planning, feedback and memory handling, while the model's weights stay fixed, and keeps only changes that do better when retested. It ships adapters for LIBERO and its LIBERO-PRO, LIBERO-Long++ and LIBERO-Ultra-Long variants, RoboCasa365, BEHAVIOR-1K, Bench2Dex and DexJoCo, with RoboDojo and others announced. The team also released SenseNova-RoboRSI-27B-Preview, an open-weight 27-billion-parameter vision-language model trained on validated runs from the same harness. The code and the model weights are Apache-2.0. The project is marked pre-alpha, and SenseNova used it in its study of GPT-6 Astra as a robot controller.

An agent in RoboRSI is a model behind an OpenAI-compatible API that accepts images and tool calls, plus a configuration of prompts, context, caching and budgets. The runtime hands it observations from the benchmark and executes what it chooses: scripted actions, end-effector tools, or a call to a vision-language-action (VLA) policy that carries out a short skill. For LIBERO, an agent can download and deploy the π0.5 policy from openpi automatically and use it as its low-level executor, so the language model plans and reviews while the VLA moves the arm. Each benchmark keeps its native execution and scoring, and runs are recorded as full traces that can be exported as training episodes.

Evaluation is one command. scripts/evaluation/evaluate_model.py takes one or more benchmarks, downloads or builds the environment, prepares the in-context-learning (ICL) material, and runs tasks and seeds with live progress, resumable run IDs and a summary.json per run. Prebuilt Docker images for most benchmarks are on GHCR, and existing benchmark services can be reached over HTTP instead of being installed locally. The ICL experience packages, the notes and references an agent studies before a test episode, are published per benchmark in the SenseNova-RoboRSI-Packages dataset, and the logs behind the published runs in SenseNova-RoboRSI-Log.

The RSI loop treats the harness as the thing being trained. Execution traces and task feedback suggest changes to action prediction, planning, feedback and memory organisation; each candidate is retested under the same task conditions, and agent versions are tracked so validated improvements carry forward. RoboRSI-27B-Preview extends the loop to the model: validated interaction experience becomes training data for goal understanding, action decisions and correction from feedback, and the updated model goes back into the same runtime for retesting.

The team reports, with GPT-6 Astra driving the harness, an average score of 60.08 and a 54.17% success rate on RoboDojo-Sim, against 28.97 and 22.48% for GPT-6 Astra alone, and 94.50% on LIBERO-PRO's Task and Swap suites. The 27B model, built on Qwen3.8-27B, reaches 92.5% on LIBERO-PRO with VLA-assisted execution and 95.0% with direct control, and raises RoboCasa365 success over its base model from 26.0% to 46.5% while cutting average output tokens from 59.5K to 15.9K. A real-robot demonstration shows it placing a block into a box in 13 minutes, where the base model had not succeeded after 120 minutes.

Why it matters

Frontier language models can now drive robots well enough that the harness around them, what the model sees, which actions it is offered and how its mistakes are fed back, decides much of the result. RoboRSI makes that harness an open, shared object: one runtime that plugs a model into several established manipulation benchmarks under their own scoring, records everything, and improves the harness from its own runs. It comes with the experience packages, run logs and an open 27B model, so a lab can reproduce the published numbers, compare its own model under the same conditions, or start from an open-weight agent instead of a paid API.

Problem it solves

Evaluating a language-model agent on robot benchmarks usually means writing a separate integration for each simulator, each with its own observation format, action interface and scoring, and the comparisons that result are hard to reproduce. Improving the agent is then manual prompt work. RoboRSI gives one runtime and configuration format across benchmarks that keep their native scoring, records full traces, and turns those traces into tested improvements to the harness and, with the 27B model, into training data.

Key takeaways

• One runtime connects a model, prompts, skills and tools to LIBERO and its variants, RoboCasa365, BEHAVIOR-1K, Bench2Dex and DexJoCo, each scored by its own rules
• Recursive self-improvement updates the harness from recorded runs while model weights stay fixed, keeping only changes that score better when retested
• A language model can plan and review while a VLA such as π0.5 executes short skills; LIBERO agents can deploy π0.5 automatically
• With GPT-6 Astra the team reports 60.08 average score and 54.17% success on RoboDojo-Sim, against 28.97 and 22.48% for the model alone
• SenseNova-RoboRSI-27B-Preview is an open-weight 27B vision-language model on Qwen3.8-27B, released under Apache-2.0
• Experience packages, run logs and prebuilt Docker images are published alongside the code

Who it is for

• Researchers evaluating language-model or VLM agents on robot manipulation benchmarks
• Teams comparing harness designs: action interfaces, VLA-assisted execution, context and memory
• Labs that want an open-weight embodied model to deploy and fine-tune rather than a paid API
• Benchmark maintainers looking for a common agent interface across simulators

Limitations

• Marked pre-alpha: the repository is weeks old, the technical report is still to come, and the RoboDojo integration and several other benchmarks are listed as coming soon
• The headline results are the team's own; the RoboDojo numbers are being submitted to the official leaderboard and may change, and the RoboCasa365 comparison uses a different evaluation setting from the official entries
• The strongest results use GPT-6 Astra, a paid frontier model; the open 27B model scores far lower on RoboDojo (26.13 average score, 20.83% success)
• Native benchmark images target Linux, enhanced LIBERO and RoboCasa365 need an NVIDIA GPU in Docker, and serving the 27B model needs a GPU server with vLLM
• Real-robot use is shown in a demonstration only; the released adapters are for simulation benchmarks
• Experience packages carry component-specific terms, some non-commercial, and external simulators, datasets and models keep their own licences