Zetta ζ: An Efficient Closed-Loop Embodied Harness for Self-Evolving Physical Intelligence

Authors: Xin Ding, Liang Mi, Mingzhe Huang, Zixuan Wang, Chao Zhang, Zixu Hao, Fu Chen, Xiangyu Li, Yikai Zheng, Yaoyu Guo, Weijun Wang, Kun Li, Hao Wu, Yunxin Liu, Ting Cao · arXiv 2026

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

A frozen VLA is wrapped in code-based critics and recovery skills that agents diagnose, repair and validate from failed simulated rollouts, raising RoboCasa success from 73.6% to 93.6%.

Why it matters

It shows a way to raise a pretrained manipulation policy's reliability without retraining its weights: detect physical failure states such as grasp loss or stalled transport in code and run targeted recoveries. The learned critic-recovery pairs are defined on object-relative geometry and contact predicates, so they carry over to related tasks, and the paper treats rollout throughput as the rate limit on that improvement.

Problem

VLA and world-action policies run open-loop at inference, so a slip or small collision can cascade into task failure. Language-model agent harnesses reflect only after an episode ends, because large models cannot decide at action frequency, and post-hoc reflection cannot test alternatives or see the exact failure state. Learning from self-exploration is further limited by how fast rollouts can be generated.

Proposed solution

The base policy (π0.5 or GR00T N1.5) and an orchestrator agent stay fixed while a harness of code-based runtime critics, recovery playbooks and tools evolves. Critics run at action frequency and propose interventions that the orchestrator approves; offline agents cluster failed rollouts by first missing milestone and earliest divergence from successful runs, diagnose the root cause top-down from evaluation logic to control parameters, write a minimal patch with a re-entry contract for handing control back, and admit it only after it fixes every seed in its cluster and holds on held-out seeds. Z-Infra, built on Ray, separates environment workers from GPU model workers with batched inference, VLM/action-expert partitioning and optional quantisation.

Key insights

Code-based critics can monitor execution at action frequency, leaving the slow language-model agent to approve interventions and to do offline diagnosis.
Diagnosing failures top-down (evaluation, critic, state, planning, recovery, then parameters) and patching at the highest layer that fixes the fault avoids overfitting low-level parameters to one failure.
Gains often arrive abruptly once the agent finds the real physical bottleneck, such as grasp retention or approach geometry, after earlier symptomatic fixes stall.
Critic-recovery pairs defined on object-relative geometry and generic contact predicates transfer zero-shot to related tasks.
Agent-in-the-loop baselines that call an LLM at every decision point are slow; invoking agents only offline and running critics online cuts per-episode latency.

Main results

RoboCasa, 18 Atomic-Seen tasks with GR00T N1.5 as the frozen base: macro-average success rose from 73.56% to 93.56% over four repair rounds (78.71%, 84.85%, 90.54%, 93.56%), evaluated on 50 held-out seeds per task.
LIBERO-Pro with π0.5 as the frozen base: Goal (T) rose from 31.0% to 92.5% and Goal (S) from 38.0% to 89.0%; the abstract's 90.8% is the average of these two Goal settings, while Table 3's overall average including LIBERO-10 is 71.13% (from 32.00%), with LIBERO-10 (T) 50.0% to 63.0% and (S) 9.0% to 40.0%.
Zero-shot transfer of skills learned on PnP-Stove: PnP-Sink 58% to 82%, PnP-Cabinet 62% to 80%, PnP-Toaster 72% to 90%; skills from TurnOffStove raised three articulated tasks from a 64% to an 80% average.
Abrupt gains: Goal-T2 (wine bottle in bowl) went from 10% to 15% after an early fix, then to 95% once a grasp-retention critic was added; Goal-S6 (cream cheese) went from 5% to 90%.
Speed against the RPent agent baseline: the abstract gives an 11.1× inference speedup (91% lower latency), while Section 4.6 gives an 11.9× reduction in per-episode latency and Fig. 14 shows 12.0× at concurrency 16.
Z-Infra throughput on 8 A100 GPUs: 22.09 episodes/min at concurrency 16, 7.7× the harness without Z-Infra (2.88) and 12.8× RPent (1.72), reaching 35.1 episodes/min at concurrency 64; the introduction's 20.6× figure compares that 35.1 with RPent's 1.7.

Limitations

All experiments are in simulation (LIBERO-Pro and RoboCasa on MuJoCo/robosuite); the authors leave extension to real robots and the sim-to-real gap to future work.
The authors state the results hold under their current rollout budget, and that self-evolution scales success toward the frozen policy's capability ceiling rather than beyond it.
Benchmark tables compare against the frozen base policies only; the only embodied-agent baseline is RPent, used for latency and throughput, and the paper does not name which language model runs the orchestrator and evolutionary agents.
Headline numbers depend on the reporting choice: 90.8% on LIBERO-Pro covers the Goal suites, while the overall LIBERO-Pro average including LIBERO-10 is 71.13%.
Diagnosis relies on simulator internals such as contact forces, internal states and official task predicates, which a real deployment may not expose in the same form.

Field impact

Hugging Face daily papers: 140 upvotes, in the top 5% of robotics papers there over the last six months. The paper gives a project page; it does not state a code release.

Abstract

Embodied agents are increasingly used to close the gap left by end-to-end policy models. Yet the agentic path has not realized closed-loop learning in physical execution: existing harnesses remain largely open-loop, following fixed skills during rollout and reflecting only after an episode completes. Such post-hoc reflection cannot govern execution as it unfolds, because physical interaction requires decisions to track rapidly changing robot-environment states at a frequency beyond today's large agentic models. We present Zetta, a closed-loop embodied harness that evolves code-based runtime critics and recovery skills online while keeping the base policy frozen. Through three timescale-separated loops, Zetta provides action-frequency governance, rollout-level critic-recovery proposal, and validation-gated skill updates. Together with Z-Infra, a rollout infrastructure decoupling agent logic from heterogeneous execution resources, Zetta achieves state-of-the-art success on LIBERO-Pro and RoboCasa under our current rollout budget, reaching 90.8% and 93.6%, with a 11.1× inference speedup; success continues to scale with self-exploration experience; learned skills transfer zero-shot, and clear robotic "Aha Moments" emerge. These results show that closed-loop harness self-evolution opens a scaling path for reliable physical intelligence.

Open Source Robotics