A Generalist Agent
Authors: Scott Reed, Konrad Żołna, Emilio Parisotto, Sergio Gómez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Giménez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley Edwards, Nicolas Heess, Yutian Chen, Raia Hadsell, Oriol Vinyals, Mahyar Bordbar, Nando de Freitas · TMLR 2022
Licences
- Open Access
- Paper: CC BY 4.0
Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.
TL;DR
Gato turns text, images, proprioception and actions into one token sequence and trains a single 1.2B transformer on 604 tasks, including block stacking with a real Sawyer arm.
Why it matters
The paper shows that one network with one set of weights can play Atari, caption images, chat and stack blocks with a real robot arm. Its robotics fine-tuning experiments test whether such a generalist adapts to new tasks with little data: on unseen block shapes the fine-tuned model recovers the expert's success rate with 10 episodes.
Problem
Control policies are usually built per domain with hand-crafted inductive biases, and even multi-task agents normally keep separate parameters for each task. Robotics data is scarce and covers few embodiments, so each hardware change means collecting new data and retraining.
Proposed solution
All data is serialised into a flat token sequence: SentencePiece text tokens, 16×16 image patches embedded by a ResNet block, and continuous values mu-law encoded and discretised into 1024 bins. A 1.2B-parameter decoder-only transformer (24 layers, embedding size 2048) is trained offline by supervised next-token prediction with the loss masked to text and action tokens. The task is selected by prompting the model with a demonstration from that task instead of a task identifier.
Key insights
- Across 79M, 364M and 1.18B parameters, in-distribution performance improves with model size for the same number of tokens processed.
- For few-shot fine-tuning on held-out cartpole.swingup and assembly-v2, pretraining on all data worked best; pretraining only on image and text data gave no benefit or negative transfer on these non-image tasks.
- Transfer is not universal: on held-out Atari boxing, a randomly initialised model beat every pretrained variant, which the authors attribute to Atari images being visually distinct from the other data.
- Model size was capped at the scale that allows real-time control of a real robot; to fit the 20 Hz loop, robot actions were sampled in one inference step with context length 1.
- Conditioning on a successful demonstration replaces task identifiers and, the authors argue, screens off confounders that cause 'self-delusion' when tasks share observation and action specifications.
Main results
- One set of weights scores above 50% of the expert score on over 450 of the 604 tasks.
- Real Sawyer RGB Stacking, Skill Generalization (5 held-out shape triplets, 200 episodes each): 50.2% average success, against 49% for the single-task BC-IMP baseline.
- Real-robot Skill Mastery: 75.6% average success, against 74.6% for BC-IMP.
- New real-robot task 'stack blue on green', fine-tuned on 500 demonstrations (2 h 45 min): 60% success, against 0.5% (1/200) for a behaviour-cloning baseline trained from scratch.
- Robot fine-tuning on Skill Generalization, real and simulated: matches the expert with 10 episodes and exceeds it at 100 or 1000 episodes.
- Atari: average human score or better on 23 games and over twice human score on 11; Meta-World: over 50% on 44 of 45 tasks; BabyAI BossLevel: 75%.
Limitations
- Context is limited to 1024 tokens; image observations can take over a hundred tokens each, so the agent may see only a few timesteps, and prompt-based in-context learning on new environments gave no significant gain over prompt-less evaluation.
- No web-scale dataset for control exists; Gato is trained by imitation on data from specialist RL agents, and the authors call acquiring suitable data an open research question.
- On the real robot, the 1.18B model on NVIDIA GeForce RTX 3090s still overran the 20 Hz control rate by about 0.01 s, even with context length 1.
- The single-task RL agents that generated the Atari data still outperform Gato, and its chat replies are often superficial or factually incorrect.
- The model card states it is not intended for commercial or production use, and that bias from the vision and language data is not mitigated beyond filtering sexually explicit content.
Field impact
Published in TMLR (11/2022) with Featured and Outstanding certifications, and listed for the ICLR 2024 Journal Track. Semantic Scholar counts 1,203 citations (2026-11-14). The paper includes a model card (Appendix A) and gives no code or model-weight link.
Abstract
Inspired by progress in large-scale language modeling, we apply a similar approach towards building a single generalist agent beyond the realm of text outputs. The agent, which we refer to as Gato, works as a multi-modal, multi-task, multi-embodiment generalist policy. The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens. In this report we describe the model and the data, and document the current capabilities of Gato.
Links
- arXiv 2205.06175: https://arxiv.org/abs/2205.06175
- PDF: https://arxiv.org/pdf/2205.06175v3