Language-Driven Representation Learning for Robotics

Authors: Siddharth Karamcheti, Suraj Nair, Annie S. Chen, Thomas Kollar, Chelsea Finn, Dorsa Sadigh, Percy Liang · RSS 2023

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

Voltron mixes language-conditioned masked reconstruction with caption generation on Something-Something-v2 and beats data-matched MVP and R3M on a new five-problem robotics evaluation suite.

Why it matters

Robot learning needs visual features for perception, language grounding and control, not only for control. With pretraining data and model capacity held equal, the paper shows that language during pretraining improves results even on tasks with no language input, and it provides one evaluation suite for comparing representations across these problems.

Problem

Visual representations pretrained on human video, such as MVP (masked autoencoding) and R3M (contrastive learning), were evaluated mainly on visuomotor control. Tested more broadly, they are inconsistent: MVP does well on grasp affordance prediction but struggles on language-conditioned imitation, while R3M does the reverse and degrades completely on grasp affordance prediction.

Proposed solution

Voltron feeds video frames and their captions into a masked-autoencoding pipeline whose Transformer encoder embeds vision and language jointly. With probability α it generates the caption from the visual context instead of conditioning on it, trading low-level reconstruction features against high-level semantics; three variants are studied: V–Cond (one frame, α = 0), V–Dual (two frames, α = 0) and V–Gen (two frames, α = 0.5). The authors also build an evaluation suite of five problems: grasp affordance prediction, referring expression grounding, single-task visuomotor control in simulation, language-conditioned imitation on a real Franka arm, and zero-shot intent scoring.

Key insights

Conditioning masked reconstruction on language helps even without language at test time: V–Cond beats R-MVP and the larger MVP (EgoSoup) on grasp affordance prediction.
No Voltron variant is best everywhere: language generation (V–Gen) is needed for intent scoring but hurts some control tasks, and single-frame V–Cond does well on non-episodic tasks but trails dual-frame models on control.
Generative language modelling held up better than masked language modelling: a reproduced M3AE overfit its language loss to perplexity 1 within the first 20 epochs and scored lower on every evaluation tested.
Fusing vision and language inside the encoder during pretraining, rather than combining separate text and image encodings afterwards, is the authors' explanation for the large gains on referring expression grounding.
Multiheaded attention pooling (MAP) for feature extraction improved every ViT model and nearly doubled visuomotor control success compared with the extraction procedure of the original MVP work.

Main results

Grasp affordance prediction (ARC Grasping, ViT-S): V–Cond scores 85.15 / 80.71 / 47.45 at Top-1 / Top-1% / Top-5% precision, against 72.94 / 61.47 / 39.77 for R-MVP and 40.38 / 40.55 / 28.66 for R-R3M (Table 2).
Referring expression grounding (OCID-Ref, total accuracy at 0.25 IoU): V–Cond 89.38 against R-R3M 63.30, R-MVP + DistilBERT 49.58 and CLIP ViT-B 68.35 (Table 3).
Scaling V–Cond from ViT-S (22M parameters) to ViT-B (86M) raises grasp Top-5% precision from 47.45 to 62.33 and referring expression total accuracy from 89.38 to 90.77.
Ablation (Table 4): without language, grasp Top-1% drops from 80.71 to 65.83, referring accuracy from 89.38 to 53.44 and Franka Kitchen success (n = 25) from 38.2 ± 5.09 to 33.1 ± 4.79; R-M3AE scores 52.79, 51.61 and 24.0 ± 4.21.
Zero-shot intent scoring on WHiRL videos: V–Gen tracks progress on 'opening the faucet' for both human and robot videos, while CLIP and R3M give noisy scores (qualitative, Figure 7).
Real-robot language-conditioned imitation (5 tasks, 20 demonstrations each): Voltron models give the highest success and, with R-MVP, stay robust to visual distractors; these numbers are reported only in Figure 6.

Limitations

The authors lacked the compute to train at the scale of the original MVP and R3M; head-to-head comparisons use their own reproductions on Something-Something-v2, and official MVP, R3M and CLIP models are marked as not directly comparable (stated).
No single representation is strongest on all tasks; trade-offs between low- and high-level features remain (stated).
Absolute success rates on Franka Kitchen control remain low; learning closed-loop policies from limited data remains open (stated).
Pretraining used at most two-frame contexts because four frames exceeded available memory, and α = 1 (generation only) was not evaluated.
Intent scores are not perfect: after the faucet is turned on, scores stay high for the human video but taper off for the robot video, for reasons the authors could not explain.

Field impact

Finalist for the RSS 2023 Best Paper Award, with 242 citations. The authors release the pretrained V–Cond, V–Dual and V–Gen models (plus a ViT-Base V–Cond and intermediate checkpoints), pretraining and adaptation code, preprocessed Something-Something-v2 data with the exact training batches, and a standalone evaluation suite.

Abstract

Recent work in visual representation learning for robotics demonstrates the viability of learning from large video datasets of humans performing everyday tasks. Leveraging methods such as masked autoencoding and contrastive learning, these representations exhibit strong transfer to policy learning for visuomotor control. But, robot learning encompasses a diverse set of problems beyond control including grasp affordance prediction, language-conditioned imitation learning, and intent scoring for human-robot collaboration, amongst others. First, we demonstrate that existing representations yield inconsistent results across these tasks: masked autoencoding approaches pick up on low-level spatial features at the cost of high-level semantics, while contrastive learning approaches capture the opposite. We then introduce Voltron, a framework for language-driven representation learning from human videos and associated captions. Voltron trades off language-conditioned visual reconstruction to learn low-level visual patterns, and visually-grounded language generation to encode high-level semantics. We also construct a new evaluation suite spanning five distinct robot learning problems – a unified platform for holistically evaluating visual representations for robotics. Through comprehensive, controlled experiments across all five problems, we find that Voltron's language-driven representations outperform the prior state-of-the-art, especially on targeted problems requiring higher-level features.

Open Source Robotics