LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning

Authors: Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, Peter Stone · NeurIPS 2023

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

LIBERO is a simulated benchmark of 130 language-conditioned manipulation tasks with human demos, built to study how robots transfer declarative and procedural knowledge across tasks.

Why it matters

Because three suites separate spatial, object and goal knowledge, a drop in performance can be traced to the type of knowledge being lost. The experiments also challenge common practice: the lifelong learning algorithms tested had lower forward transfer than plain sequential fine-tuning, and naive supervised pretraining could hurt later lifelong learning.

Problem

Lifelong learning research has mostly studied transfer of declarative knowledge (entities and concepts) in vision and language tasks. Robot decision-making also needs procedural knowledge, knowing how to act, and there was no way to analyze systematically which kind of knowledge an agent transfers or forgets when it fails after learning new tasks.

Proposed solution

LIBERO generates tasks procedurally on top of robosuite: language templates are extracted from Ego4D activity annotations, PDDL scene files specify objects, layouts and initial states, and goals are conjunctions of predicates such as On(A, B) or Open(X). Four suites are provided: LIBERO-Spatial, LIBERO-Object and LIBERO-Goal (10 tasks each, each isolating one type of knowledge) and LIBERO-100 (90 short-horizon and 10 long-horizon tasks), with 50 teleoperated demonstrations per task. The study benchmarks three policy architectures (ResNet-RNN, ResNet-T, ViT-T) and five learning schemes (sequential fine-tuning, ER, EWC, PackNet and multitask learning) trained with behavioral cloning.

Key insights

Policy architecture matters as much as the lifelong learning algorithm: transformer temporal backbones (ResNet-T, ViT-T) worked much better than ResNet-RNN on average.
No single visual encoder was best everywhere: with ER, ResNet-T beat ViT-T on all suites except LIBERO-Object, which the authors link to the ViT handling greater object variety.
All three lifelong learning algorithms had lower forward transfer than sequential fine-tuning; PackNet prevents forgetting but its shrinking per-task capacity hurt it on LIBERO-Long.
Semantically rich task descriptions encoded with BERT, CLIP or GPT-2 gave no statistically significant gain over Task-ID embeddings.
Behavioral cloning loss is a misleading signal for checkpoint selection; EWC sometimes had the lowest loss without a good success rate, so success rate should be used.

Main results

With ResNet-T, sequential fine-tuning had the highest forward transfer on every suite: 0.54 (Long), 0.72 (Spatial), 0.78 (Object) and 0.77 (Goal).
PackNet had the lowest forgetting (negative backward transfer 0.06–0.17) and the best AUC on Spatial, Object and Goal (0.63, 0.60, 0.75), but ER had the best AUC on LIBERO-Long (0.32 versus 0.25).
EWC performed worst among the lifelong learning algorithms, with AUC 0.02 on LIBERO-Long and 0.06 on Spatial and Goal.
On LIBERO-Long with PackNet, ViT-T reached AUC 0.34 versus 0.25 for ResNet-T and 0.03 for ResNet-RNN.
Language embeddings on LIBERO-Long (ResNet-T with ER) gave forward transfer of 0.48 (BERT), 0.52 (CLIP), 0.46 (GPT-2) and 0.50 (Task-ID), with no significant difference.
Results are averaged over three seeds, with each task trained for 50 epochs on 50 demonstrations and evaluated over 20 rollouts.

Limitations

Stated by the authors: the experiments are an initial study, leaving open better architectures for spatial and temporal information, algorithms with better forward transfer, and pretraining that helps lifelong learning.
Stated by the authors: because the lifelong learner mainly learns from humans, preserving user privacy may become crucial in the long run.
Results depend on task ordering: performance of ER and PackNet varied across five orderings, and the difference was statistically significant for PackNet.
All tasks run in simulation on robosuite; the paper reports no real-robot experiments.
Only behavioral cloning from demonstrations is benchmarked; reinforcement learning is set aside because of the difficulty of sparse rewards.

Field impact

Cited 607 times (count as given). The paper releases the procedural task generation pipeline, four task suites totaling 130 tasks and human-teleoperated demonstrations for every task, with code and datasets linked from the project website.

Abstract

Lifelong learning offers a promising paradigm of building a generalist agent that learns and adapts over its lifespan. Unlike traditional lifelong learning problems in image and text domains, which primarily involve the transfer of declarative knowledge of entities and concepts, lifelong learning in decision-making (LLDM) also necessitates the transfer of procedural knowledge, such as actions and behaviors. To advance research in LLDM, we introduce LIBERO, a novel benchmark of lifelong learning for robot manipulation. Specifically, LIBERO highlights five key research topics in LLDM: 1) how to efficiently transfer declarative knowledge, procedural knowledge, or the mixture of both; 2) how to design effective policy architectures and 3) effective algorithms for LLDM; 4) the robustness of a lifelong learner with respect to task ordering; and 5) the effect of model pretraining for LLDM. We develop an extendible procedural generation pipeline that can in principle generate infinitely many tasks. For benchmarking purpose, we create four task suites (130 tasks in total) that we use to investigate the above-mentioned research topics. To support sample-efficient learning, we provide high-quality human-teleoperated demonstration data for all tasks. Our extensive experiments present several insightful or even unexpected discoveries: sequential finetuning outperforms existing lifelong learning methods in forward transfer, no single visual encoder architecture excels at all types of knowledge transfer, and naive supervised pretraining can hinder agents' performance in the subsequent LLDM. Check the website at https://libero-project.github.io for the code and the datasets.

Open Source Robotics