Predicting Object Interactions with Behavior Primitives: An Application in Stowing Tasks

Authors: Haonan Chen, Yilong Niu, Kaiwen Hong, Shuijing Liu, Yixuan Wang, Yunzhu Li, Katherine Driggs-Campbell · CoRL 2023

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

A particle-based GNN predicts object motion at behavior-primitive key poses; searching primitive parameters against 3-4 keyframes from one demo lets an arm stow objects into cluttered shelves.

Why it matters

The network is queried only at the key poses of each primitive, which the authors note cuts the number of forward passes and the cumulative prediction error over long horizons. Trained only on simulated box-shaped objects, the system transferred zero-shot to a real Kinova Gen3 arm and handled out-of-distribution objects such as a bottle, a lint roller and heavy or trapezoid-shaped items.

Problem

Stowing an object into a cluttered shelf means first making space by pushing and sliding the objects already there, which involves multi-object contacts and frictions that are hard to model, over a long horizon. Pre-programmed systems do not adapt to new shelf arrangements, category-level manipulation methods need costly data collection and human labelling, and model-free reinforcement learning needs too much training time to learn long-horizon tasks from scratch on a real robot.

Proposed solution

Each object is modelled as uniformly distributed particles, and a graph neural network trained in simulation predicts each object's rigid transformation after a behavior primitive (sweep, push or transport); the graph includes gripper-to-object edges, edges to static shelf and table particles, and an auxiliary loss on predicted gripper positions. At run time, random shooting samples each primitive's parameters, the network predicts the outcome, and the parameters whose predicted object positions are closest to the demonstration keyframe are executed with an Operational Space Controller.

Key insights

Representing the robot action as gripper particles with planned motion, and adding edges between dynamic objects and static shelf and table particles, improves prediction over a RoboCraft-style GNN on the harder sweep and transport primitives.
Predicting object states only at primitive key poses, instead of at every control step, reduces forward passes and accumulated error on long-horizon tasks.
A learned dynamics model combined with search outperformed model-free RL (SAC, PPO) and PPO with a primitive-parameterized action space on the same long-horizon stowing task in simulation.
Random Shooting, the Cross-Entropy Method and Gradient Descent performed similarly as the optimizer over primitive parameters.
A 2-skill set (push and transport) reached the same 95% real-world success as the 3-skill set, which the authors attribute to the model capturing interactions between the held object and the objects on the shelf.

Main results

Real world, six setups with 10 trials each: 95% success with 3 skills and 95% with 2 skills, against 18% for a hand-tuned heuristic without dynamics prediction.
RoboCraft as the learned dynamics in the same pipeline: Table 3 reports a 48% average, while its six per-setup scores in the same table (4, 7, 4, 6, 7 and 5 out of 10) add up to 33/60, which is 55%.
Simulation control error (MSE): 0.905 mm for the method, against 22.925 mm for parameterized PPO, 34.861 mm for the heuristic, 87.479 mm for PPO and 265.411 mm for SAC.
Transport-primitive prediction error: MSE 0.666 mm, EMD 31.232 mm and Chamfer distance 38.068 mm, against 1.091, 42.162 and 55.615 mm for RoboCraft; on the simpler push primitive RoboCraft has lower MSE (0.216 vs 0.292 mm), within one standard deviation.
Deformable objects (appendix): 6/10 with 3 skills and 4/10 with 2 skills, against 2/10 for RoboCraft and 0/10 for the heuristic.
Training data: 300 random-parameter episodes per primitive; push and sweep prediction accuracy plateaus at about 50 samples and transport at about 140.

Limitations

Relies on manual human labelling of ordered keyframes from demonstrations, which the authors say could restrict scalability to larger and more complex scenarios.
Objects are represented as box-shaped point clouds in training and inference, which the authors say may not reflect real object geometry where interactions and contacts are more complex.
Real-world perception treats every object as a cuboid and identifies it by SIFT matching against side and top images captured beforehand for a database of known objects.
Success drops with deformable objects (best 6/10).
The primitive library (sweep, push, transport) is hand-designed; the authors list primitive design and skill composition as open research directions.

Field impact

Finalist for the CoRL 2023 Best Paper / Best Student Paper Award and cited 32 times. The paper contributes a stowing benchmark for long-horizon manipulation built in robosuite and MuJoCo; it gives no code or data link.

Abstract

Stowing, the task of placing objects in cluttered shelves or bins, is a common task in warehouse and manufacturing operations. However, this task is still predominantly carried out by human workers as stowing is challenging to automate due to the complex multi-object interactions and long-horizon nature of the task. Previous works typically involve extensive data collection and costly human labeling of semantic priors across diverse object categories. This paper presents a method to learn a generalizable robot stowing policy from predictive model of object interactions and a single demonstration with behavior primitives. We propose a novel framework that utilizes Graph Neural Networks to predict object interactions within the parameter space of behavioral primitives. We further employ primitive-augmented trajectory optimization to search the parameters of a predefined library of heterogeneous behavioral primitives to instantiate the control action. Our framework enables robots to proficiently execute long-horizon stowing tasks with a few keyframes (3-4) from a single demonstration. Despite being solely trained in a simulation, our framework demonstrates remarkable generalization capabilities. It efficiently adapts to a broad spectrum of real-world conditions, including various shelf widths, fluctuating quantities of objects, and objects with diverse attributes such as sizes and shapes.

Open Source Robotics