Grasp Pose Detection in Point Clouds

Authors: Andreas ten Pas, Marcus Gualtieri, Kate Saenko, Robert Platt · IJRR 2017

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

Sample thousands of 6-DOF gripper poses on any visible surface, score each with a small CNN on multi-view projections of the local cloud, pick one by heuristics: 93% success in dense clutter.

Why it matters

The method needs no segmentation, no object identity and no CAD model at run time, which suits open-world picking. The ablations on the robot show how much each stage contributes: point cloud quality, grasp classification and grasp selection heuristics each change the success rate by 9 to 40 points.

Problem

Classic grasp pipelines register a CAD model to the point cloud, which is hard with noisy, partial clouds and assumes a model exists for every object. Earlier grasp detection methods reached 75–95% grasp success on isolated objects or light clutter, too low for practical use and not representative of real clutter.

Proposed solution

Grasp candidates are sampled on any visible surface by building a local Darboux frame at random cloud points and pushing a gripper model forward until contact, keeping poses that are collision-free with points inside the closing region. Each candidate is encoded as a 15-channel (or cheaper 3-channel) image from three projections of occupied voxels, occluded voxels and surface normals, and scored by a four-layer LeNet-style CNN. The CNN is trained on 1.5M labelled exemplars generated by checking frictionless antipodal grasps on BigBird meshes registered to real point clouds.

Key insights

Grasp representation matters: the 15-channel descriptor with surface normals and multiple views beats the Herzog/Kappler 3-channel heightmap by at least 10% classification accuracy
Pretraining on simulated CAD data helps early (same accuracy at 4000 iterations as 20000 without) but the benefit fades, and simulation-only training reached only 83% versus about 90% with real data
Knowing the object category or instance raises accuracy: about 97% classification for known cylindrical instances versus about 92% with no prior knowledge
Grasp detection needs high precision more than high accuracy, so the paper measures recall at 99% precision
Detected grasps can serve as object proposals for an object classifier, letting the robot grasp a named object

Main results

Active point cloud (wrist sensor with InfiniTAM): 93% grasp success (268 of 288 attempts) over 30 rounds of 10 objects, with 89% of objects cleared
Passive point cloud (two body-mounted Asus Xtion Pro): 84% success (116 of 138), 77% of objects cleared
Without the grasp selection heuristics: 75% success (117 of 155); without grasp classification: 53% success (75 of 142)
Recall at 99% precision: 89% for the 15-channel representation, 82% for 12 channels, 19% for the Herzog/Kappler representation
Runtime per 1k candidates on a 66k-point cloud: 2.1 s with the 3-channel descriptor versus about 8 s with 15 channels
Grasps as object proposals in clutter: 85.4% object classification accuracy (1023 of 1197 proposals) over 9 objects

Limitations

With no segmentation, the detector can produce a grasp that treats two adjacent objects as one; this is expected to grow with larger-aperture hands
Grasping a specific object relies on object classification that reached only 85% in the lab; improving it is needed before real-world use
The method is restricted to two-finger hands (or hands that act like one), and the sampling advantage may disappear for hands with more complex closing regions
The Baxter gripper's 3–7 cm grasp width limited the test objects, and two objects needed to be turned upright by hand when they landed upside down
Training data from simulated depth images transfers poorly to real sensors (83% versus about 90% accuracy)

Field impact

722 citations. The authors release their implementation as a C++ ROS package (github.com/atenpas/gpd) and define a dense clutter benchmark protocol for grasping experiments.

Abstract

Recently, a number of grasp detection methods have been proposed that can be used to localize robotic grasp configurations directly from sensor data without estimating object pose. The underlying idea is to treat grasp perception analogously to object detection in computer vision. These methods take as input a noisy and partially occluded RGBD image or point cloud and produce as output pose estimates of viable grasps, without assuming a known CAD model of the object. Although these methods generalize grasp knowledge to new objects well, they have not yet been demonstrated to be reliable enough for wide use. Many grasp detection methods achieve grasp success rates (grasp successes as a fraction of the total number of grasp attempts) between 75% and 95% for novel objects presented in isolation or in light clutter. Not only are these success rates too low for practical grasping applications, but the light clutter scenarios that are evaluated often do not reflect the realities of real world grasping. This paper proposes a number of innovations that together result in a significant improvement in grasp detection performance. The specific improvement in performance due to each of our contributions is quantitatively measured either in simulation or on robotic hardware. Ultimately, we report a series of robotic experiments that average a 93% end-to-end grasp success rate for novel objects presented in dense clutter.

Open Source Robotics