RoPotter: Toward Robotic Pottery and Deformable Object Manipulation with Structural Priors

Authors: Uksang Yoo, Adam Hung, Jonathan Francis, Jean Oh, Jeffrey Ichnowski · Humanoids 2024

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

RoPotter teaches a robot to shape clay on a pottery wheel into two bowl types from 40 demonstrations each, using a 2D cross-section of the clay and mesh-based recovery of hidden points.

Why it matters

It defines a long-horizon task for continuous volumetric deformable manipulation using a simple rigid finger with no degrees of freedom, and shows that cheap task priors (symmetry and local rigidity) can stand in for learned dynamics models. Its evaluation, comparing produced bowls with demonstrated ones by Chamfer distance and CLIP similarity, is a workable approach for tasks with no clear success label.

Problem

Volumetric deformable objects such as clay are hard to model: learned dynamics models need many samples and face a large sim-to-real gap in contact, so earlier clay work used discrete, task-specific actions with a clear view of the object between them. On a pottery wheel the clay deforms continuously under contact and is partly hidden by the robot's finger, so neither approach fits.

Proposed solution

RoPotter uses structural priors from the pottery task. Because the wheel makes the clay radially symmetric, RoPotter-2D represents the clay by a cross-section of points within 5 mm of a plane through its centre of mass, while RoPotter-Mesh fits a mesh to the initial cross-section, pushes vertices out of the finger, moves visible vertices to observed points and applies As-Rigid-As-Possible (ARAP) deformation to estimate hidden clay. A diffusion policy built on 3D Diffusion Policy (DP3), with a 2D encoder variant, is trained on these states from teleoperated demonstrations.

Key insights

Radial symmetry on the wheel lets the 3D clay state be reduced to a 2D cross-section that still captures diameter, height and wall thickness.
Handling finger contact and enforcing local rigidity with ARAP recovers clay hidden behind the finger; adding the ARAP step reduced Chamfer distance by 66.3%.
Because the opened diameter depends on clay mass, the policy must adapt the wall-lifting phase to the observed state; the DP3 baseline adapted least, making bowls too tall with more clay and too short with less.
The demonstrated bowls themselves vary by about 2.0 mm, which bounds how closely any policy can match them on Chamfer distance.

Main results

RoPotter-Mesh produced 44.4% lower final shape error than the DP3 baseline on the wide-bowl task with 250 g of clay (Chamfer distance 2.75 mm vs 4.95 mm).
Mesh reconstruction ablation (Chamfer distance on cross-section / full point cloud): full method 1.50 / 2.58 mm; without ARAP 4.45 / 7.74 mm; without contact handling 15.41 / 17.01 mm; observed reference 0.0 / 2.46 mm.
For all tasks with 250 g or 350 g of clay, the Chamfer distance distribution against demonstrated bowls was lower for both proposed policies than for the baseline.
DP3 was best only on the tall bowl with 350 g (1.96 mm), where RoPotter-2D also stayed under 2.0 mm (1.99 mm); DP3 failed one tall-bowl trial with 300 g.
CLIP similarity to the demonstrated bowls ranged from 0.940 to 0.974 across methods and agreed with the geometric metric.

Limitations

The authors state that RoPotter-Mesh relies on a well-initialised mesh and consistent, accurately observed robot actions, assumptions that hold in their controlled, structured setup but not in general.
Policies are not goal-conditioned; each bowl shape is learned from its own demonstrations, and defining a useful goal representation is left to future work.
The task has no clear success criterion, and the authors note that Chamfer distance is biased by outlier points and averages out high-frequency features.
The method assumes radial symmetry and quasi-static clay, ignoring dynamic effects.
RoPotter-Mesh policy results are reported only for the wide bowl; the tall-bowl comparison covers DP3 and RoPotter-2D.

Field impact

Finalist for the Humanoids 2024 Best Oral Presentation Award and cited 11 times. The authors make supplemental materials, experiment data and visualisations available at robot-pottery.github.io.

Abstract

Humans are capable of continuously manipulating a wide variety of deformable objects into complex shapes. This is made possible by our intuitive understanding of material properties and mechanics of the object, for reasoning about object states even when visual perception is occluded. These capabilities allow us to perform diverse tasks ranging from cooking with dough to expressing ourselves with pottery-making. However, developing robotic systems to robustly perform similar tasks remains challenging, as current methods struggle to effectively model volumetric deformable objects and reason about the complex behavior they typically exhibit. To study the robotic systems and algorithms capable of deforming volumetric objects, we introduce a novel robotics task of continuously deforming clay on a pottery wheel. We propose a pipeline for perception and pottery skill-learning, called RoPotter, wherein we demonstrate that structural priors specific to the task of pottery-making can be exploited to simplify the pottery skill-learning process. Namely, we can project the cross-section of the clay to a plane to represent the state of the clay, reducing dimensionality. We also demonstrate a mesh-based method of occluded clay state recovery, toward robotic agents capable of continuously deforming clay. Our experiments show that by using the reduced representation with structural priors based on the deformation behaviors of the clay, RoPotter can perform the long-horizon pottery task with 44.4% lower final shape error compared to the state-of-the-art baselines.

Open Source Robotics