Fabrica: Dual-Arm Assembly of General Multi-Part Objects via Integrated Planning and Learning

Authors: Yunsheng Tian, Joshua Jacob, Yijiang Huang, Jialiang Zhao, Edward Gu, Pingchuan Ma, Annan Zhang, Farhad Javid, Branden Romero, Sachin Chitta, Shinjiro Sueda, Hui Li, Wojciech Matusik · CoRL 2025

Licences

Open Access
Paper: CC BY 4.0

Everything below draws on the full paper under its CC BY 4.0 licence; its figure is credited to the authors.

TL;DR

Fabrica plans sequence, grasps, fixtures and motions for dual-arm assembly from part meshes, then uses RL insertion policies that transfer zero-shot, reaching 80% successful real steps.

Why it matters

It is an end-to-end route from a CAD assembly to robot execution without human demonstrations or domain-specific engineering, and the planner was demonstrated in simulation on several dual-arm setups. Planning takes minutes and a generalist insertion policy can handle an assembly it was not trained on, which targets the slow, manual reprogramming that today's assembly cells need for each new product.

Problem

Assembly robots today are programmed by hand with purpose-built infrastructure and stiff controllers, so they adapt slowly to new products and are sensitive to uncertainty. Research has mostly covered two-part, top-down insertion with one arm, while multi-part assembly needs varied insertion directions and grasps and a second arm holding parts against insertion forces. This couples a large discrete-continuous planning problem with contact-rich control that must generalise across part geometries.

Proposed solution

A hierarchical planner takes part meshes and the workcell, finds precedence tiers with a physics-based disassembly planner, pre-filters dual-arm grasp pairs with parallel IK and collision checks, and uses dynamic programming over an assemble/hold state tree to choose a stability-optimised sequence and grasps. It then generates a pickup fixture with moulded cavities and plans transit motions with RRT-Connect. For each insertion, a PPO policy trained in Isaac Gym works in a path-centric frame that maps every straight insertion to a top-down one and outputs residual corrections to the planned motion through a task-space impedance controller.

Key insights

Optimising sequence and grasps for stability, not just feasibility, helps downstream control: removing grasp optimisation cut overall real step success from 80% to 64%, and poor sequencing caused instability and part drift.
A path-centric, SE(3)-equivariant transform lets one policy learn only top-down insertion and drop orientation from its observations and actions; without it real step success fell to 61%.
Residual actions on top of the planned path and a plain negative-distance reward are enough, because the planner starts each insertion close to the goal.
Training with 3 mm pose noise and deploying with a Policy-Level Action Integrator, which applies actions to the last desired state, eases sim-to-real transfer.
Generalist policies trained on the other six assemblies did as well on an unseen assembly as specialists trained on it.

Main results

In the real world, 80% of assembly steps succeeded without intervention for both assembly-specialist and out-of-distribution generalist policies, against 55% for adapted ASAP and 23% for open-loop tracking.
Complete multi-step real assemblies succeeded 81% of the time with one intervention and 95% (specialist) or 100% (generalist) with two; without intervention all methods were near zero.
Planning a full assembly took 173 s (Beam) to 593 s (Duct) with parallelised stages.
In simulation (1,024 trials per assembly), RL policies reached 58.6-100% successful steps, against 0-24.2% for open-loop tracking.
The planner was demonstrated on Franka Emika Panda, UFactory xArm7 and UR5e arms with different grippers.
Benchmark of seven assemblies with 5 to 9 parts: beam, plumber block, car, gamepad, cooling manifold, duct and stool.

Limitations

Assumes insertion-only mating (no screwing or sliding), no reorientation of the subassembly, each part moved once, and parts light relative to robot payload (authors).
The benchmark is small because parallel grippers have limited grasp width; large or complex parts would need multi-finger hands or tool changing (authors).
Sensitive to calibration of the two arms and to wear of the pickup fixture; sub-millimetre shifts can cause failures (authors).
No perception: parts start in a custom pickup fixture rather than being picked from a bin (authors).
A clear sim-to-real gap remains: small parts slip between gripper pads, errors accumulate on large assemblies, and the holding gripper is not modelled in RL training (authors).

Field impact

Winner of the CoRL 2025 Best Paper Award, with 29 citations so far. The paper introduces a seven-assembly benchmark suite for multi-part assembly and links a project website (fabrica.csail.mit.edu); the licensed PMLR copy does not name a code repository.

Abstract

Multi-part assembly poses significant challenges for robots to execute long-horizon, contact-rich manipulation with generalization across complex geometries. We present Fabrica, a dual-arm robotic system capable of end-to-end planning and control for autonomous assembly of general multi-part objects. For planning over long horizons, we develop hierarchies of precedence, sequence, grasp, and motion planning with automated fixture generation, enabling general multi-step assembly on any dual-arm robots. The planner is made efficient through a parallelizable design and is optimized for downstream control stability. For contact-rich assembly steps, we propose a lightweight reinforcement learning framework that trains generalist policies across object geometries, assembly directions, and grasp poses, guided by equivariance and residual actions obtained from the plan. These policies transfer zero-shot to the real world and achieve 80% successful steps. For systematic evaluation, we propose a benchmark suite of multi-part assemblies resembling industrial and daily objects across diverse categories and geometries. By integrating efficient global planning and robust local control, we showcase the first system to achieve complete and generalizable real-world multi-part assembly without domain knowledge or human demonstrations. Project website: http://fabrica.csail.mit.edu/

Open Source Robotics