Community builds of SO-101 Open Robot Arm

6-DOF robot arm for teleoperation and AI robotics research

License

Apache-2.0

• An SO-101 Balances a Ping-Pong Ball, Seeing It Only Through the Wrist Camera (zecice): An SO-101 holds a table-tennis paddle in its gripper and keeps a ping-pong ball balanced on it. The wrist camera is its only sensor for the ball; everything else comes from the arm's own joint readings. The controller was trained in MuJoCo and runs unchanged on the real arm, which the builder shows in a short clip on X: a hand pushes the ball towards the edge of the paddle and the arm brings it back to the middle. How it works The pipeline in the repository has four stages. Frames from the wrist camera (640x480, scaled down to 128x96) go to a colour-threshold detector that finds the orange ball. A small MLP estimator, trained in simulation, turns the last 3 detections into the ball's position on the paddle. The velocity is a filtered finite difference of those positions. A controller trained with reinforcement learning (PPO) then outputs 5 joint-position changes at 30 Hz. The paddle is clamped edge-on to the camera, so the camera sees it from the side and the ball near one image column. What made the jump from simulation work The builder's sim-to-real lesson is to train the controller with the real system's flaws from the start, a form of domain randomization: - Control latency randomised over 0 to 4 frames (0 to 133 ms), with the last 3 actions in the observation. - Soft, randomised servo gains, because cheap servos sag and lag. - A noisy ball state like the estimator's output, with its velocity computed by the same finite-difference filter the robot uses. A controller trained on the exact velocity failed completely on velocity derived from the camera. What did not work is written down too: an end-to-end CNN policy on wrist images, trained by behavior cloning and DAgger from the PPO controller, was fine in simulation but barely saw the real ball. The plain colour detector and the small estimator replaced it. On the real arm The hardware is an SO-101 follower arm, a wrist camera, a standard paddle clamped in the gripper and an orange ball. The arm runs through the SO-101 follower and Feetech drivers in LeRobot. The README lists what mattered on the real arm: - Servo P gain 32 rather than LeRobot's default of 16, at which the arm sags and ignores 4 to 7 degree commands. The run script sets it. - A level home pose: a few degrees of tilt and the ball runs off, because the policy has no integral action. Each arm records its own home pose with an included guide script. - Squeeze the paddle by load (about 10%) and never command the gripper fully closed: a stall trips the Feetech overload protection and the motor drops off the bus until the power is cycled. - Measure your own latency with the included script; the controller is trained for 0 to 4 extra steps of 33 ms. The README also warns that the arm moves on its own and stays powered after the scripts exit. Results and what is released The builder reports 97.5% of episodes balanced for 10 seconds in randomised simulation, 113 of 120 in simulation with the full camera pipeline, and that it balances on the real arm. The trained controller and estimator weights are in the repository, so the whole pipeline can be tried in the simulator without an arm. Training the controller took about 29 million steps and about 2 hours on a laptop (6 CPU cores for the simulation, an RTX A1000 GPU for the network). It was developed on Linux; macOS and Windows are untested. The code and weights are MIT. The SO-101 model files come from TheRobotStudio's SO-ARM100 repository under Apache-2.0, and the paddle mesh from a table-tennis control project under MIT.
• Zero-Shot MolmoAct2 on an SO-101, With the Calibration Trap Documented (Irene Grace): Running AI2's MolmoAct2 on an SO-101 with no training and no demonstrations: you give it a sentence and it works out the motions. The model takes an RGB frame from a side-view RealSense D455, a frame from a wrist webcam, and the current joint state, and returns a chunk of joint targets that are executed with temporal ensembling at 30 Hz. What you need An SO-101 follower arm, a RealSense D455 for the scene view (RGB only — depth is not fed to the model), and any 640x480 USB webcam on the wrist. The RealSense needs a USB 3 data cable and a USB 3 port; on USB 2.1 it drops to 15 fps, which still works but warms up slowly. The calibration trap This is the part that makes the repository worth reading even if you never run it. MolmoAct2 was trained on LeRobot datasets using the v2.1 joint-angle convention, while LeRobot 0.5.x records calibrations in v3.0. Get the offsets wrong and, in the repository's own words, "the arm will slam hard into the table on startup" — named as the most common failure mode. The default joint offsets and signs in the inference script apply the official conversion, but only if you calibrated with exactly LeRobot 0.5.1, which is why that pin is described as non-negotiable. The repository also tells you to replace the shipped calibration file with your own, because the included one is specific to the author's arm. Honest about what does not work The known-limitations section is unusually frank. The wrist webcam is out of distribution — MolmoAct2 was trained on two third-person RealSense views, not a wrist view, so there is a --scene-only flag that passes the side view twice and skips the wrist image when behaviour degrades. Depth reasoning is off, so the model sees RGB only. Single arm, no leader or teleop fallback. GPU required for bfloat16 inference; CPU is untested. What to expect About 15 GB of weights on first run, up to 30 seconds of camera warmup, and roughly 700 ms for the first inference chunk on a laptop GPU. There is a --dry-run mode that prints joint targets without moving the arm, which is the sane way to check the model is loading before letting it near the table.
• GPT-6 Astra Paints the Golden Gate Bridge on a Borrowed SO-101 (thijs): An SO-101 borrowed from a colleague, a paintbrush taped into the gripper, one camera, and a general-purpose agent told to paint the Golden Gate Bridge. No policy, no demonstrations, no robot-specific code released — the model worked out how to drive the arm and got visibly better across attempts. How it was driven In the builder's own description of the setup: the agent is "instructed to plan out 1 minute of actions and monitor it in the background to make adjustments as it goes." Between runs he gave it feedback on how it was doing, sometimes asking it "what can you do better?" and having it rewrite its own instructions. Calibration is the part worth noting. He seeded it with a few anchor points, but on later attempts the agent worked out the calibration itself and corrected for its own positioning error — self-calibration that was not programmed in, emerging from the feedback loop. The result The posted timelapse runs about a minute and shows the progression across attempts: early passes are barely recognisable, later ones read clearly as the bridge. The builder's own framing of what it is — offered before anyone else raised it — was "algorithmic art, similar to how pen plotter is," and that he does not think it replaces artists. What is not here There is no repository, no blog post and no released prompt. Asked directly in the replies whether a write-up existed, the builder did not publish one. Everything above comes from the thread itself. The arm was borrowed from Toby Simonds, who is credited in the thread. Why it circulated The clip was picked up well beyond robotics — Sam Altman answered it with "i want one!", and it led to a TBPN segment in which the builder argued that cheap open-source arms plugged into agents are about to have their own version of the 3D-printing craze.
• robot2: A Plain HTTP Control Stack for Driving an SO-101 from an Agent (Alex Nichol): Alex Nichol rewrote his home robotics stack from scratch in Go and put the whole thing behind a small HTTP API, then pointed a general-purpose agent at it. The repository describes itself plainly as "an attempt to rewrite a lot of my home robotics stack in a cleaner (human-driven) way" — human-driven being the point, in a build whose whole purpose is letting a model drive the arm. The API Three groups of endpoints, all returning JSON wrapped as {"data": ...} or {"error": ...}, with non-200 status codes distinguishing robot faults from bad requests: - Motors — /motor/names, /motor/status, /motor/limits, /motor/move (raw positions in 0–4095), and /motor/stoprelax, a blocking call that suppresses the gradual relaxation which otherwise settles the motors toward the direction of torque. That last one exists so the arm does not sag while the model is thinking. - Kinematics — /kinematics/limitsangles, /kinematics/currentangles, and /kinematics/move, which takes every joint angle in radians and requires all of them to be specified. - Cameras — /camera/tracknames and /camera/snapshot, which returns raw JPEG bytes for a named track. That is the entire surface an agent needs: read the joint angles, look through a camera, command new angles. What is in the repository Go throughout, with packages for the API, camera capture over V4L2, and kinematics, plus a set of command-line tools that reveal what the build actually required: calibrate and calibrate_single, fit_to_table and manual_table_fit for working out where the table is, ik_move for inverse-kinematic positioning, kinematics_log, placement_data for recording placements, setid for assigning servo IDs, and timelapse. Why the endpoint design matters Most agent-to-arm demonstrations wrap an existing framework. This one starts from the observation that an agent is a thing that makes HTTP requests, and builds the smallest surface that lets it do useful work — a design a reader can copy onto a different arm in an afternoon.
• Full-Stack Manipulation Framework: From 3D Printing to Diffusion Policies (Thanh Nguyen (LAAS-CNRS)): A comprehensive full-stack manipulation framework from a PhD roboticist at LAAS-CNRS, covering everything from 3D printing an SO-ARM100 to running diffusion policies and vision-language-action (VLA) models — the most complete software guide for the SO-100/101 platform. What Was Built Thanh Nguyen, a PhD researcher in the Gepetto robotics team at LAAS-CNRS (Toulouse, France), created a modular full-stack framework that takes the SO-ARM100 from a simple servo-driven arm to a research-grade manipulation platform: - Hardware stack — URDF model, system identification, 3D printing guidelines - State estimation — Multi-sensor fusion with camera, motion capture, and IMU - Simulation — Full integration with MuJoCo, Gazebo, and NVIDIA Isaac Sim - Model-based control — MoveIt 2 motion planning, visual servoing, MPC - Learning-based stack — LeRobot diffusion policies, RL training, VLA (vision-language-action) research - Evaluation — Benchmarking framework for comparing approaches Technical Architecture The project follows a clean modular structure: ``` fullstack-manip/ ├── hardware/ # Robot models, CAD, system identification ├── state_estimation/ # Multi-sensor fusion, calibration ├── simulation/ # MuJoCo, Gazebo, Isaac Sim environments ├── control/ # Low and high-level controllers ├── planning/ # MoveIt 2 motion planning ├── perception/ # Visual servoing, vision modules ├── learning/ # RL, VLA, datasets, LeRobot integration ├── evaluation/ # Benchmarking and metrics └── docs/ # Comprehensive documentation ``` Each module is designed to work independently or as part of the full pipeline, making it easy to use just the simulation stack, just the learning pipeline, or the complete system. Why It Matters While most SO-100/101 builds focus on a single aspect (typically LeRobot imitation learning), this project uniquely covers the entire robotics software stack — from classical model-based control through modern learning-based approaches. This makes it invaluable as: 1. An educational resource — PhD-level documentation covering both classical and modern approaches 2. A research platform — Ready-made infrastructure for comparing model-based vs. learning-based manipulation 3. A reference implementation — Shows how to properly integrate ROS2, MoveIt, MuJoCo, Isaac Sim, and LeRobot with the SO-ARM100 The 67-commit repository with Apache 2.0 license and comprehensive documentation (12+ detailed technical docs) represents one of the most thorough software treatments of any open source robot arm build. About the Builder Thanh Nguyen works in the Gepetto team at LAAS-CNRS, one of France's leading robotics labs known for contributions to humanoid locomotion and optimal control. This academic rigor is reflected in the project's systematic approach to documentation and software architecture.
• Hackathon-Winning Bimanual Shirt-Folding Robot (Pranav Saroha): A 16-year-old high school student built a bimanual SO-100 setup that folds T-shirts at 85% success rate — winning 1st place in the US and 2nd worldwide at the LeRobot Worldwide Hackathon. The Build Pranav Saroha, a sophomore at Valley Christian High School in San Jose, teamed up with Shaan Patel (19, Santa Clara University) as team Ez_2_AI to build a dual-arm SO-100 robot capable of folding laundry and sorting medicine. Their setup featured: - Bimanual configuration — two SO-100 arms working together for complex manipulation tasks - 4-camera recording system for capturing training demonstrations - Custom 3D-printed gripper designed specifically for medicine pill sorting - Multiple AI models trained: pi0, SmolVLA, and ACT using imitation learning The Hackathon At the LeRobot Worldwide Hackathon (June 13-15, 2025), a 48-hour event organized by HuggingFace with 250+ teams from 44 countries and 3,000+ participants: - Achieved 70% T-shirt folding success rate during the event - Later improved to 85% success rate with a different model - Also demonstrated medicine pill sorting into daily compartments — motivated by wanting to help his grandfather manage small pills - Won 1st place in the United States and 2nd worldwide Beyond the Hackathon Pranav has become one of the most active SO-100 data contributors on HuggingFace, uploading 42 models and 117 datasets — showing sustained, serious robotics research work well beyond the hackathon. His build guide on Medium covers sourcing, assembly pitfalls, calibration, and training on both Mac and NVIDIA GPUs. Why It Matters This build proves that the SO-100/SO-101 platform is accessible enough for a high school student to build, yet capable enough to compete with — and beat — hundreds of experienced teams worldwide. The shirt-folding task is particularly notable as it's a benchmark that even well-funded companies like Physical Intelligence are actively working on.
• leChef: Sandwich-Making Robot with Lateral Rail (Team leChef (Chiodi)): 1st Place Winner of the LeRobot Worldwide Hackathon (June 2025), a global event organized by Hugging Face with 3,000+ participants across 100+ locations. leChef is a multi-task robotic manipulation system built on the SO-100 arm, enhanced with a custom lateral mobility rail. This additional degree of freedom allows the arm to move horizontally along a track, expanding its workspace and enabling assembly-line style food preparation. How It Works The robot constructs a sandwich by sequentially picking up ingredients and stacking them on a plate: 1. Bread — base layer 2. Cheese — first topping 3. Ham — protein layer 4. Salad — greens 5. Bread — top layer Each task creates a progressively changing environment as ingredients stack up, making this ideal for studying manipulation under evolving scene configurations. Technical Approach All five tasks were designed to be learned by a single ACT (Action Chunking with Transformers) model with minimal modifications. The team collected 200 teleoperation episodes (40 per task) to train the policy. The lateral rail adds a 7th degree of freedom to the standard SO-100 arm, enabling the robot to slide between ingredient stations — similar to a production line.