Visuomotor Policy

• Also called: end-to-end visuomotor control, vision-based policy
• Category: Learning & autonomy › Learning from demonstration
• Tags: imitation-learning, robot-learning, manipulation, computer-vision, end-to-end-learning

A control policy that maps camera images, often together with joint state, directly to robot actions, learning perception and control as a single model.

Definition

A visuomotor policy takes visual observations, usually one or more camera images, often combined with proprioceptive state such as joint positions and gripper width, and outputs motor commands: joint torques, joint velocities, end-effector poses or gripper commands. Perception and control are learned together, so the visual features the policy extracts are shaped by what helps it act, rather than being fixed outputs such as object poses from a separate perception module. Sergey Levine, Chelsea Finn and colleagues brought the term into wide use with their 2016 work on end-to-end training of deep visuomotor policies.

Most visuomotor policies are trained by imitation learning from teleoperated demonstrations, by reinforcement learning, or by a combination of the two. Typical architectures pass images through a convolutional or transformer encoder, merge the result with proprioception, and decode actions with a multilayer perceptron, a transformer, or a generative model such as a diffusion policy that can represent several valid actions. Data efficiency is the central problem: images are high-dimensional, demonstrations are expensive, and policies can latch onto irrelevant details such as lighting or background.

A large body of work addresses this by pretraining the visual encoder on other data, such as large image datasets or videos of people, and then training only a small policy head on robot demonstrations. How well few-demonstration visuomotor policies learn on top of a frozen encoder has become a common way to evaluate such representations. Other methods add data augmentation, multiple camera views including wrist cameras, and action chunking to reduce compounding errors. Vision-language-action models extend the idea by also conditioning the policy on language instructions.

Why it matters

Visuomotor policies let robots handle tasks where the relevant state is hard to engineer, such as deformable objects, clutter or precise insertion, by learning directly from pixels. They are at the core of most current imitation learning systems for manipulation.

Key applications

• Imitation learning for tabletop and kitchen manipulation
• Grasping and pick-and-place from camera input
• Deformable object handling such as cloth folding
• Evaluating pretrained visual representations for robotics
• Drone and vehicle control from onboard cameras

In practice

A team teaching a robot arm to hang a mug on a rack collects teleoperated demonstrations with a fixed scene camera and a wrist camera, recording images, joint positions and commanded end-effector motions. They train a policy that encodes each image with a pretrained vision backbone, concatenates the features with the joint state and predicts a short chunk of future actions. Random crops and color jitter during training help the policy ignore small camera shifts and lighting changes. On the robot, the policy runs in a closed loop at the camera frame rate, and the team tests it with mugs in new positions and with distractor objects to check that it attends to the right parts of the image.

Common misconceptions

• Visuomotor policies use images alone — most combine images with proprioception, which is often critical for precise control
• End-to-end means no design choices — architecture, action representation, camera placement and pretrained encoders all strongly affect results
• A policy that works in the training scenes will work in new ones — visuomotor policies are sensitive to lighting, backgrounds and camera pose unless trained for that variation

Who should know this

• Robot learning researchers
• Manipulation engineers building imitation learning systems
• Computer vision researchers working on representations for robotics
• Teams collecting robot demonstration data

Related technologies

• Diffusion Policy
• Action Chunking with Transformers (ACT)
• Pretrained Visual Representations
• Vision-Language-Action Models
• Convolutional Neural Networks

Related terms

• Behavior Cloning
• Imitation Learning
• Diffusion Policy
• Action Chunking
• Proprioception
• VLA

Related projects

• LeRobot: HuggingFace ML library for real-world robotics in PyTorch
• OpenPI: Physical Intelligence VLA foundation models for general robot control
• Isaac Lab: NVIDIA robot learning framework: one task API across Newton, PhysX and Isaac Sim
• NVIDIA Isaac GR00T: NVIDIA open foundation model for humanoid robot reasoning and skills
• OpenVLA: 7B-parameter Vision-Language-Action model for robotic manipulation
• Mobile ALOHA: Bimanual mobile manipulation with 4 robot arms on a wheeled base for teleoperation

Videos

• End-to-End Training of Deep Visuomotor Policies: RAIL
• Fall 2022 6.4210/2 Lecture 17: Visuomotor policies (behavior cloning): underactuated