Action Chunking

• Also called: action chunks, chunked action prediction, action sequence prediction
• Category: Learning & autonomy › Learning from demonstration
• Tags: imitation-learning, robot-learning, visuomotor-policy, manipulation, behavior-cloning, action-representation

A policy design in which a robot predicts a short sequence of future actions at once and executes several of them before querying the policy again, instead of predicting one action per step.

Definition

Action chunking is a technique in robot learning where a policy outputs a block, or chunk, of consecutive future actions from a single observation rather than a single action for the current time step. The robot then executes some or all of that chunk before the policy is queried again. The idea was brought to wide attention in robot imitation learning by the Action Chunking with Transformers (ACT) policy, introduced by Zhao, Kumar, Levine and Finn in 2023 alongside the ALOHA bimanual teleoperation system.

The main motivation is compounding error. A policy trained by behavior cloning that predicts one action at a time can drift slightly away from the demonstrated states, and each small mistake moves it further into situations the training data never covered. Predicting a chunk shortens the effective horizon of the task: many control steps become a smaller number of chunk-level decisions, so there are fewer opportunities for errors to accumulate.

Chunking also helps with human demonstrations, which are often non-Markovian. Operators pause, hesitate or follow habits that depend on what they did a moment earlier, not only on what the camera currently shows. A model that outputs a coherent multi-step segment can reproduce these temporally correlated patterns more faithfully than one forced to decide each step in isolation.

There is a trade-off between smoothness and reactivity: a robot executing a full chunk cannot react to new observations until it ends, a problem when objects move or contact is uncertain. Common remedies include executing only the first part of each chunk before predicting again, as diffusion policies typically do in a receding-horizon loop, and temporal ensembling, where overlapping predictions for the same time step from successive chunks are averaged with weights. Chunk length is a tuning parameter that depends on control rate and task dynamics.

Action chunking now appears across many policy classes, including transformer policies, diffusion policies and vision-language-action models, and it interacts with how actions are represented, for example when continuous action sequences are converted into discrete tokens.

Why it matters

Action chunking is one of the design choices that made imitation learning from a modest number of human demonstrations work reliably on fine-grained manipulation tasks. It reduces compounding errors and produces smoother motion, and it is now a default component of many visuomotor policies and robot foundation models.

Key applications

• Bimanual manipulation — learning fine-grained two-arm tasks from teleoperated demonstrations
• Visuomotor policies — predicting smooth action sequences from camera images and joint states
• Diffusion policies — generating multi-step action sequences executed in a receding-horizon loop
• Vision-language-action models — outputting action segments from language instructions and images
• Action tokenization — compressing action sequences into discrete tokens for transformer-based policies

In practice

A team training a manipulation policy from teleoperated demonstrations typically records joint positions or end-effector poses at a fixed control rate, then trains the policy to predict the next several actions from the current camera images and joint states. At deployment, the controller runs the policy, executes part of the predicted chunk, and calls the policy again, often blending overlapping predictions to avoid jerks at chunk boundaries. Engineers tune the chunk length and the number of actions executed per query by watching for two failure modes: jittery or hesitant motion when chunks are too short, and slow reactions to disturbances when chunks are too long.

Common misconceptions

• Action chunking means running the robot open-loop — most implementations query the policy again before a chunk finishes, keeping the system closed-loop at the chunk level
• Longer chunks are always better — long chunks reduce compounding error but make the robot slower to react to disturbances and moving objects
• Action chunking is specific to ACT — the same idea is used in diffusion policies, token-based autoregressive policies and many vision-language-action models

Who should know this

• Robot learning researchers training policies from demonstrations
• Engineers deploying visuomotor policies on manipulators
• Teams building teleoperation data collection pipelines
• Students learning imitation learning methods

Related technologies

• Action Chunking with Transformers (ACT)
• Diffusion Policy
• Behavior Cloning
• Vision-Language-Action Models
• Temporal Ensembling
• ALOHA

Related terms

• Imitation Learning
• Behavior Cloning
• Diffusion Policy
• Teleoperation

Related projects

• LeRobot: HuggingFace ML library for real-world robotics in PyTorch
• OpenPI: Physical Intelligence VLA foundation models for general robot control
• Isaac Lab: NVIDIA robot learning framework: one task API across Newton, PhysX and Isaac Sim
• NVIDIA Isaac GR00T: NVIDIA open foundation model for humanoid robot reasoning and skills
• OpenVLA: 7B-parameter Vision-Language-Action model for robotic manipulation
• Mobile ALOHA: Bimanual mobile manipulation with 4 robot arms on a wheeled base for teleoperation

Videos

• RI Seminar: Max Simchowitz: Generative Control, Action Chunking, and Moravec’s Paradox: CMU Robotics Institute
• Machine Learning Series - ACT Action Chunking Transformers: Trossen Robotics