DreamZero

NVIDIA World Action Model that jointly predicts future video and robot actions

License

Apache-2.0

Overview

NVIDIA 14B parameter World Action Model that jointly predicts future video frames and robot actions using a pretrained video diffusion backbone. Achieves zero-shot generalization to unseen tasks and few-shot adaptation to new robot embodiments with as little as 30 minutes of data. Built on an autoregressive video diffusion architecture, DreamZero delivers over 2x improvement in generalization compared to state-of-the-art vision-language-action models.

DreamZero is a World Action Model (WAM) from NVIDIA Research that uses a pretrained video diffusion backbone to jointly predict future world states and robot actions. Unlike conventional vision-language-action (VLA) models that map observations directly to actions, DreamZero leverages video as a dense representation of how environments evolve, enabling strong zero-shot generalization to unseen tasks and environments.

Key results include over 2x improvement on unseen tasks compared to state-of-the-art VLAs on both the AgiBot dataset (62.2% vs 27.4% task progress) and DROID dataset (49% vs 25-32% on unseen verbs). The model achieves real-time closed-loop control at 7Hz through DiT caching optimizations, with inference times of ≈0.6s on GB200 and ≈3s on H100 GPUs.

DreamZero also demonstrates cross-embodiment transfer, adapting to the YAM robot with only 30 minutes of teleoperation data, achieving a 42% improvement over baselines. The 14B parameter model supports distributed multi-GPU inference via WebSocket and is released under the Apache 2.0 license.

Why it matters

DreamZero from NVIDIA demonstrates that a pretrained video diffusion model can be repurposed as a world model for robotics. It jointly predicts future video frames AND robot actions, achieving zero-shot generalization to tasks it was never trained on.

Problem it solves

Training robot policies requires expensive real-world demonstrations for each task. DreamZero leverages pretrained video models that already understand physical world dynamics, enabling robots to perform new tasks zero-shot without task-specific training data.

Key takeaways

• 14B parameter World Action Model built on pretrained video diffusion backbone
• Jointly predicts future video frames and robot actions in a unified model
• Achieves zero-shot generalization to unseen manipulation tasks
• Leverages internet-scale video pre-training for physical world understanding
• Bridges the gap between video generation models and robot control

Who it is for

• World model researchers
• Video-to-action model developers
• NVIDIA ecosystem researchers
• Teams exploring zero-shot robot policy transfer

Limitations

• 14B parameters requires substantial GPU resources
• Zero-shot performance may not match task-specific fine-tuning
• Depends heavily on video pre-training distribution
• Relatively new approach with limited real-world validation