Curriculum Learning
- Also called: curriculum training, task curriculum
- Category: Learning & autonomy › RL & simulation
- Tags: reinforcement-learning, robot-learning, training, simulation, locomotion
A training strategy that gives a learning system easier tasks or examples first and raises the difficulty gradually, so it can master skills that are too hard to learn from scratch.
Definition
Curriculum learning is the practice of ordering training experience from easy to hard instead of presenting it all at once. The idea mirrors how people are taught: a student learns arithmetic before calculus. In machine learning it was formalized by Bengio et al. in 2009, who showed that a well-chosen ordering of examples can speed up training and improve the final result.
In robotics, curricula are most common in reinforcement learning, where an agent that starts on a task far beyond its ability may never receive a useful reward signal. A curriculum keeps the agent at the edge of its competence. Difficulty can be raised along many axes: terrain roughness and slope for a legged robot, step height for stair climbing, distance to the goal for navigation, object variety for grasping, the strength of external pushes, or how sparse the reward is.
Curricula can be hand-designed, with fixed stages and thresholds for moving on, or automatic. Automatic curricula adjust difficulty from the agent's performance, for example promoting a simulated robot to harder terrain when it succeeds and demoting it when it fails. Related ideas include automatic goal generation, where the training system proposes goals of suitable difficulty, and self-play, where opponents improve together.
A curriculum adds design choices that can go wrong. If difficulty rises too quickly, learning stalls; if too slowly, compute is wasted. A curriculum that never reaches the full range of conditions seen in deployment can leave the policy unprepared for them.
Why it matters
Many robot skills, such as climbing, jumping or walking on rough ground, cannot be learned by trial and error from scratch because random behavior almost never succeeds. Curriculum learning makes these skills learnable by building them up in steps, and it is a standard ingredient of simulation-trained locomotion controllers.
Key applications
- Legged locomotion training over progressively harder terrain
- Learning climbing and stair-traversal skills
- Navigation tasks with gradually increasing goal distance or clutter
- Manipulation tasks with increasing object variety or precision requirements
In practice
In GPU-based locomotion training, a common setup is a terrain curriculum: thousands of simulated robots are spread over a grid of terrain tiles ordered by difficulty, such as slopes, stairs and rough patches of increasing height. Each robot starts on easy tiles; if it travels far enough in an episode it moves to harder tiles, and if it fails it moves back. Teams often run parallel curricula on commanded speed and disturbance strength, and they log how the robots are distributed across difficulty levels as a direct measure of training progress. Before deployment, they evaluate the final policy at the hardest levels and on conditions outside the curriculum, to check that it has not been tuned only to the training ladder.
Common misconceptions
- A curriculum always helps — a poorly designed curriculum can slow training or bias the policy toward the conditions it emphasized
- Curricula must be designed by hand — many systems adjust difficulty automatically from the agent's measured performance
- Curriculum learning is only for reinforcement learning — it was first formalized for supervised learning and applies there as well
Who should know this
- Reinforcement learning practitioners
- Legged robot control engineers
- Simulation and training-environment designers
- Machine learning students working on robot learning
Related technologies
- Reinforcement Learning
- Isaac Lab
- Domain Randomization
- Reward Shaping
- Automatic Curriculum Learning
- Self-Play
Related terms
Related projects
- GENESIS: World's fastest physics engine: 43M FPS for embodied AI learning
- LeRobot: HuggingFace ML library for real-world robotics in PyTorch
- MuJoCo: DeepMind physics engine for robotics and ML research
- PyBullet: Python physics simulator for robotics and RL research with fast CPU-based simulation
- OpenPI: Physical Intelligence VLA foundation models for general robot control
- Microduck: Pollen Robotics' $399 walking duck robot: open-source software, closed hardware
Videos
- Curriculum Learning in Deep Neural Networks: Connor Shorten
- Teacher-Student Curriculum Learning: Connor Shorten