Teacher-Student Training

• Also called: privileged learning, learning by cheating, teacher-student distillation, privileged teacher
• Category: Learning & autonomy › RL & simulation
• Tags: reinforcement-learning, sim-to-real, policy-distillation, robot-learning, locomotion

A training scheme in which a teacher policy with privileged information, such as true simulator state, is distilled into a student policy that uses only sensors available on the real robot.

Definition

Teacher-student training is a two-stage approach to robot learning, used mostly in simulation, that separates learning what to do from learning how to infer it from realistic sensors. In the first stage, a teacher policy is trained, usually with reinforcement learning, while given privileged information that a real robot would not have: exact terrain heights, friction coefficients, object poses, contact states or the full simulator state. With this information, the task is much easier to learn.

In the second stage, a student policy is trained to reproduce the teacher's behavior using only observations available on hardware, such as joint encoders, an IMU, camera images or a short history of past observations. Because the teacher can be queried at any state, the student is trained with supervised learning, often with DAgger-style data collection in which the student drives the simulation and the teacher supplies the correct action at each visited state. The student often uses a recurrent network or a history encoder so it can infer hidden quantities, such as ground friction, from how the robot has been moving.

The approach is closely related to asymmetric actor-critic methods, where only the critic sees privileged state, and to policy distillation in general. In autonomous driving it was presented as "learning by cheating" by Chen et al. in 2019, and it became widely used in legged locomotion after Lee et al. in 2020 trained quadruped controllers this way and deployed them on rough natural terrain.

The main limitation is that the student can only be as good as the information its sensors carry: if the teacher relies on something the student cannot infer, the student will fall short.

Why it matters

Learning directly from noisy, partial sensor data with reinforcement learning is slow and unstable. Teacher-student training lets the hard control problem be solved first with full information and then transfers that skill to a policy that can run on a real robot, which has made it a standard part of sim-to-real pipelines for legged robots and dexterous hands.

Key applications

• Legged locomotion over rough terrain using mainly proprioception
• Dexterous in-hand manipulation with policies first trained on full object state
• Vision-based navigation and driving policies distilled from state-based experts
• Sim-to-real transfer where key physical parameters cannot be measured on hardware

In practice

A team training a quadruped controller in a GPU simulator such as Isaac Lab first trains a teacher with PPO, feeding it proprioception plus privileged terrain samples and contact information, with domain randomization over friction and payload. Once the teacher is reliable, they train a student that receives only proprioceptive history, sometimes with noisy height-map or depth input, to match the teacher's actions or latent features. Training data comes from rolling out the student in many parallel environments and labeling each visited state with the teacher's output. The student is then deployed on the real robot, and engineers test it in the situations where hidden information matters most, such as slippery floors or obstacles the robot cannot see.

Common misconceptions

• The student needs real-world training data — both stages are normally trained entirely in simulation, with the student deployed zero-shot
• The student simply copies the teacher — it must infer the hidden information from its own sensor history, which is the hard part
• Teacher-student training replaces reinforcement learning — the teacher is usually trained with RL; only the student stage is supervised distillation

Who should know this

• Robot learning researchers working on sim-to-real transfer
• Legged locomotion control engineers
• Dexterous manipulation researchers
• Reinforcement learning practitioners deploying policies on hardware

Related technologies

• Isaac Lab
• DAgger
• Asymmetric Actor-Critic
• Policy Distillation
• Proximal Policy Optimization (PPO)
• MuJoCo

Related terms

• Reinforcement Learning
• Sim2Real
• Domain Randomization
• Curriculum Learning

Related projects

• LeRobot: HuggingFace ML library for real-world robotics in PyTorch
• MuJoCo: DeepMind physics engine for robotics and ML research
• PyBullet: Python physics simulator for robotics and RL research with fast CPU-based simulation
• OpenPI: Physical Intelligence VLA foundation models for general robot control
• Microduck: Pollen Robotics' $399 walking duck robot: open-source software, closed hardware
• Isaac Lab: NVIDIA robot learning framework: one task API across Newton, PhysX and Isaac Sim