PHC

Physics-based perpetual humanoid controller with fail-state recovery for real-time avatars

License

BSD-3-Clause

Overview

PHC (Perpetual Humanoid Control) is a physics-based humanoid controller that achieves high-fidelity motion imitation and fault-tolerant behavior for real-time simulated avatars and humanoid robots. Presented at ICCV 2023, it can perpetually control simulated humanoids without requiring resets, even in the presence of noisy input and unexpected falls.

The key innovation is the Progressive Multiplicative Control Policy (PMCP), which dynamically allocates new network capacity to learn increasingly difficult motion sequences. This allows efficient scaling to the full AMASS motion database (11,313 sequences) with up to 100% success rate, while adding new capabilities like fail-state recovery without catastrophic forgetting.

PHC supports multiple control interfaces including video-based avatar control via webcam, natural language commands through Motion Diffusion Model integration, and VR controller tracking. It works with SMPL, SMPLX (with finger control), Unitree H1, and Unitree G1 humanoid morphologies.

The system runs on NVIDIA IsaacGym and includes IsaacLab integration. PHC has become a foundational component used by projects like PULSE and ASAP for humanoid control research.

Developed at Carnegie Mellon University in collaboration with NVIDIA.

PHC (Perpetual Humanoid Control) is a physics-based humanoid controller developed at Carnegie Mellon University in collaboration with NVIDIA, presented at ICCV 2023. It solves a fundamental problem in humanoid simulation: conventional controllers fail permanently when they encounter unexpected perturbations or difficult motion sequences, requiring manual resets. PHC instead achieves perpetual control — it can recover from falls and continue operating indefinitely without resets.

The key innovation is the Progressive Multiplicative Control Policy (PMCP), which progressively allocates new network capacity to handle increasingly difficult motions. This allows efficient training on the full AMASS motion database (11,313 sequences) with up to 100% success rate, while adding recovery capabilities without catastrophic forgetting of previously learned skills. When the controller falls, it can get back up and resume tracking the target motion.

PHC supports multiple control interfaces including video-based avatar control via webcam, natural language commands through Motion Diffusion Model integration, and VR controller tracking for immersive applications. It works with SMPL, SMPLX (with finger-level control), and humanoid robot morphologies including Unitree H1 and G1, with IsaacLab integration for modern simulation workflows.

PHC has become a foundational building block in the humanoid control research ecosystem, with projects like PULSE and ASAP building upon its perpetual control capabilities. Its ability to maintain stable control through arbitrary perturbations makes it practical for real-time applications where manual resets are impossible.

Why it matters

PHC solves a fundamental problem in humanoid simulation — controllers that fail permanently after unexpected perturbations or noisy input. Its perpetual control capability with automatic fail-state recovery makes it practical for real-time applications like VR avatars and video-driven humanoid control.

Problem it solves

Physics-based humanoid controllers that crash and require manual resets after falls or unexpected perturbations, making them impractical for real-time applications

Key takeaways

• 100% success rate on 11,313 AMASS motion sequences with PHC+ model
• Progressive Multiplicative Control Policy enables scaling without catastrophic forgetting
• Supports video, language, and VR interfaces for humanoid control
• Works with SMPL, SMPLX, Unitree H1, and Unitree G1 morphologies

Who it is for

• Humanoid control researchers
• VR and avatar developers
• Physics simulation researchers
• Reinforcement learning practitioners

Limitations

• Requires NVIDIA GPU with IsaacGym support
• Training requires AMASS motion capture dataset (license required)
• SMPL/SMPLX body model license required for commercial use
• Complex multi-step setup with many dependencies