Driving · 2026-11-02
$\nabla$Mimic: Ultra-Efficient Humanoid Control with Differentiable Simulation
Weyl Lu, Yubei Chen
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
∇Mimic trains humanoid motion tracking using exact simulator gradients on contact-rich control, addressing exploding/expensive gradients. It bounds rare unbounded per-sample gradients via median-relative gradient clipping and reuses simulated windows via multistep regression onto gradient-derived action targets. A GPU differentiable simulator, NablaEngine, provides fast exact adjoints for MuJoCo contact dynamics on Unitree G1. On all 40 LAFAN1 clips, it matches SuperMimic’s score 0.82 in 20 minutes (1.2 h for SuperMimic).
Why it matters
The authors argue that exact simulator gradients can outperform surrogate world-model gradients if two obstacles are removed: (1) rare unbounded derivatives from contact events that break first-order methods, and (2) high cost of computing simulator adjoints. Their approach keeps first-order learning stable on contact dynamics at scale and demonstrates world-model-level tracking performance without a world model, plus open-source infrastructure for first-order humanoid policy learning.
Method
- Bounds rare extreme gradients using median-relative per-sample gradient clipping at every step of the backward pass.
- Reuses each costly simulated window by multistep regression onto gradient-derived action targets, bounding each action target independent of gradient scale.
- Uses NablaEngine: a differentiable GPU simulation engine for Unitree G1 with fused kernels and an adjoint of a whole control step as a single megakernel, using analytic gradients matching float64 finite differences.
Limitation
Exact gradients contain rare unbounded samples because contact dynamics is not differentiable everywhere and its derivative diverges near some events; even with the proposed fixes, the paper describes that without NablaEngine’s rules such samples can be non-finite or extremely large.
Abstract (from arXiv)
Robot policy learning is dominated by zeroth-order methods such as PPO, which treat the simulator as a black box and pay for it with enormous sample counts. First-order policy learning differentiates through the dynamics, but on contact-rich whole-body control it has rarely worked: its gradients explode. A recent world-model method showed that derivative information can make humanoid motion tracking far more efficient, using surrogate gradients of a learned model. We show that exact gradients of the simulator itself do better, once two obstacles are removed. Exact gradients contain rare unbounded samples, because contact dynamics is not differentiable everywhere and its derivative diverges near some events; and they are expensive, because the simulator's adjoint has to replay the contact solve. $\nabla$Mimic bounds these samples with median-relative per-sample gradient clipping at every step of the backward pass, and reuses each costly simulated window by multistep regression onto gradient-derived action targets, which bounds every action target independently of the gradient scale. NablaEngine, a differentiable GPU simulation engine for the Unitree G1, removes the cost: it computes MuJoCo's full contact dynamics with fused per-world kernels and the adjoint of a whole control step as a single megakernel, with analytic gradients that match float64 finite differences, $11.5\times$ faster than the available MuJoCo Warp adjoint. On whole-body tracking of all 40 LAFAN1 clips with one policy on one GPU, $\nabla$Mimic reaches the score reported by the state-of-the-art world-model method, 0.82, in 20 minutes instead of 1.2 h, and 0.874 in 1.8 h. SHAC, the standard first-order baseline, stops on exploding gradients within minutes. We release NablaEngine and $\nabla$Mimic as open-source infrastructure for first-order humanoid policy learning.
Related papers
- ODE Policy: Robot Learning via Action Dynamics, Not Actions
- Contact-UAN: Unsupervised Actuator Networks as a Residual World Model for Humanoid Control
- DiffWBC: Unlocking Steerable General Humanoid Motion Tracking via One-Step Diffusion Whole-Body Control
- VioLA: Learning Generalist Humanoid Control Policies from Human Data
- SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-11-02 · Indexed by Pub-AI: 2026-11-02 UTC
$\nabla$Mimic: Ultra-Efficient Humanoid Control with Differentiable Simulation
First announced by arXiv: 2026-11-02 · Original record ↗
- arxiv
- 2610.41654
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-11-02 · First detected: 2026-11-02 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2610.41654v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
Robot policy learning is dominated by zeroth-order methods such as PPO, which treat the simulator as a black box and pay for it with enormous sample counts. First-order policy learning differentiates through the dynamics, but on contact-rich whole-body control it has rarely worked: its gradients explode. A recent world-model method showed that derivative information can make humanoid motion tracking far more efficient, using surrogate gradients of a learned model. We show that exact gradients of the simulator itself do better, once two obstacles are removed. Exact gradients contain rare unbounded samples, because contact dynamics is not differentiable everywhere and its derivative diverges near some events; and they are expensive, because the simulator's adjoint has to replay the contact solve. $\nabla$Mimic bounds these samples with median-relative per-sample gradient clipping at every step of the backward pass, and reuses each costly simulated window by multistep regression onto gradient-derived action targets, which bounds every action target independently of the gradient scale. NablaEngine, a differentiable GPU simulation engine for the Unitree G1, removes the cost: it computes MuJoCo's full contact dynamics with fused per-world kernels and the adjoint of a whole control step as a single megakernel, with analytic gradients that match float64 finite differences, $11.5\times$ faster than the available MuJoCo Warp adjoint. On whole-body tracking of all 40 LAFAN1 clips with one policy on one GPU, $\nabla$Mimic reaches the score reported by the state-of-the-art world-model method, 0.82, in 20 minutes instead of 1.2 h, and 0.874 in 1.8 h. SHAC, the standard first-order baseline, stops on exploding gradients within minutes. We release NablaEngine and $\nabla$Mimic as open-source infrastructure for first-order humanoid policy learning.
