Latent Dynamics · 2026-11-02

DiffWBC: Unlocking Steerable General Humanoid Motion Tracking via One-Step Diffusion Whole-Body Control

Yifei Huang, Yazhao Yang, Weijia Li, Zhiheng Ma, Dian Shen

arXiv:2610.41354PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

DiffWBC is a 50 Hz one-step diffusion whole-body controller for humanoid motion tracking. It generates a distribution over future trajectory plans from a reference horizon in a single forward pass, then uses an inverse-dynamics head for actions. The authors report lowest local tracking error among six general trackers on held-out benchmarks and show test-time steering by optimizing generator input noise with differentiable costs, plus simulation-free failure discovery via plan dispersion.

Why it matters

The authors aim to overcome deterministic trackers’ rigidity at deployment, enabling online steering, body-group yielding, and identifying/repairing physical tracking failures without costly physics simulation or retraining. DiffWBC unifies general tracking, test-time steering, and simulation-free failure discovery in one frozen controller, with reported results on held-out benchmarks and a physical Unitree G1.

Method

  • Formulates a 50 Hz closed-loop whole-body controller as a one-step diffusion model: a single forward pass generates a plan over future states/actions from proprioception plus a sliding multi-body-group reference window.
  • Uses an inverse-dynamics head to map predicted states to joint-level actuation commands; test-time steering freezes weights and performs gradient descent on the generator’s latent input under differentiable costs and body-group masks.
  • Introduces simulation-free failure discovery by using dispersion across generation noise as an uncertainty/fragility proxy, then fine-tunes on mined hard clips.

Limitation

The authors state that in a robotics design test, executed actions cannot be executed open loop (the optimized actions cannot be executed open loop; success 0.10), so the closed-loop action head remains necessary.

Abstract (from arXiv)

General humanoid motion trackers typically rely on deterministic policies that cannot adapt online without brittle reference editing or costly retraining. We present DiffWBC, shifting whole-body control to a generative paradigm via a one-step diffusion model. By modeling distributions over future trajectories, DiffWBC unifies general tracking, test-time steering, and simulation-free failure discovery within a single controller: (1) Generative general tracking: DiffWBC requires a single forward pass per 50 Hz control step and can be distilled from an RL teacher or trained from scratch via model-based RL, achieving the lowest local tracking error among six general trackers on held-out benchmarks (20.1 mm on BONES) and deploying on a physical Unitree G1. (2) Zero-shot test-time steering: differentiable cost gradients and body-group masks modulate the frozen controller online, substantially outperforming kinematic reference editing (0.84 clean pass at 1.25 m ceiling vs. 0.10) and zero-shot generalizing across hurdles, spatial reaching, actuator faults, and tray balancing. (3) Simulation-free failure discovery: plan dispersion across generation noise effectively identifies physical tracking failures (91% precision in top 2%) without physics simulations; targeted fine-tuning on flagged clips resolves 50% of failure cases. Videos and interactive demonstrations are available at https://diffwbc.github.io/#selfassess.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-02 · Indexed by Pub-AI: 2026-11-02 UTC

preprint

OneSteer: Unlocking Steerable General Humanoid Motion Tracking via One-Step Diffusion Whole-Body Control

Yifei Huang, Yazhao Yang, Li Weijia, Tianle Shi, Ji Jin, Zhihui Cai, Zihao Liu, Jianzhuang Zhao, Songlin Dong, Xinyuan Chang, Yihong Gong, Dian Shen, Zhiheng Ma

First announced by arXiv: 2026-11-02 · Original record ↗

arxiv
2610.41354
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-02 · First detected: 2026-11-02 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2610.41354v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

Deployment objectives often specify desired outcomes without providing revised trajectories, while parts of the original reference must still be followed. We present OneSteer, a one-step diffusion whole-body controller that combines general motion tracking with selective test-time steering. To accommodate new objectives, differentiable task costs guide optimization of the generator's input noise with frozen weights, while body-group reference masks relax selected tracking requirements. Predicted state-action plans provide a shared interface for task optimization and feedback-conditioned execution. For nominal tracking, the model trained from scratch achieves the lowest local tracking error among the evaluated baselines on successful held-out AMASS and BONES clips. For steering, one frozen distilled controller adapts across diverse deployment objectives in simulation, raising the clean passage rate under a 1.25 m bar from 2% to 90%. On a physical Unitree G1, OneSteer completes all 27 nominal tracking trials across nine diverse whole-body motion sequences without falling. With a steering cost derived from a signed distance field built using only onboard LiDAR, it also passes under a 1.1 m low bar in four of five attempts and steps over a 12 cm hurdle without falling in all three attempts. Beyond control, prediction variation in the model trained from scratch ranks likely tracking failures without physics rollouts. This scoring method is more than 10x faster than simulation-based scoring with five rollouts per clip and guides targeted fine-tuning. Project page: https://onesteer.github.io