Driving · 2026-10-09
Humanoid World Action Model With Joint State--Action Generation
Yan Yang, Jikun Rong, Minzhao Zhu, Zheyi Zhao, Qirui Hu, Zihan Lan, Weixin Mao, Yinhao Li, Zhen Fu, Hua Chen
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
HWAM (Humanoid World Action Model) targets a humanoid action–execution gap by jointly generating high-level reference actions and the robot’s post-execution proprioceptive state. It trains with three conditional paths: Policy (state–action only), Forward Dynamics Modeling (future video conditioned on actions and realized states), and Inverse Dynamics Modeling (actions and realized states from visual transitions). The authors report the highest success rate among evaluated baselines on three LimX OLI tasks.
Why it matters
In hierarchical humanoid control, policy-produced reference actions may differ from executed motion. The authors argue that without explicitly modeling realized body state, future visual prediction must explain both execution discrepancies and scene evolution. HWAM makes the post-execution proprioceptive state an explicit prediction target and uses it to ground both forward and inverse visual objectives, aiming to link reference actions to physical and visual outcomes.
Method
- Joint target: aligns each reference action with its temporally corresponding post-execution proprioceptive state, concatenating them into a state–action chunk for shared denoising.
- Three training paths share the same experts: Policy denoises state–action conditioned on current observations (no future video); FDM predicts future video conditioned on actions and realized states; IDM reconstructs joint trajectory from visual transitions.
- Deployment matches hierarchy: predicted states participate in joint denoising, but only predicted actions are sent to the low-level controller.
Limitation
The authors state that HWAM does not explicitly identify low-level dynamics, and its predicted states are not sent to the controller; they serve as jointly generated variables that connect high-level actions to physical and visual consequences.
Abstract (from arXiv)
Humanoid robots are a promising platform for general-purpose manipulation. Recent Vision-Language-Action (VLA) policies learn actions directly from multimodal observations, while World Action Models (WAMs) further incorporate future visual prediction to improve action generation. However, in hierarchical humanoid systems, VLA and WAM policies output reference actions that are subsequently realized through whole-body control, robot dynamics, balance, and contact. This hierarchy creates an action--execution gap: the reference produced by the policy can differ from the motion realized by the robot. Without explicitly modeling the realized body state, future visual prediction must jointly explain scene evolution and discrepancies between reference actions and executed motion, making it difficult to associate an action with its physical outcome. We propose HWAM, a Humanoid World Action Model with joint state--action generation, which makes the robot's post-execution proprioceptive state an explicit prediction target. By jointly generating reference actions and their realized body states, HWAM directly incorporates supervision of executed motion into action learning. HWAM is trained through three complementary conditional paths. The Policy path jointly denoises state--action trajectories conditioned only on current observations, matching deployment conditions. Forward Dynamics Modeling (FDM) predicts future visual observations conditioned on actions and post-execution states, while Inverse Dynamics Modeling (IDM) reconstructs the joint trajectory from visual transitions. Together, these paths connect policy references, realized body motion, and visual outcomes. HWAM achieves the highest success rate among evaluated baselines on three real-robot tasks on the LimX OLI humanoid. On Candy Picking, HWAM achieves a 70.6% success rate, compared with 43.3% for Fast-WAM.
Related papers
- GreenWAM: A World Foundation Model for Robotics
- Language-Forcing: Advancing Humanoid Loco-Manipulation via Semantic Alignment of Whole-Body World Action Models
- Being-M0.7: A Latent World-Action Model for Humanoid Robots
- SkeleWAM: Skeleton World-Action Modeling for Efficient Robotic Manipulation
- TS-WAM: Temporal Latent Actions and World-Action Coupling for Vision-Language Robot Control
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-08 · Indexed by Pub-AI: 2026-10-09 UTC
HWAM: Humanoid World Action Model With Joint State--Action Generation
Preprint posted: 2026-10-08 · Original record ↗
- arxiv
- 2610.12026
Explicit source versions (1)
- v2 · 2026-10-12
- Preprint available
Source date: 2026-10-08 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
- Explicit source version detected · v2
Source date: 2026-10-12 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
Abstract · arxiv
Humanoid robots are a promising platform for general-purpose manipulation. Recent Vision-Language-Action (VLA) policies learn actions directly from multimodal observations, while World Action Models (WAMs) further incorporate future visual prediction to improve action generation. However, in hierarchical humanoid systems, VLA and WAM policies output reference actions that are subsequently realized through whole-body control, robot dynamics, balance, and contact. This hierarchy creates an action--execution gap: the reference produced by the policy can differ from the motion realized by the robot. Without explicitly modeling the realized body state, future visual prediction must jointly explain scene evolution and discrepancies between reference actions and executed motion, making it difficult to associate an action with its physical outcome. We propose HWAM, a Humanoid World Action Model with joint state--action generation, which makes the robot's post-execution proprioceptive state an explicit prediction target. By jointly generating reference actions and their realized body states, HWAM directly incorporates supervision of executed motion into action learning. HWAM is trained through three complementary conditional paths. The Policy path jointly denoises state--action trajectories conditioned only on current observations, matching deployment conditions. Forward Dynamics Modeling (FDM) predicts future visual observations conditioned on actions and post-execution states, while Inverse Dynamics Modeling (IDM) reconstructs the joint trajectory from visual transitions. Together, these paths connect policy references, realized body motion, and visual outcomes. HWAM achieves the highest success rate among evaluated baselines on three real-robot tasks on the LimX OLI humanoid. On Candy Picking, HWAM achieves a 70.6% success rate, compared with 43.3% for Fast-WAM.