Robotics · 2026-10-02
UniWAM: Unified World-Action Model
Jiayi Chen, Wenxuan Song, Jingbo Wang, Shuai Zhou, Xicheng Gong, Zehua Fan, Ziyang Zhou, Junwu E, Haodong Yan, Fuhao Li, Qize Yu, Xu Huang, Pengwei Wang, Wen Chen, Shunbo Zhou, Haoang Li
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
UniWAM is a unified world-action model that integrates a VLM-based physical reasoner, a VGM-based world generator, and an action predictor in a Mixture-of-Transformers (MoT) architecture. It uses robot, human egocentric, and VQA data with a data cleaning/annotation pipeline, plus pre-training/post-training recipes (natural-language action supervision, future visual noise augmentation, history-conditioned flow matching) to reduce denoising steps while maintaining performance and claims SOTA across multiple evaluations.
Why it matters
The authors target the gap between vision-language-action models (limited world-dynamics grounding from action-only supervision) and world-action/video models (weaker semantic/reasoning and robustness under distribution shifts). By unifying physical reasoning, visual generation, and action prediction with complementary supervision and denoising-step reductions, UniWAM aims to improve instruction following, robustness, generalization, and long-horizon task execution.
Method
- Unified MoT architecture: physical reasoner, world generator, and action predictor connected via joint multimodal attention, trained jointly for semantic understanding, visual generation, and action prediction.
- Pre-training data/supervision mixture: three-source scheme (robot teleoperation, human egocentric, and VQA). Low-level actions are represented in natural language; supervision is assigned to different components to preserve language knowledge.
- Post-training: future-visual noise augmentation and history-conditioned flow matching to reduce reliance on precise future predictions and use encoded action history to initialize action generation; authors report fewer denoising steps while maintaining performance.
Abstract (from arXiv)
Vision-language-action models benefit from the understanding and reasoning capabilities of pretrained vision-language models, but action-only supervision provides limited grounding in world dynamics. Conversely, world-action models inherit spatiotemporal priors from video generation models, yet remain limited in semantic understanding and reasoning under distribution shifts. We introduce UniWAM, a unified architecture that integrates a physical reasoner, a world generator, and an action predictor to jointly learn semantic understanding of the physical world, visual generation, and action prediction. To ensure the quality of the training data, we developed a rigorous data cleaning and annotation pipeline for both human egocentric data and robot data. To adapt the vision-language component to embodied tasks while preserving its inherited language capabilities, we represent low-level actions in natural language and introduce a pre-training recipe that assigns complementary supervision from visual question answering (VQA) data, human egocentric data, and robot demonstrations to the appropriate model components. During post-training, future visual noise augmentation reduces reliance on precise future predictions, while history-conditioned flow matching uses encoded action history to initialize action generation. Together, these designs significantly reduce denoising steps while maintaining performance. UniWAM achieves state-of-the-art (SOTA) performance across multiple evaluations, including in-distribution performance, robustness, generalization, instruction following, and long-horizon task execution. Furthermore, we uncover a log-linear scaling law of unified human-robot co-training, demonstrating the effectiveness of large-scale pre-training on a mixture of human and robot data.
Related papers
- UniWALM: Unified World Action Language Model for Embodied Robot Control
- EWAM: Emergent Depth-Wise Specialization in a Unified Embodied Model -- From Semantic Understanding through Visual Foresight to Action
- LAW: Unifying Language, Action, and World Modeling for Agentic Physical Intelligence
- Language-Forcing: Advancing Humanoid Loco-Manipulation via Semantic Alignment of Whole-Body World Action Models
- SUAVE: Unified Video-Action Models via Masked Diffusion
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-01 · Indexed by Pub-AI: 2026-10-02 UTC
UniWAM: Unified World-Action Model
Preprint posted: 2026-10-01 · Original record ↗
- arxiv
- 2610.02054
Explicit source versions (1)
- v4 · 2026-10-14
- Preprint available
Source date: 2026-10-01 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
- Explicit source version detected · v4
Source date: 2026-10-14 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
Abstract · arxiv
Vision-language-action models benefit from the understanding and reasoning capabilities of pretrained vision-language models, but action-only supervision provides limited grounding in world dynamics. Conversely, world-action models inherit spatiotemporal priors from video generation models, yet remain limited in semantic understanding and reasoning under distribution shifts. We introduce UniWAM, a unified architecture that integrates a physical reasoner, a world generator, and an action predictor to jointly learn semantic understanding of the physical world, visual generation, and action prediction. To ensure the quality of the training data, we developed a rigorous data cleaning and annotation pipeline for both human egocentric data and robot data. To adapt the vision-language component to embodied tasks while preserving its inherited language capabilities, we represent low-level actions in natural language and introduce a pre-training recipe that assigns complementary supervision from visual question answering (VQA) data, human egocentric data, and robot demonstrations to the appropriate model components. During post-training, future visual noise augmentation reduces reliance on precise future predictions, while history-conditioned flow matching uses encoded action history to initialize action generation. Together, these designs significantly reduce denoising steps while maintaining performance. UniWAM achieves state-of-the-art (SOTA) performance across multiple evaluations, including in-distribution performance, robustness, generalization, instruction following, and long-horizon task execution. Furthermore, we uncover a log-linear scaling law of unified human-robot co-training, demonstrating the effectiveness of large-scale pre-training on a mixture of human and robot data.