Robotics · 2026-10-01
Magic-W0: A Structured World-Action Foundation Model for Physical Intelligence
Xuhua Chen, Zhenhan Yin, Yuan Zhang, Lingfeng Zhang, He Zheng, Tong Mu, Shun Zuo, Dian Zhou, Di Wu, Xuan Zhou, Shaojie Wan, Rongtian Shen, Qiulong Xu, Yiduo Li, Yinglong Wang, Yanqian Wang, Kun Wang, Tao Zhang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
Magic-W0 is a world-action foundation model that organizes robot interaction as a Structured World Transition: Current State–Transition–Future State. Current State uses VLM context plus Current 3D Geometry, Transition is 3D Motion induced by actions, and Future State uses Future Semantics. A layer-aligned bidirectional architecture couples action hypotheses with structured world prediction to inform continuous action generation.
Why it matters
The authors target an explicit, control-oriented representation of how candidate actions change the 3D world and how that predicted world should feed back into action generation. Magic-W0 is designed to go beyond predicting future observations/features by structuring world predictions into geometry, motion, and task-relevant semantics and coupling them bidirectionally with continuous action generation during a unified forward pass.
Method
- Model physical interaction as Structured World Transition (Current State–Transition–Future State): VLM context + Current 3D Geometry, then 3D Motion for action-induced changes, then Future Semantics for task-relevant outcomes.
- Use a layer-aligned world–action interaction architecture where action hypotheses condition future world-transition prediction and predicted world representations continuously inform continuous action generation.
- Pre-train on large-scale egocentric human manipulation, UMI, real-robot trajectories, and simulation data with latent supervision for geometry, 3D motion, and future semantics from pre-trained visual models (Track4World and DINOv3 teachers).
Abstract (from arXiv)
World-action models (WAMs) augment robot policies with action-conditioned environment dynamics, yet existing approaches largely rely on future observation reconstruction or generic latent prediction and lack structured, control-oriented world representations tightly coupled with action generation. We introduce Magic-W0, a world-action foundation model that jointly models structured physical state evolution and continuous actions. Magic-W0 represents interaction as a Structured World Transition consisting of Current State, Transition, and Future State. Current State combines vision-language context with Current 3D Geometry; Transition is represented by 3D Motion capturing action-induced three-dimensional changes; and Future State is represented by Future Semantics describing task-relevant outcomes. To couple prediction and control, we propose a layer-aligned world-action interaction architecture in which evolving action hypotheses condition world-transition prediction, while predicted world representations continuously inform action generation. Magic-W0 is pre-trained on large-scale egocentric human manipulation, UMI, real-robot, and simulation data, with latent supervision for geometry, 3D motion, and future semantics from pre-trained visual models. Inference-time interventions show that structured world representations respond systematically to changes in candidate actions and that action-related information propagates through shared 3D representations into future semantic predictions. On RoboDojo-Sim, Magic-W0 achieves an average Score of 27.10, the highest among the compared WAMs. Across multiple real-robot tasks, it also demonstrates strong downstream performance after fine-tuning with limited downstream data, supporting generalization and rapid adaptation.
Related papers
- UniW(A)M: A Unified Mixture-of-Experts Framework for World and World-Action Models
- LAW: Unifying Language, Action, and World Modeling for Agentic Physical Intelligence
- Box-WAM: Box Trajectories as Compact State Representations for World Action Models
- UniWAM: Unified World-Action Model
- CogWAM: Aligning Semantic Cognition with World Action Modeling via Event-Driven Interfaces
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-09-30 · Indexed by Pub-AI: 2026-10-01 UTC
Magic-W0: A Structured World-Action Foundation Model for Physical Intelligence
Preprint posted: 2026-09-30 · Original record ↗
- arxiv
- 2609.39870
Explicit source versions (1)
- v2 · 2026-10-03
- Explicit source version detected · v2
Source date: 2026-10-03 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
- Preprint available
Source date: 2026-09-30 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World-action models (WAMs) augment robot policies with action-conditioned environment dynamics, yet existing approaches largely rely on future observation reconstruction or generic latent prediction and lack structured, control-oriented world representations tightly coupled with action generation. We introduce Magic-W0, a world-action foundation model that jointly models structured physical state evolution and continuous actions. Magic-W0 represents interaction as a Structured World Transition consisting of Current State, Transition, and Future State. Current State combines vision-language context with Current 3D Geometry; Transition is represented by 3D Motion capturing action-induced three-dimensional changes; and Future State is represented by Future Semantics describing task-relevant outcomes. To couple prediction and control, we propose a layer-aligned world-action interaction architecture in which evolving action hypotheses condition world-transition prediction, while predicted world representations continuously inform action generation. Magic-W0 is pre-trained on large-scale egocentric human manipulation, UMI, real-robot, and simulation data, with latent supervision for geometry, 3D motion, and future semantics from pre-trained visual models. Inference-time interventions show that structured world representations respond systematically to changes in candidate actions and that action-related information propagates through shared 3D representations into future semantic predictions. On RoboDojo-Sim, Magic-W0 achieves an average Score of 27.10, the highest among the compared WAMs. Across multiple real-robot tasks, it also demonstrates strong downstream performance after fine-tuning with limited downstream data, supporting generalization and rapid adaptation.