Driving · 2026-10-20
PrismWAM: Decoupling State Representations in World Action Models
Tianyu Xin, Wanqi Zhong, Bowen Hu, Zhangchi Guo, Lijie Wang, Yiming Wang, Haoran Liao, Heyang Yu, Xiaoyu Tian, Zhiding Yu, Yiming Li
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
PrismWAM proposes decoupling semantic context from physical state in World Action Models. It factorizes embodied control into a Semantic Planning → Physical Rollout → Executable Action pipeline: from initial RGB it predicts a goal image (sparse), while a depth-based rollout predicts dense geometric evolution (dense) and generates continuous actions, enabling strong manipulation in simulation and the real world without embodied action pretraining.
Why it matters
The authors argue that pretrained video latents entangle semantics (appearance) with physical state (action-relevant geometry/pose/occupancy). PrismWAM separates these roles via an architecture that uses sparse RGB goal prediction to guide a depth rollout, improving manipulation and generalization and suggesting an action-centric state design for world-action modeling.
Method
- Decoupling principle: separate semantic context from physical evolution so action generation uses physical-state representations rather than monolithic video latents.
- Semantic Planning → Physical Rollout → Executable Action: semantic planner predicts an intended-future goal image from initial RGB; depth rollout predicts dense geometric changes in depth space; actions are generated from the depth rollout.
- Mixture-of-Transformers architecture with chain attention: RGB does not read depth/actions; depth does not read actions; action prediction uses evolving depth features without accessing future-RGB targets.
Abstract (from arXiv)
World Action Models (WAMs) typically represent the world through latent embeddings from pretrained video models, which entangle heterogeneous aspects of the world, from how it looks (e.g., texture and color) to how it is physically configured (e.g., occupancy and pose). This raises an underexplored question: $\textbf{what should constitute the state of a WAM?}$ We argue that the world state should separate how the world appears from how the world evolves under action: the former captures semantic context, while the latter is governed by physical state. To this end, we propose $\textbf{PrismWAM}$, which decouples physics from semantics while jointly modeling state and action. PrismWAM factorizes an embodied task into a $\mathrm{Semantic\ Planning}\rightarrow\mathrm{Physical\ Rollout}\rightarrow\mathrm{Executable\ Action}$ pipeline: given only the initial RGB observation, semantic planning learns to predict the final goal image, while physical rollout predicts the fine-grained evolution toward this goal in depth space; executable actions are then generated from the depth rollout. Without any embodied action pretraining or additional data augmentation, PrismWAM achieves strong manipulation performance in both simulation and the real world. Notably, PrismWAM even surpasses a broad range of models with large-scale embodied action pretraining (e.g., OpenWAM-$\alpha$). More broadly, we reframe World Action Models from modeling how the world looks to how it responds to action, paving the way for action-centric world models grounded in occupancy, flow, tactile sensing, proprioception, and beyond.
Related papers
- UniWAM: Unified World-Action Model
- {\Delta}WAM: Distilling Action Tangent Fields into World Action Models
- Thinker-WM: A Unified World Model with Shared Multimodal Representations for Embodied Intelligence
- Magic-W0: A Structured World-Action Foundation Model for Physical Intelligence
- WorldCast: Exploring Intelligent Visual Generation in World State Prediction
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-19 · Indexed by Pub-AI: 2026-10-20 UTC
PrismWAM: Decoupling State Representations in World Action Models
Preprint posted: 2026-10-19 · Original record ↗
- arxiv
- 2610.26564
Explicit source versions (1)
- v1 · 2026-10-19
- Preprint available
Source date: 2026-10-19 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World Action Models (WAMs) typically represent the world through latent embeddings from pretrained video models, which entangle heterogeneous aspects of the world, from how it looks (e.g., texture and color) to how it is physically configured (e.g., occupancy and pose). This raises an underexplored question: $\textbf{what should constitute the state of a WAM?}$ We argue that the world state should separate how the world appears from how the world evolves under action: the former captures semantic context, while the latter is governed by physical state. To this end, we propose $\textbf{PrismWAM}$, which decouples physics from semantics while jointly modeling state and action. PrismWAM factorizes an embodied task into a $\mathrm{Semantic\ Planning}\rightarrow\mathrm{Physical\ Rollout}\rightarrow\mathrm{Executable\ Action}$ pipeline: given only the initial RGB observation, semantic planning learns to predict the final goal image, while physical rollout predicts the fine-grained evolution toward this goal in depth space; executable actions are then generated from the depth rollout. Without any embodied action pretraining or additional data augmentation, PrismWAM achieves strong manipulation performance in both simulation and the real world. Notably, PrismWAM even surpasses a broad range of models with large-scale embodied action pretraining (e.g., OpenWAM-$α$). More broadly, we reframe World Action Models from modeling how the world looks to how it responds to action, paving the way for action-centric world models grounded in occupancy, flow, tactile sensing, proprioception, and beyond.