Latent Dynamics · 2026-10-14
UniW(A)M: A Unified Mixture-of-Experts Framework for World and World-Action Models
Jingbo Xu, Jianjie Fang, Zhuohang Li, Zihao Song, Zhaolu Wang, Xvyuan Liu, Xin Zhang, Haisheng Su, Yinzhou Tang, Chen Gao, Wei Wu, Yong Li
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
UniW(A)M is a mixture-of-experts world-(action) model built on a diffusion Transformer that unifies action-conditioned world modeling with four world-action interfaces via video–action visibility masks and CATR expert routing. It co-trains prediction and action generation, then uses PAIR to adaptively select an interface per execution chunk while keeping the backbone frozen. The authors report 88.3% average real-world success vs 76.7% for the strongest baseline.
Why it matters
The authors address limited knowledge reuse between world models and world-action models by jointly training shared dynamics for action-conditioned prediction and multiple action-generation interfaces. Their chunk-level router (PAIR) selects among interfaces to trade execution efficiency and foresight under OOD shifts while freezing the backbone, aiming to improve both success and robustness.
Method
- Unifies action-conditioned world modeling and four WAM interfaces (VGM, VLA, WM, IDM compositions) using video–action visibility masks inside a shared diffusion Transformer.
- Uses mixture-of-experts modules with Capability-Aligned Token Routing (CATR) so shared experts model common dynamics while atomic capability experts specialize by predictive role.
- Deploys Plug-in Adaptive Interface Router (PAIR) that selects one of four WAM interfaces per action chunk from a small subset of target-task chunk scenarios, keeping the UniW(A)M backbone frozen.
Abstract (from arXiv)
World models (WMs) and world-action models (WAMs) focus on future world prediction and executable action generation, respectively. Although these models learn shared physical dynamics and interaction semantics from embodied trajectories, they are typically trained in isolation, which restricts knowledge reuse between prediction and control. The central challenge is therefore to develop a unified learning framework that shares these dynamics while preserving the distinct modeling capabilities required by each paradigm. We propose UniW(A)M, a mixture-of-experts world-(action) model built upon diffusion Transformers that unifies action-conditioned world modeling and diverse WAM variants. Shared experts capture common dynamics, while atomic capability experts are reused across interfaces so that each interface keeps its own conditional pathway. At deployment, a plug-in adaptive interface router (PAIR) learns from a small subset of target-task chunk scenarios to select a suitable WAM interface for each execution chunk, balancing task success and execution efficiency while keeping the backbone frozen. We evaluate UniW(A)M on LIBERO, RoboTwin 2.0, and three real-world manipulation tasks, where it achieves 88.3% average real-world success versus 76.7% for the strongest baseline. On WorldArena, it leads action-conditioned world modeling on all three tasks. AC-WM co-training contributes to action generation on every action interface and PAIR raises OOD success from 63.3% to 76.7%.
Related papers
- Magic-W0: A Structured World-Action Foundation Model for Physical Intelligence
- UniWAM: Unified World-Action Model
- OpenWAM: An Open Framework for Composable World-Action Models
- Direct Experience World-Model Optimization: Learning the World Beyond Action Imitation
- MARS: Systematic Multi-Level Acceleration for World Action Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-13 · Indexed by Pub-AI: 2026-10-14 UTC
UniW(A)M: A Unified Mixture-of-Experts Framework for World and World-Action Models
Preprint posted: 2026-10-13 · Original record ↗
- arxiv
- 2610.18697
Explicit source versions (1)
- v2 · 2026-10-19
- Explicit source version detected · v2
Source date: 2026-10-19 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
- Preprint available
Source date: 2026-10-13 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World models (WMs) and world-action models (WAMs) focus on future world prediction and executable action generation, respectively. Although these models learn shared physical dynamics and interaction semantics from embodied trajectories, they are typically trained in isolation, which restricts knowledge reuse between prediction and control. The central challenge is therefore to develop a unified learning framework that shares these dynamics while preserving the distinct modeling capabilities required by each paradigm. We propose UniW(A)M, a mixture-of-experts world-(action) model built upon diffusion Transformers that unifies action-conditioned world modeling and diverse WAM variants. Shared experts capture common dynamics, while atomic capability experts are reused across interfaces so that each interface keeps its own conditional pathway. At deployment, a plug-in adaptive interface router (PAIR) learns from a small subset of target-task chunk scenarios to select a suitable WAM interface for each execution chunk, balancing task success and execution efficiency while keeping the backbone frozen. We evaluate UniW(A)M on LIBERO, RoboTwin 2.0, and three real-world manipulation tasks, where it achieves 88.3% average real-world success versus 76.7% for the strongest baseline. On WorldArena, it leads action-conditioned world modeling on all three tasks. AC-WM co-training contributes to action generation on every action interface and PAIR raises OOD success from 63.3% to 76.7%.