Driving · 2026-10-07
OpenWAM: An Open Framework for Composable World-Action Models
Heng Yu, David D. Yuan, Juze Zhang, Changan Chen, Yao Feng, Michelle Baldonado, Steve Cousins, Li Fei-Fei, Jiajun Wu, Ehsan Adeli
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
OpenWAM is an open world-action modeling framework that couples a common causal robot-video backbone (from Wan2.2-5B) with configurable video–action interaction programs (joint, VTA, ATV, decoupled) using a shared Mixture-of-Transformers architecture. Starting from causal robot-video pretraining on 10,000+ hours, it achieves higher VTA success on LIBERO-Long via causal adaptation and enables local-context inverse/forward dynamics learning with counterfactual transitions.
Why it matters
The authors address difficulty comparing WAM design choices by fixing a shared causal robot-video foundation and evaluation protocol while varying video–action interaction and conditioning interfaces. They also study whether counterfactual transitions improve reuse of separately trained dynamics components when the visual predictor is adapted to new tasks, extending beyond demonstrations alone.
Method
- Causal robot-video pretraining on heterogeneous robot/human interaction videos (from Wan2.2-5B) without action labels, using chunk-causal prediction for temporal causality.
- A shared Mixture-of-Transformers video–action model with reconfigurable future cross-modal attention: Joint, VTA, ATV, and Decoupled (different generation order/visibility).
- Separately trained local-context IDM/FDM that compose via explicit trajectory interfaces; in simulation, broaden training with counterfactual action continuations from the same initial state (LIBERO-Long-CF).
Abstract (from arXiv)
World-action models (WAMs) couple future prediction with robot control, yet existing systems often vary the video backbone, interaction structure, supervision, and inference procedure simultaneously, making their design choices difficult to compare. We introduce OPENWAM, an open world-action modeling framework built around a common causal robot-video foundation and configurable video-action interaction. Starting from Wan2.2-5B, we perform causal robot-video pretraining on over 10,000 hours of video, then integrate an action expert through a shared Mixture-of-Transformers architecture that supports joint, video-then-action, action-then-video, and decoupled generation. OPENWAM achieves high success rates on four LIBERO suites and real-world bimanual tasks; robot-video training with causal adaptation improves VTA success on LIBERO-Long from 68.4% to 97.8%. The same configurable architecture naturally extends to inverse and forward dynamics, allowing us to study how counterfactual transitions improve independently trained dynamics models beyond demonstrations alone. When only the video predictor is adapted to a new task, a frozen local-context inverse dynamics model trained on counterfactual data and demonstrations achieves 84.0% mean success across four held-out LIBERO-90 tasks, compared with 47.0% for a full-context inverse model and 21.5% for a local-context model trained only on demonstrations. For forward dynamics, counterfactual supervision reduces RGB prediction error by 34.5% and raises outcome identification from 21.1% to 71.3% among 16 same-state outcomes. OPENWAM provides a common testbed for comparing WAM interaction designs and for studying dynamics learning from video data beyond successful demonstrations.
Related papers
- Long-WAM: Scaling the Context of World-Action Models
- De-Diffusing WAMs: Action-Sufficient Direct Prediction for Robot Control
- EVO-WAM: Evolving World Action Models through Video-Action Verification
- Native Action-Prior Learning from Videos for World Action Models
- {\Delta}WAM: Distilling Action Tangent Fields into World Action Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-06 · Indexed by Pub-AI: 2026-10-07 UTC
OpenWAM: An Open Framework for Composable World-Action Models
Preprint posted: 2026-10-06 · Original record ↗
- arxiv
- 2610.07922
Explicit source versions (1)
- v1 · 2026-10-06
- Preprint available
Source date: 2026-10-06 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World-action models (WAMs) couple future prediction with robot control, yet existing systems often vary the video backbone, interaction structure, supervision, and inference procedure simultaneously, making their design choices difficult to compare. We introduce OPENWAM, an open world-action modeling framework built around a common causal robot-video foundation and configurable video-action interaction. Starting from Wan2.2-5B, we perform causal robot-video pretraining on over 10,000 hours of video, then integrate an action expert through a shared Mixture-of-Transformers architecture that supports joint, video-then-action, action-then-video, and decoupled generation. OPENWAM achieves high success rates on four LIBERO suites and real-world bimanual tasks; robot-video training with causal adaptation improves VTA success on LIBERO-Long from 68.4% to 97.8%. The same configurable architecture naturally extends to inverse and forward dynamics, allowing us to study how counterfactual transitions improve independently trained dynamics models beyond demonstrations alone. When only the video predictor is adapted to a new task, a frozen local-context inverse dynamics model trained on counterfactual data and demonstrations achieves 84.0% mean success across four held-out LIBERO-90 tasks, compared with 47.0% for a full-context inverse model and 21.5% for a local-context model trained only on demonstrations. For forward dynamics, counterfactual supervision reduces RGB prediction error by 34.5% and raises outcome identification from 21.1% to 71.3% among 16 same-state outcomes. OPENWAM provides a common testbed for comparing WAM interaction designs and for studying dynamics learning from video data beyond successful demonstrations.