Robotics · 2026-10-08
Long-WAM: Scaling the Context of World-Action Models
Wei Huang, Bohan Zhang, Chenzhi Liu, Isabella Liu, Shuai Yang, Weian Mao, Luozhou Wang, Yicheng Xiao, Weifeng Lin, Qixin Hu, Bryan Chu, Sifei Liu, Linxi Fan, Xiaojuan Qi, Song Han, Yukang Chen
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
Long-WAM is a model–system framework for scaling causal world-action models under real-time robot control constraints. The authors report that access to longer history is not the same as using it: longer context helps far more when the video foundation is pretrained autoregressively (AR). On RoboCasa GR-1, increasing context from 0.0 to 19.2s raises success 63.3%→78.7%.
Why it matters
The paper links improved closed-loop robot control to two factors: (1) causal video pretraining that learns history-to-future structure, and (2) deployment techniques that keep prediction within action deadlines. The authors show longer context can materially increase task success, but only when paired with AR video foundations and real-time inference infrastructure.
Method
- Staged training: first learn autoregressive (AR) robot-video predictive dynamics without action labels (LongLive2.0-Robot), then adapt to actions while preserving causal history-to-future structure (inverse dynamics modeling, IDM).
- Context scaling study: vary causal observation prefix length while keeping the action horizon/prediction horizon fixed; zero history is current observation only.
- Real-time deployment stack: streaming VAE encoding, asynchronous execution without blending/prefix guidance, and hardware-specific acceleration (NVFP4 quantization, CUDA Graph replay, KV reuse, kernel tuning) for RTX 5090, DGX Spark, and Jetson AGX Thor.
Limitation
Each context window in the study is trained as its own model (the paper notes this is a clean view of how much history helps at every length), rather than a single policy that adjusts its window at test time.
Abstract (from arXiv)
Real-time robot control demands enough visual history to infer motion and task progress, but processing that history can delay action. We present Long-WAM, a model-system framework for scaling the context of causal world-action models under real-time control constraints. Our central finding is that access to history is not the same as using it: longer histories pay off far more when the video foundation is pretrained autoregressively (AR). We first learn causal prediction from robot and egocentric videos without action labels, then preserve this history-to-future structure during world-action adaptation. On RoboCasa GR-1, increasing context from 0.0 to 19.2 seconds raises success from 63.3% to 78.7%, whereas a bidirectionally pretrained initialization shows no net gain; robot-domain AR pretraining further raises peak success on GR-1 and LIBERO-Long. Long-WAM also achieves the best results among compared methods on LIBERO-Long, RoboTwin 2.0, and DOMINO. Streaming observation encoding, asynchronous execution, and hardware-specific acceleration enable deployment on RTX 5090, DGX Spark, and Jetson AGX Thor without dropping future prediction; on RTX 5090, each action chunk, including future-video latent prediction, takes 107.4 ms. Real-time deployment on Unitree G1 and YAM supports dynamic and long-horizon manipulation, including 95% success on dynamic cup stacking, where Pi0.5 and Fast-WAM succeed in none of 20 trials. As a memory-informed executor, Long-WAM also complements higher-level planning in composite tasks.
Related papers
- RM-WAM: Reliable Memory-Augmented World Action Models for Long-Horizon Robot Manipulation
- OpenWAM: An Open Framework for Composable World-Action Models
- RealtimeWAM: How Fast Can I Run My World Action Model?
- {\Delta}WAM: Distilling Action Tangent Fields into World Action Models
- RAP-WAM: Learning World-Action Models through Predictive Rehearsal and Action Perturbations
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-07 · Indexed by Pub-AI: 2026-10-08 UTC
Long-WAM: Scaling the Context of World-Action Models
Preprint posted: 2026-10-07 · Original record ↗
- arxiv
- 2610.10528
Explicit source versions (1)
- v1 · 2026-10-07
- Preprint available
Source date: 2026-10-07 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Real-time robot control demands enough visual history to infer motion and task progress, but processing that history can delay action. We present Long-WAM, a model-system framework for scaling the context of causal world-action models under real-time control constraints. Our central finding is that access to history is not the same as using it: longer histories pay off far more when the video foundation is pretrained autoregressively (AR). We first learn causal prediction from robot and egocentric videos without action labels, then preserve this history-to-future structure during world-action adaptation. On RoboCasa GR-1, increasing context from 0.0 to 19.2 seconds raises success from 63.3% to 78.7%, whereas a bidirectionally pretrained initialization shows no net gain; robot-domain AR pretraining further raises peak success on GR-1 and LIBERO-Long. Long-WAM also achieves the best results among compared methods on LIBERO-Long, RoboTwin 2.0, and DOMINO. Streaming observation encoding, asynchronous execution, and hardware-specific acceleration enable deployment on RTX 5090, DGX Spark, and Jetson AGX Thor without dropping future prediction; on RTX 5090, each action chunk, including future-video latent prediction, takes 107.4 ms. Real-time deployment on Unitree G1 and YAM supports dynamic and long-horizon manipulation, including 95% success on dynamic cup stacking, where Pi0.5 and Fast-WAM succeed in none of 20 trials. As a memory-informed executor, Long-WAM also complements higher-level planning in composite tasks.