Driving · 2026-10-05
GlanceWAM: Sparse Test-Time Imagination for World-Action Models
Linhan Wang, Zijian An, Mingyuan Zhang, Chen Dai, Yi Xu, Can Cui, Jiayan Wang, Zichong Yang, Yinlin Chen, Lifeng Zhou, Chang-Tien Lu
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
GlanceWAM decouples visual imagination from high-frequency control by generating a single seconds-scale lookahead latent asynchronously off the critical path and conditioning action decoding on it purely in latent space. The action head runs at control rate (48 ms per chunk), while a non-interfering attention mask and staleness-robust horizon training handle asynchronous lookahead aging. Trained only on demonstrations, it improves manipulation success on RoboCasa kitchen and LIBERO and reduces control-path latency vs synchronous WAMs.
Why it matters
The authors address a stated “speed–success dilemma” in world-action models: synchronous test-time video imagination is latency-prohibitive, but removing imagination reduces task success. GlanceWAM’s asynchronous, sparse latent-space lookahead aims to keep the benefits of visual foresight while meeting real-time control budgets, and it reports improvements over synchronous and imagination-free baselines on multiple benchmarks, plus validation on real robots.
Method
- Use a shared latent video DiT backbone with an asynchronous proposer that generates one lookahead latent frame seconds ahead in the background on a slow clock.
- Decode action chunks at control rate (48 ms per chunk) purely in latent space without decoding lookahead to raw pixels, using the held lookahead latent and a decaying offset.
- Enable training/deployment consistency with a non-interfering (prefix-LM) attention mask to isolate lookahead tokens and staleness-robust horizon training via randomized offsets with temporal conditioning and lookahead token dropout.
Abstract (from arXiv)
Video generative models provide rich physical priors for robot learning, yet existing world-action models (WAMs) face a fundamental trade-off: synchronous video generation at control rate is latency-prohibitive, while abandoning test-time visual imagination sacrifices task success. We show that visual imagination achieves both real-time inference and superior success rates when generated asynchronously off the critical path and consumed directly in latent space. We introduce GlanceWAM, which decouples imagination from control on a single shared video DiT backbone: an asynchronous proposer glances ahead on a slow clock to imagine a single lookahead frame seconds into the future in the background, while an action head decodes action chunks at control rate (48 ms) purely in latent space without blocking. Enabled by a non-interfering attention mask that isolates video representations and staleness-robust horizon training that accommodates asynchronous lookahead aging, GlanceWAM breaks the speed-success dilemma. Trained purely on demonstrations, it attains 72.2% on the 24-task RoboCasa kitchen benchmark (vs. 67.1% for synchronous Cosmos Policy) and 99.0% on LIBERO while cutting per-chunk control latency $24\times$ relative to synchronous world-action models (48 ms on one A100). In single-arm and bimanual real-robot manipulation, it achieves higher average success than $\pi_{0.5}$ without any robot-data pretraining. Code is available at https://github.com/linhanwang/GlanceWAM.
Related papers
- FLEX-WAM: Flexible Block-Causal World-Action Models for Long-Horizon Imagination and Planning
- Sparse-WAM: Accelerating World Action Models via Action-Guided Sparse Imagination
- The Planning Limits of Latent World Models
- WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction
- RealtimeWAM: One-Step Asynchronous World Action Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-08-25 · Indexed by Pub-AI: 2026-10-05 UTC
GlanceWAM: Sparse Test-Time Imagination for World-Action Models
Preprint posted: 2026-08-25 · Original record ↗
- arxiv
- 2608.23927
Explicit source versions (1)
- v2 · 2026-09-29
- Preprint available
Source date: 2026-08-25 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
- Explicit source version detected · v2
Source date: 2026-09-29 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
Abstract · arxiv
Video generative models provide rich physical priors for robot learning, yet existing world-action models (WAMs) face a fundamental trade-off: synchronous video generation at control rate is latency-prohibitive, while abandoning test-time visual imagination sacrifices task success. We show that visual imagination achieves both real-time inference and superior success rates when generated asynchronously off the critical path and consumed directly in latent space. We introduce GlanceWAM, which decouples imagination from control on a single shared video DiT backbone: an asynchronous proposer glances ahead on a slow clock to imagine a single lookahead frame seconds into the future in the background, while an action head decodes action chunks at control rate (48 ms) purely in latent space without blocking. Enabled by a non-interfering attention mask that isolates video representations and staleness-robust horizon training that accommodates asynchronous lookahead aging, GlanceWAM breaks the speed-success dilemma. Trained purely on demonstrations, it attains 72.2% on the 24-task RoboCasa kitchen benchmark (vs. 67.1% for synchronous Cosmos Policy) and 99.0% on LIBERO while cutting per-chunk control latency $24\times$ relative to synchronous world-action models (48 ms on one A100). In single-arm and bimanual real-robot manipulation, it achieves higher average success than $π_{0.5}$ without any robot-data pretraining. Code is available at https://github.com/linhanwang/GlanceWAM.