Driving · 2026-11-03
SparseWAM: Driving World-Action Models are Worth 16 Register Tokens
Yixiang Yang, Lizhong Zhang, Xingyu Zhang, Zhiwen Chen, Jie Ling, Qiankun Yu, Xingliang Liu, Yang Lei, Zhipeng Zhang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
SparseWAM is a driving world-action model that forecasts future scene dynamics using a sparse register space of 16 tokens per frame, instead of dense future videos or long patch-token features. The method uses a modality-decoupled Mixture-of-Transformers with a causal attention mask for imagine-then-act reasoning at deployment and supports multimodal sampling via rectified flow. The authors report 92.6 PDMS on NAVSIM v1, 90.1 EPDMS on NAVSIM v2, and 39.6 on NavHard with 164 ms latency, and release code upon acceptance.
Why it matters
The paper targets the imagination-vs-efficiency gap in driving world-action models: dense future-video generation causes high latency, while some efficient variants remove the world branch and lose test-time imagination. By compressing decision-relevant scene information into a 16-token sparse register space and keeping a causal imagine-then-act process at inference, SparseWAM aims to retain explicit future reasoning while reducing computational cost.
Method
- Learn a driving-oriented sparse register space by compressing frozen DINOv2 features with register tokens and shaping it via joint planning-and-scoring pretraining (planning imitation + score-distillation).
- Train a Mixture-of-Transformers denoiser with rectified flow to jointly model future registers and ego trajectories, using separate world/action experts and AdaLN conditioning by noise level.
- Use a causal attention mask so conditioning/clean history can’t be contaminated by noisy futures, and action tokens can condition on denoised future registers (imagine-then-act at deployment); sample multiple noise seeds for multimodal plans.
Limitation
The paper does not state a specific limitation.
Abstract (from arXiv)
World Action Models (WAMs) transfer scene-dynamics priors to planning by jointly forecasting future scenes and ego trajectories. However, WAMs built on video generators must generate dense future videos at inference, incurring prohibitive latency, while efficient variants discard the world branch at deployment and thus forgo test-time imagination. Even when dense video latents are replaced with patch-level DINO or JEPA features, these representations still involve long token sequences and encode decision-irrelevant information. We therefore propose SparseWAM, which forecasts in a sparse register space of only 16 tokens per frame, compressed from frozen DINOv2 features and shaped by joint planning-and-scoring pretraining to concentrate decision-relevant information. Within this space, SparseWAM employs a modality-decoupled Mixture-of-Transformers with a causal attention mask that lets action tokens condition on imagined future registers, retaining explicit imagine-then-act reasoning at deployment. Its compactness further enables low-cost multimodal sampling. As a result, our SparseWAM achieves 92.6 PDMS on NAVSIM v1, 90.1 EPDMS on NAVSIM v2, and 39.6 on NavHard at only 164 ms latency, roughly 1/5 that of DriveVA, demonstrating the efficiency of the sparse register space. Code will be released upon acceptance.
Related papers
- Sparse-WAM: Accelerating World Action Models via Action-Guided Sparse Imagination
- {\Delta}WAM: Distilling Action Tangent Fields into World Action Models
- FLEX-WAM: Flexible Block-Causal World-Action Models for Long-Horizon Imagination and Planning
- CtrlWAM: Controllable World Action Models with Aligned Intent and Foresight
- RealtimeWAM: One-Step Asynchronous World Action Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-11-03 · Indexed by Pub-AI: 2026-11-03 UTC
SparseWAM: Driving World-Action Models are Worth 16 Register Tokens
First announced by arXiv: 2026-11-03 · Original record ↗
- arxiv
- 2611.00044
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-11-03 · First detected: 2026-11-03 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.00044v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
World Action Models (WAMs) transfer scene-dynamics priors to planning by jointly forecasting future scenes and ego trajectories. However, WAMs built on video generators must generate dense future videos at inference, incurring prohibitive latency, while efficient variants discard the world branch at deployment and thus forgo test-time imagination. Even when dense video latents are replaced with patch-level DINO or JEPA features, these representations still involve long token sequences and encode decision-irrelevant information. We therefore propose SparseWAM, which forecasts in a sparse register space of only 16 tokens per frame, compressed from frozen DINOv2 features and shaped by joint planning-and-scoring pretraining to concentrate decision-relevant information. Within this space, SparseWAM employs a modality-decoupled Mixture-of-Transformers with a causal attention mask that lets action tokens condition on imagined future registers, retaining explicit imagine-then-act reasoning at deployment. Its compactness further enables low-cost multimodal sampling. As a result, our SparseWAM achieves 92.6 PDMS on NAVSIM v1, 90.1 EPDMS on NAVSIM v2, and 39.6 on NavHard at only 164 ms latency, roughly 1/5 that of DriveVA, demonstrating the efficiency of the sparse register space. Code will be released upon acceptance.
