Driving · 2026-10-01
Sparse-WAM: Accelerating World Action Models via Action-Guided Sparse Imagination
Xinling Xie, Haodong Wang, Jiazhi Mi, Zhiming Liu, Zicong Hong, Xiaoyi Pang, Qianli Liu, Yangjia Hu, Ying Chen, Zhengyang Yan, Song Guo
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
Sparse-WAM is a training-free framework for action-guided sparse imagination in world-action models. It uses action-to-future attention to select frame-specific core tokens plus shared spatial anchors, then uses Pilot to execute these selections efficiently via lightweight attention profiling and cross-step reuse, accelerating WAM inference while largely preserving performance.
Why it matters
The authors address high inference cost in diffusion-based world-action models caused by repeatedly processing dense future-frame tokens during denoising. By using action relevance (action-to-future attention) for token selection and reusing token selections across denoising steps, Sparse-WAM targets latency reduction without additional model training.
Method
- Reveals substantial spatial overlap in action-to-future attention between consecutive denoising steps, motivating reuse of token selections across steps.
- Action-Guided Token Selection retains Kc frame-specific core tokens plus Ks shared spatial anchors per future frame based on temporally aligned action-query groups and cross-frame consistency.
- Pilot implements online scoring without extra network forward, reuses selections within an action chunk, and uses cached predictions for omitted future regions during sparse denoising.
Limitation
Future work could broaden evaluation and deployment: extend beyond the NVIDIA RTX 4090, and further study additional WAMs beyond the three evaluated.
Abstract (from arXiv)
World-action models (WAMs) leverage pretrained video models to improve generalization in robot control by jointly predicting future visual states and actions. This capability comes at a substantial inference cost, as dense future-frame tokens are repeatedly processed during denoising. Prior methods address this by token pruning that prioritizes visual fidelity to reduce denoising costs in video diffusion models. However, these methods do not use action relevance to determine which future-frame tokens to retain during joint denoising in WAMs. In this paper, we propose Sparse-WAM, a training-free framework for action-guided sparse imagination that selectively processes future-frame tokens to accelerate WAM inference. We observe substantial overlap in the spatial distribution of attention from action tokens to future-frame tokens (action-to-future attention) between consecutive denoising steps, despite continued updates to the future representations. Motivated by this, we develop Action-Guided Token Selection to retain frame-specific action-relevant regions together with cross-frame context. However, a naive implementation can incur attention-scoring and token-packing overhead that offsets the computational savings from pruning. We therefore introduce Pilot, an efficient engine that reduces sparse inference overhead through lightweight scoring and cross-step reuse of token selections. On LIBERO with FastWAM-Joint and RoboLab-120 with Cosmos 3 Edge, Sparse-WAM achieves inference speedups of approximately $2.0\times$ and $1.8\times$, respectively, over dense eager inference on an NVIDIA RTX 4090, while largely preserving task performance.
Related papers
- 1Step-WAM: Your World Action Model but Faster using 1-Step Action Generation
- MARS: Systematic Multi-Level Acceleration for World Action Models
- {\Delta}WAM: Distilling Action Tangent Fields into World Action Models
- Complementary Future Token Pruning for World Action Models
- FLEX-WAM: Flexible Block-Causal World-Action Models for Long-Horizon Imagination and Planning
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-09-30 · Indexed by Pub-AI: 2026-10-01 UTC
Sparse-WAM: Accelerating World Action Models via Action-Guided Sparse Imagination
Preprint posted: 2026-09-30 · Original record ↗
- arxiv
- 2609.38984
Explicit source versions (1)
- v1 · 2026-09-30
- Preprint available
Source date: 2026-09-30 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World-action models (WAMs) leverage pretrained video models to improve generalization in robot control by jointly predicting future visual states and actions. This capability comes at a substantial inference cost, as dense future-frame tokens are repeatedly processed during denoising. Prior methods address this by token pruning that prioritizes visual fidelity to reduce denoising costs in video diffusion models. However, these methods do not use action relevance to determine which future-frame tokens to retain during joint denoising in WAMs. In this paper, we propose Sparse-WAM, a training-free framework for action-guided sparse imagination that selectively processes future-frame tokens to accelerate WAM inference. We observe substantial overlap in the spatial distribution of attention from action tokens to future-frame tokens (action-to-future attention) between consecutive denoising steps, despite continued updates to the future representations. Motivated by this, we develop Action-Guided Token Selection to retain frame-specific action-relevant regions together with cross-frame context. However, a naive implementation can incur attention-scoring and token-packing overhead that offsets the computational savings from pruning. We therefore introduce Pilot, an efficient engine that reduces sparse inference overhead through lightweight scoring and cross-step reuse of token selections. On LIBERO with FastWAM-Joint and RoboLab-120 with Cosmos 3 Edge, Sparse-WAM achieves inference speedups of approximately $2.0\times$ and $1.8\times$, respectively, over dense eager inference on an NVIDIA RTX 4090, while largely preserving task performance.