Driving · 2026-10-06
RealtimeWAM: One-Step Asynchronous World Action Models
Chengtao Lv, Jinyang Du, Shuyi Feng, Yang Yong, Shiqiao Gu, Shunzi Yang, Ruihao Gong, Shen Ren, Tianwei Zhang, Wenya Wang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
RealtimeWAM is a one-step, asynchronous post-trained World Action Model for MoT-based WAMs. It uses Teacher-Anchored Consistency Distillation (TACD) to address a local-global error gap for accurate one-step action generation, and Cross-Expert Wavefront Pipelining (CEWP) to overlap the Video and Action experts at the block level by sharing video KV caches. The authors report near-lossless performance (<1% drop) with ~25× end-to-end speedup on H100.
Why it matters
The paper targets two remaining inference bottlenecks in MoT-based WAMs: multi-step action denoising latency and sequential execution waiting between the video and action experts. By combining one-step distillation (TACD) with finer-grained expert pipelining (CEWP), the authors aim for real-time robot action generation while preserving task success.
Method
- Teacher-Anchored Consistency Distillation (TACD): augment local consistency supervision with explicit supervision from the frozen teacher’s multi-step rollout endpoint (via interval-averaged teacher velocity) to mitigate a local-global error gap.
- Teacher rollout: obtain teacher endpoint through a 10-step rollout of the frozen teacher, and supervise the student’s velocity prediction against the teacher’s interval-averaged velocity using a teacher-anchored loss weighted by λ.
- Cross-Expert Wavefront Pipelining (CEWP): overlap Video and Action experts via block-wise sharing of video KV cache, synchronizing only immediately before action attention consumes the corresponding cache entry.
Limitation
RealtimeWAM targets MoT-based WAMs and is not directly applicable to shared-backbone architectures such as DreamZero. The authors also note training overhead from teacher-anchored loss requiring multi-step rollouts from the frozen teacher, though they state it remains manageable.
Abstract (from arXiv)
World Action Models (WAMs) incorporate visual representations from video generation backbones to guide action prediction. Recent efficient WAMs adopt Mixture-of-Transformers (MoT) architectures and compute video representations once for reuse by the action expert. However, intra-expert iteration (\ie, multi-step action denoising) and inter-expert waiting (\ie, sequential execution of the video and action experts) still limit inference efficiency. To this end, we present RealtimeWAM, an extremely efficient WAM variant with one-step action generation and asynchronous inference, addressing these two bottlenecks. To reduce intra-expert iteration, we propose Teacher-Anchored Consistency Distillation (TACD) to address a local-global error gap: low local consistency error alone does not guarantee accurate final actions. TACD supplements local consistency with explicit supervision from the frozen teacher's multi-step rollout endpoint, enabling accurate one-step action generation. Additionally, we propose Cross-Expert Wavefront Pipelining (CEWP) to eliminate unnecessary expert-level waiting. It overlaps the two experts through block-wise sharing of the video KV cache, synchronizing only immediately before the corresponding action attention consumes it. Extensive experiments across diverse benchmarks (\eg, LIBERO, LIBERO-Plus and RoboTwin) and model variants (\eg, Fast-WAM and Faster-WAM) demonstrate the superiority of RealtimeWAM. Notably, RealtimeWAM maintains near-lossless performance (\ie, $<1\%$ drop) across these benchmarks while delivering significant end-to-end speedup (\eg, $\sim25\times$ on H100). Our code and checkpoints are available via this \href{https://github.com/ModelTC/LightX2V/tree/main/examples/realtimewam}{link}.
Related papers
- RealtimeWAM: How Fast Can I Run My World Action Model?
- 1Step-WAM: Your World Action Model but Faster using 1-Step Action Generation
- WAM-Cache: Staleness-Bounded KV Reuse for Efficient World Action Models
- Sparse-WAM: Accelerating World Action Models via Action-Guided Sparse Imagination
- MARS: Systematic Multi-Level Acceleration for World Action Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-05 · Indexed by Pub-AI: 2026-10-06 UTC
RealtimeWAM: One-Step Asynchronous World Action Models
Preprint posted: 2026-10-05 · Original record ↗
- arxiv
- 2610.06617
Explicit source versions (1)
- v2 · 2026-10-16
- Preprint available
Source date: 2026-10-05 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
- Explicit source version detected · v2
Source date: 2026-10-16 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
Abstract · arxiv
World Action Models (WAMs) incorporate visual representations from video generation backbones to guide action prediction. Recent efficient WAMs adopt Mixture-of-Transformers (MoT) architectures and compute video representations once for reuse by the action expert. However, intra-expert iteration (\ie, multi-step action denoising) and inter-expert waiting (\ie, sequential execution of the video and action experts) still limit inference efficiency. To this end, we present RealtimeWAM, an extremely efficient WAM variant with one-step action generation and asynchronous inference, addressing these two bottlenecks. To reduce intra-expert iteration, we propose Teacher-Anchored Consistency Distillation (TACD) to address a local-global error gap: low local consistency error alone does not guarantee accurate final actions. TACD supplements local consistency with explicit supervision from the frozen teacher's multi-step rollout endpoint, enabling accurate one-step action generation. Additionally, we propose Cross-Expert Wavefront Pipelining (CEWP) to eliminate unnecessary expert-level waiting. It overlaps the two experts through block-wise sharing of the video KV cache, synchronizing only immediately before the corresponding action attention consumes it. Extensive experiments across diverse benchmarks (\eg, LIBERO, LIBERO-Plus and RoboTwin) and model variants (\eg, Fast-WAM and Faster-WAM) demonstrate the superiority of RealtimeWAM. Notably, RealtimeWAM maintains near-lossless performance (\ie, $<1\%$ drop) across these benchmarks while delivering significant end-to-end speedup (\eg, $\sim25\times$ on H100). Our code and checkpoints are available via this \href{https://github.com/ModelTC/LightX2V/tree/main/scripts/realtimewam}{link}.