Robotics · 2026-10-30
LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
Jialei Chen, Kai Wang, Kang Chen, Shuaihang Chen, Feng Gao, Wenhao Tang, Zhiyuan Li, Weilin Liu, Zhuyu Yao, Boxun Li, Yuanbo Xu, Chao Yu
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
LaWAM proposes a latent-action-conditioned Latent World Model (LaWM) that predicts compact latent visual subgoals (future observation features) instead of generating future pixels/videos. The policy uses these predicted latent subgoals to produce dynamics-aware robot action chunks with low-latency, single-pass latent prediction. The authors report state-of-the-art or competitive success on LIBERO and RoboTwin and real-world manipulation.
Why it matters
Robot policies need foresight about how actions change the scene. Prior World-Action Models often rely on expensive pixel-space future video generation and incur high iterative inference latency. LaWAM aims to expose predictive dynamics through latent visual subgoals so action generation can be conditioned on predicted futures without costly pixel/video reconstruction.
Method
- Train a latent action model in a frozen vision foundation model’s latent feature space, and repurpose its forward decoder (LaWM) to predict future observation features for a fixed physical horizon.
- At inference, predict a latent action from current observation and instruction; decode it in a single forward pass into a predicted latent visual subgoal for the next action chunk.
- Integrate LaWM into a language-conditioned policy (with an Alternate-DiT action expert) via latent-action distillation and subgoal conditioning, including knowledge insulation (stop-gradient through the subgoal branch for action loss).
Limitation
The authors report a limitation in deformable-object manipulation: the current LaWM feature resolution is not yet fine-grained enough to precisely capture subtle cloth deformations.
Abstract (from arXiv)
Vision-Language-Action models (VLAs) leverage large-scale vision-language pretraining for semantic robot control, but often lack explicit foresight into how robot actions change the scene. World-Action Models (WAMs) address this limitation by conditioning policies on predicted futures, yet existing approaches typically rely on computationally expensive video generation with substantial pixel-level redundancy. We present LaWAM, a Latent World Action Model that exposes predictive dynamics to robot policies through compact latent visual subgoals instead of reconstructed future video. At the core of LaWAM is a latent-action-conditioned Latent World Model (LaWM). We obtain LaWM by training a latent action model in the latent space of a pretrained vision foundation model and repurposing its forward decoder to predict future observation features for scene evolution. LaWAM then conditions action generation on these predicted latent visual subgoals to enable dynamics-aware robot control. LaWAM achieves state-of-the-art or competitive success rates (SRs) across LIBERO (98.6% SR), RoboTwin (91.22% SR), and real-world manipulation tasks while retaining low-latency inference. LaWAM runs in 187 ms per action-chunk prediction and achieves up to 24x lower wall-clock latency than pixel-space WAMs.
Related papers
- WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction
- GAWM: A Compact Latent World-Action Model for Multi-Task Robot Manipulation
- TS-WAM: Temporal Latent Actions and World-Action Coupling for Vision-Language Robot Control
- ViDAL: A Visual Dynamics-Grounded Action Latent Space for Vision-Language-Action Models
- SmolWAM: A Latent World-Action Model for Robotic Manipulation on Edge Devices
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: Date not supplied · Indexed by Pub-AI: 2026-10-28 UTC
LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
Source date: Date not supplied · Original record ↗
- arxiv
- 2606.15768
Imported from the existing registry. No new publication milestone asserted.
Abstract · arxiv
Vision-Language-Action models (VLAs) leverage large-scale vision-language pretraining for semantic robot control, but often lack explicit foresight into how robot actions change the scene. World-Action Models (WAMs) address this limitation by conditioning policies on predicted futures, yet existing approaches typically rely on computationally expensive video generation with substantial pixel-level redundancy. We present LaWAM, a Latent World Action Model that exposes predictive dynamics to robot policies through compact latent visual subgoals instead of reconstructed future video. At the core of LaWAM is a latent-action-conditioned Latent World Model (LaWM). We obtain LaWM by training a latent action model in the latent space of a pretrained vision foundation model and repurposing its forward decoder to predict future observation features for scene evolution. LaWAM then conditions action generation on these predicted latent visual subgoals to enable dynamics-aware robot control. LaWAM achieves state-of-the-art or competitive success rates (SRs) across LIBERO (98.6% SR), RoboTwin (91.22% SR), and real-world manipulation tasks while retaining low-latency inference. LaWAM runs in 187 ms per action-chunk prediction and achieves up to 24x lower wall-clock latency than pixel-space WAMs.