Latent Dynamics · 2026-10-28
GAWM: A Compact Latent World-Action Model for Multi-Task Robot Manipulation
Yang Liu, Xiaoli Liu, Liuzhao Li, Yuduo Song, Muyuan Liu, Xiang Gao
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
GAWM is a compact, task-conditioned latent world-action model for multi-task robot manipulation. It uses a frozen DINOv3 encoder to compress each camera view into 64 tokens, forecasts future tokens at two horizons in a task-conditioned predictor, and decodes an action chunk from current and predicted tokens. With 77M params and no robot/video pretraining, GAWM-S outperforms billion-parameter WAMs on RoboTwin 2.0 and is comparable on LIBERO, with lower real-robot inference latency.
Why it matters
The authors address high training cost and inference latency of many world-action models built on billion-parameter vision-language/video backbones. GAWM shows that reactive future conditioning can be achieved with a compact, latent JEPA-style predictor (task-conditioned rather than action-conditioned and without search), improving performance especially when training data or training time is limited, and enabling lower-latency real-robot deployment.
Method
- Encode each camera view with a frozen DINOv3 ViT-S into 64 learned-query tokens (64 tokens per view; 128 tokens for two cameras).
- Use a lightweight transformer latent predictor conditioned on a task code (not actions) to forecast the tokens at two future horizons in parallel; supervise using dense frozen-encoder patch features of future frames.
- Decode an action chunk using an ACT-style transformer decoder that regresses future actions from current tokens, predicted tokens, robot state tokens, and the task code; discard training-only patch decoder at inference.
Abstract (from arXiv)
World-action models (WAMs) incorporate the prediction of future observations into robot policy learning. However, most WAMs build on billion-parameter vision-language or video-generation backbones, which incur high training cost and inference latency. In this letter, we present GAWM (Geometry-Aware World Model), a compact multi-task WAM that predicts future observations in the latent space of a frozen DINOv3 encoder. GAWM compresses the features of each camera view into a small set of tokens. A lightweight predictor, conditioned on the task rather than on actions, forecasts these tokens at two future horizons. The forecast is supervised with the dense patch features of future frames, which requires it to encode the spatial layout of the future scene. An action decoder directly regresses a chunk of future actions from the current and predicted tokens. With fewer than 100M parameters and no robot or video pretraining, GAWM outperforms billion-parameter WAMs on RoboTwin 2.0 and performs comparably to billion-parameter VLAs on LIBERO. On a dual-arm robot, it achieves lower action-prediction error on held-out demonstrations than a fine-tuned billion-parameter VLA, with substantially lower inference latency. Ablation studies show that the latent world model substantially improves performance when training data or training time is limited. Code is available at https://github.com/gaoxiang12/ga-wm.
Related papers
- LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
- WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction
- SmolWAM: A Latent World-Action Model for Robotic Manipulation on Edge Devices
- TS-WAM: Temporal Latent Actions and World-Action Coupling for Vision-Language Robot Control
- MiniWAM: Learning Compact Future Targets for Efficient World-Action Modeling
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-27 · Indexed by Pub-AI: 2026-10-28 UTC
GAWM: A Compact Latent World-Action Model for Multi-Task Robot Manipulation
Preprint posted: 2026-10-27 · Original record ↗
- arxiv
- 2610.37091
Explicit source versions (1)
- v1 · 2026-10-27
- Preprint available
Source date: 2026-10-27 · First detected: 2026-10-28 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2610.37091v1; arxiv:announce_type=new; item pubDate (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World-action models (WAMs) incorporate the prediction of future observations into robot policy learning. However, most WAMs build on billion-parameter vision-language or video-generation backbones, which incur high training cost and inference latency. In this letter, we present GAWM (Geometry-Aware World Model), a compact multi-task WAM that predicts future observations in the latent space of a frozen DINOv3 encoder. GAWM compresses the features of each camera view into a small set of tokens. A lightweight predictor, conditioned on the task rather than on actions, forecasts these tokens at two future horizons. The forecast is supervised with the dense patch features of future frames, which requires it to encode the spatial layout of the future scene. An action decoder directly regresses a chunk of future actions from the current and predicted tokens. With fewer than 100M parameters and no robot or video pretraining, GAWM outperforms billion-parameter WAMs on RoboTwin 2.0 and performs comparably to billion-parameter VLAs on LIBERO. On a dual-arm robot, it achieves lower action-prediction error on held-out demonstrations than a fine-tuned billion-parameter VLA, with substantially lower inference latency. Ablation studies show that the latent world model substantially improves performance when training data or training time is limited. Code is available at https://github.com/gaoxiang12/ga-wm.