Robotics · 2026-10-12
SmolWAM: A Latent World-Action Model for Robotic Manipulation on Edge Devices
Timi Owolabi, Hikmah Olawore, Frances Adelakun, Ayotomiwa Oyewumi
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
SmolWAM is a latent world-action model for robotic manipulation on edge devices. It derives from the 2B-parameter Cosmos Policy by keeping the pretrained trunk and rebuilding the inference interface (single-frame visual latents, cached text features, 12/28 transformer blocks, and 1,008 tokens read out in one rectified-flow step). On LIBERO, it reaches 95.5% success at 57 ms per 16-action chunk using 2.3 TFLOP and 2.4 GiB GPU memory.
Why it matters
The authors target the high cost of video diffusion transformers used by world-action models, aiming to make a pretrained video WAM small enough for resource-constrained edge devices while still forecasting future scene motion. The paper reports end-to-end LIBERO success with substantially reduced latency and compute, plus diagnostics showing which design changes preserve or harm long-horizon control.
Method
- Derive SmolWAM from the 2B-parameter Cosmos Policy by keeping the pretrained trunk and rebuilding the inference interface: single-frame latents, cached text features padded to preserve attention sinks, 12 of 28 transformer blocks, and 1,008 tokens read out in one rectified-flow step.
- Replace T5-XXL with cached BERT-base features projected to trunk width; keep attention sink behavior via zero-padding, and add zero-initialized vision-to-language layers so text can attend to images.
- Train with a distillation curriculum that matches one-step action chunks and future visual latents from the teacher, with a layout switch from the teacher’s token layout to a compact layout.
Limitation
The authors evaluate SmolWAM only on LIBERO; future work should extend evaluation to additional robotics benchmarks such as RoboCasa and, ultimately, real-robot settings. They also observe that reducing transformer depth disproportionately affects long-horizon tasks, with the largest degradation on LIBERO-Long, and note that developing pruning strategies that preserve long-horizon control performance remains an open problem.
Abstract (from arXiv)
World-action models (WAMs) predict a robot's actions together with the observations those actions will produce. However, the strongest models rely on video diffusion transformers that are far too computationally expensive for resource-constrained edge devices. We present SmolWAM, a latent WAM that we derive from the 2B-parameter Cosmos Policy by keeping its pretrained trunk and rebuilding its inference interface: single-frame visual latents, cached text features that preserve the trunk's attention sinks, 12 of 28 transformer blocks, and 1,008 tokens from which actions and future visual latents are read out in one rectified-flow step. Trained in 3.6 GPU-hours, SmolWAM succeeds in 95.5% of 400 LIBERO episodes at 57 ms and 2.3 TFLOP per 16-action chunk with 2.4 GiB of GPU memory, 11x faster than its teacher with 18.5x fewer FLOPs and 3.2x less memory, and its world head still forecasts scene motion.
Related papers
- WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction
- MARS: Systematic Multi-Level Acceleration for World Action Models
- TS-WAM: Temporal Latent Actions and World-Action Coupling for Vision-Language Robot Control
- RM-WAM: Reliable Memory-Augmented World Action Models for Long-Horizon Robot Manipulation
- Long-WAM: Scaling the Context of World-Action Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-09 · Indexed by Pub-AI: 2026-10-12 UTC
SmolWAM: A Latent World-Action Model for Robotic Manipulation on Edge Devices
Preprint posted: 2026-10-09 · Original record ↗
- arxiv
- 2610.13740
Explicit source versions (1)
- v1 · 2026-10-09
- Preprint available
Source date: 2026-10-09 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World-action models (WAMs) predict a robot's actions together with the observations those actions will produce. However, the strongest models rely on video diffusion transformers that are far too computationally expensive for resource-constrained edge devices. We present SmolWAM, a latent WAM that we derive from the 2B-parameter Cosmos Policy by keeping its pretrained trunk and rebuilding its inference interface: single-frame visual latents, cached text features that preserve the trunk's attention sinks, 12 of 28 transformer blocks, and 1,008 tokens from which actions and future visual latents are read out in one rectified-flow step. Trained in 3.6 GPU-hours, SmolWAM succeeds in 95.5% of 400 LIBERO episodes at 57 ms and 2.3 TFLOP per 16-action chunk with 2.4 GiB of GPU memory, 11x faster than its teacher with 18.5x fewer FLOPs and 3.2x less memory, and its world head still forecasts scene motion.