Driving · 2026-10-21
PLaCA: Learning Patch-Level Future Latent Change for Action Generation
Zhan Yi, Yuchen Wang, Chao Li, Lei Hong, Boyuan Yang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
PLaCA is a world-to-action VLA framework that predicts spatially structured patch-level future latent change from current multimodal input, using change-aware supervision to weight patches by ground-truth change magnitude. A future-to-action interface conditions the action representation on the predicted future latent change. The authors report improved manipulation performance on LIBERO-Long and CALVIN.
Why it matters
The paper targets a gap between latent world modeling and downstream robotic manipulation: predicting full future frames can be expensive, and overly compressed latent states may not preserve local change structure. By predicting patch-level future latent change and directly conditioning action generation on it, PLaCA aims to provide action-relevant future information while keeping parameter updates small in a second training stage.
Method
- Predicts patch-level future latent change: a future-change decoder outputs per-camera, per-patch continuous latent change vectors without reconstructing full future latent states or images.
- Uses patch-level change-aware supervision: supervision weights are larger for patches with greater ground-truth future latent change magnitude.
- Connects future modeling to control via a gated future-to-action cross-attention interface that conditions the action representation on the final predicted patch-level future latent change.
Abstract (from arXiv)
Vision-Language-Action (VLA) models generate robot actions from visual observations and language instructions, but effective manipulation also requires anticipating how the environment will change. Existing world-action models explore future image prediction and latent dynamics modeling. However, generating complete future images can be computationally expensive, while highly compressed latent representations may obscure local changes. To address these challenges, we propose PLaCA, a world-to-action framework built around future latent change. PLaCA directly predicts spatially structured patch-level future latent change from the current multimodal input, preserving local information about expected environmental changes. We introduce patch-level change-aware supervision to emphasize regions undergoing larger changes and a future-to-action interface that explicitly incorporates predicted future latent change into action generation. Experiments on LIBERO-Long show that PLaCA achieves a 79.0% success rate, improving upon the Action-only baseline by 5.0 percentage points. On CALVIN, PLaCA improves the average sequence length from 2.784 to 3.564. These improvements are achieved while updating only 2.95% of the model parameters during the second training stage, demonstrating the effectiveness of future latent change modeling for downstream robotic manipulation.
Related papers
- WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction
- PLaW-VLA: Predictive Latent World Modeling for Vision-Language-Action Policies
- Vision-Language-Delta-Action: Learning Compact Deltas as Actionable Transition Interfaces for Robotic Manipulation
- ViDAL: A Visual Dynamics-Grounded Action Latent Space for Vision-Language-Action Models
- Beyond Actions: Learning Future Operation Targets for Vision-Language-Action Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-20 · Indexed by Pub-AI: 2026-10-21 UTC
PLaCA: Learning Patch-Level Future Latent Change for Action Generation
Preprint posted: 2026-10-20 · Original record ↗
- arxiv
- 2610.27657
Explicit source versions (1)
- v1 · 2026-10-20
- Preprint available
Source date: 2026-10-20 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · OAI arXiv created; not header datestamp (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Vision-Language-Action (VLA) models generate robot actions from visual observations and language instructions, but effective manipulation also requires anticipating how the environment will change. Existing world-action models explore future image prediction and latent dynamics modeling. However, generating complete future images can be computationally expensive, while highly compressed latent representations may obscure local changes. To address these challenges, we propose PLaCA, a world-to-action framework built around future latent change. PLaCA directly predicts spatially structured patch-level future latent change from the current multimodal input, preserving local information about expected environmental changes. We introduce patch-level change-aware supervision to emphasize regions undergoing larger changes and a future-to-action interface that explicitly incorporates predicted future latent change into action generation. Experiments on LIBERO-Long show that PLaCA achieves a 79.0% success rate, improving upon the Action-only baseline by 5.0 percentage points. On CALVIN, PLaCA improves the average sequence length from 2.784 to 3.564. These improvements are achieved while updating only 2.95% of the model parameters during the second training stage, demonstrating the effectiveness of future latent change modeling for downstream robotic manipulation.