Robotics · 2026-10-20
Vision-Language-Delta-Action: Learning Compact Deltas as Actionable Transition Interfaces for Robotic Manipulation
Lingsen Zhang, Wei Li, Jie He, Henghua Zhang, Kai Mo, Shiyong Li, Yanpeng Wang, Rui Shao
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
VLDA (Vision-Language-Delta-Action) reframes robotic manipulation from observation-to-action prediction to an understanding–transition–execution loop. It introduces a compact actionable Delta latent capturing task-specified physical state changes via a unified objective (LT-Filter + GT-Anchor), then uses Delta-Driven Action Refinement (DDAR) to seed action denoising, aiming to avoid wasting capacity on full future-state prediction.
Why it matters
The authors argue that direct VLA learns an end-to-end mapping that must implicitly infer task-relevant transitions, while world-action modeling often predicts complete future states that entangle sparse manipulation cues with static/action-irrelevant changes. VLDA proposes an intermediate transition interface (Delta) designed to be selective, future-consistent, and grounded in executable actions to improve long-horizon manipulation.
Method
- Learn a compact actionable Delta Δ_t from multimodal context h_t instead of predicting complete future states; model transition generation as continuous-time flow matching in Delta space.
- Use unified transition objectives: Local Transition Filter (LT-Filter) selects top-K regions with largest current-to-future feature-transition magnitudes to suppress static reconstruction bias; Global Transition Anchor (GT-Anchor) regularizes Delta with a coarse summary of overall future-evolution direction.
- Ground Delta in actions via Delta-Driven Action Refinement (DDAR): Delta induces a noise bias and a Delta prefix token to guide action denoising; also align Delta with the action manifold using Delta-to-Action Manifold Grounding.
Limitation
LT-Filter selects task-relevant changes based on feature difference magnitude between current and future observations, so intense background motion could occasionally pollute the top-K selection; the paper also notes fixed-horizon state transition modeling and proposes future work on variable/adaptive temporal horizons and cross-embodiment transfer.
Abstract (from arXiv)
Robotic manipulation is not merely observation-to-action generation, but the realization of task-specified state changes in the physical world. Existing vision-language-action (VLA) methods directly learn observation-to-action mappings, imposing a substantial learning burden due to the gap between semantic understanding and action execution. Although world-action models (WAMs) improve control through future prediction, modeling complete future states wastes capacity on static appearance and action-irrelevant changes. We propose Vision-Language-Delta-Action (VLDA), a paradigm that reformulates full future prediction as compact actionable transition modeling. Its core is a compact Delta variable that captures latent world-state changes, serving as an interface between vision-language understanding and action generation. VLDA learns Delta through a unified objective composed of the Local Transition Filter (LT-Filter) and Global Transition Anchor (GT-Anchor). LT-Filter captures task-relevant manipulation-centric changes while suppressing static reconstruction biases. GT-Anchor provides coarse future-level regularization to preserve the overall direction of state changes. Delta-Driven Action Refinement (DDAR) further grounds Delta in the action manifold and injects it as a structured seed to guide action denoising. We implement VLDA within LoongForge, an open-source training framework for large-scale foundation and embodied models, using its modular and distributed infrastructure for unified two-stage optimization. Experiments show that VLDA improves long-horizon consistency, robustness, and generalization, validating actionable Delta as an alternative to direct action prediction and full future-state modeling.
Related papers
- WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction
- Rewiring Semantics, Dynamics, and Control: A Simple yet Effective Action-Centric Tri-Stream Transformer
- Beyond Actions: Learning Future Operation Targets for Vision-Language-Action Models
- PLaCA: Learning Patch-Level Future Latent Change for Action Generation
- TS-WAM: Temporal Latent Actions and World-Action Coupling for Vision-Language Robot Control
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-19 · Indexed by Pub-AI: 2026-10-20 UTC
Vision-Language-Delta-Action: Learning Compact Deltas as Actionable Transition Interfaces for Robotic Manipulation
Preprint posted: 2026-10-19 · Original record ↗
- arxiv
- 2610.25874
Explicit source versions (1)
- v1 · 2026-10-19
- Preprint available
Source date: 2026-10-19 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Robotic manipulation is not merely observation-to-action generation, but the realization of task-specified state changes in the physical world. Existing vision-language-action (VLA) methods directly learn observation-to-action mappings, imposing a substantial learning burden due to the gap between semantic understanding and action execution. Although world-action models (WAMs) improve control through future prediction, modeling complete future states wastes capacity on static appearance and action-irrelevant changes. We propose Vision-Language-Delta-Action (VLDA), a paradigm that reformulates full future prediction as compact actionable transition modeling. Its core is a compact Delta variable that captures latent world-state changes, serving as an interface between vision-language understanding and action generation. VLDA learns Delta through a unified objective composed of the Local Transition Filter (LT-Filter) and Global Transition Anchor (GT-Anchor). LT-Filter captures task-relevant manipulation-centric changes while suppressing static reconstruction biases. GT-Anchor provides coarse future-level regularization to preserve the overall direction of state changes. Delta-Driven Action Refinement (DDAR) further grounds Delta in the action manifold and injects it as a structured seed to guide action denoising. We implement VLDA within LoongForge, an open-source training framework for large-scale foundation and embodied models, using its modular and distributed infrastructure for unified two-stage optimization. Experiments show that VLDA improves long-horizon consistency, robustness, and generalization, validating actionable Delta as an alternative to direct action prediction and full future-state modeling.