Driving · 2026-10-01
MotionWeave: Learning Motion-Centered Future Dynamics for Vision-Language-Action Policies
Jingqiu Wang, Yan Wang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
MotionWeave is a motion-centric future-dynamics framework for vision-language-action action-chunk prediction. It uses AIMG to form horizon-specific queries that localize interaction regions from current visual tokens, supervised with future robot-arm masks via KL loss during training. HRC composes adjacent-horizon interaction differences into motion cues injected into action tokens via a gated residual. On six MetaWorld tasks it reports 75.3% average success, +8.6% over π0.
Why it matters
The authors argue VLA policies should not redundantly reconstruct complete future visual observations. MotionWeave instead localizes action-relevant motion regions per future timestep and injects timestep-specific motion increments into action decoding, using future-frame masks only for training. This directly targets the timestep-to-consequence correspondence problem highlighted in the introduction.
Method
- AIMG conditions action tokens and proprioceptive embeddings to build horizon-specific queries that attend over current visual tokens, producing interaction tokens and horizon attention α supervised by KL to horizon-specific robot-arm masks during training.
- HRC projects interaction representations, computes adjacent-horizon differences, and injects them into action tokens through cross-attention and a gated residual update (motion increments are absorbed only per timestep).
- Training optimizes action prediction loss plus λ_motion=0.05 motion-grounding KL loss; inference removes the mask/KL branch and uses only the current observation for action-chunk generation.
Abstract (from arXiv)
Vision-Language-Action (VLA) models have recently incorporated world models to provide richer dynamic supervision beyond sparse action labels. However, explicitly predicting future images or videos may include control-irrelevant appearance, while guidance derived from holistic future visual representations and shared global action features may fail to establish timestep-specific correspondence between actions and local visual changes. To address this issue, we propose MotionWeave, a motion-centric future-dynamics framework for action-chunk prediction with two modules: the Action-Induced Motion Grounder (AIMG) and the Horizon Residual Composer (HRC). Specifically, AIMG conditions on action and proprioceptive representations to construct horizon-specific queries that localize interaction regions associated with each future action timestep from current visual tokens. HRC extracts differences between interaction representations at adjacent horizons, encodes them as temporal motion cues, and injects them into action tokens through a gated residual. During training, robot-arm masks rendered from future frames are used to construct KL-based motion-grounding supervision, while inference uses only the current observation. On six MetaWorld tasks, MotionWeave achieves a 75.3% average success rate, an absolute gain of 8.6% over {\pi}0 (66.7%), especially on sustained-interaction tasks. Our code is available at https://github.com/autu-mn/MotionWeave.
Related papers
- Evo-World: A Lightweight Vision-Language-Action Model with Future Motion Prediction via Optical-Flow Supervision
- Beyond Actions: Learning Future Operation Targets for Vision-Language-Action Models
- {\Delta}WAM: Distilling Action Tangent Fields into World Action Models
- ViDAL: A Visual Dynamics-Grounded Action Latent Space for Vision-Language-Action Models
- Forward Dynamics Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-09-30 · Indexed by Pub-AI: 2026-10-01 UTC
MotionWeave: Learning Motion-Centered Future Dynamics for Vision-Language-Action Policies
Preprint posted: 2026-09-30 · Original record ↗
- arxiv
- 2609.39324
Explicit source versions (1)
- v1 · 2026-09-30
- Preprint available
Source date: 2026-09-30 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Vision-Language-Action (VLA) models have recently incorporated world models to provide richer dynamic supervision beyond sparse action labels. However, explicitly predicting future images or videos may include control-irrelevant appearance, while guidance derived from holistic future visual representations and shared global action features may fail to establish timestep-specific correspondence between actions and local visual changes. To address this issue, we propose MotionWeave, a motion-centric future-dynamics framework for action-chunk prediction with two modules: the Action-Induced Motion Grounder (AIMG) and the Horizon Residual Composer (HRC). Specifically, AIMG conditions on action and proprioceptive representations to construct horizon-specific queries that localize interaction regions associated with each future action timestep from current visual tokens. HRC extracts differences between interaction representations at adjacent horizons, encodes them as temporal motion cues, and injects them into action tokens through a gated residual. During training, robot-arm masks rendered from future frames are used to construct KL-based motion-grounding supervision, while inference uses only the current observation. On six MetaWorld tasks, MotionWeave achieves a 75.3% average success rate, an absolute gain of 8.6% over π0 (66.7%), especially on sustained-interaction tasks. Our code is available at https://github.com/autu-mn/MotionWeave.