Driving · 2026-10-12
Beyond Actions: Learning Future Operation Targets for Vision-Language-Action Models
Min Lin, Bokai Zhou, Jianqi Lin, Kaiyuan Yang, Yicheng Chang, Junjie Yu, Xiaofan Liu, Jianhua Han, Rui Li, Shen Zhao, Xiaodan Liang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors propose Action–Future Coupled Effect Learning (AFCE) for vision-language-action models: instead of predicting action chunks directly, the policy predicts temporally ordered physical “Effect” tokens grounded in demonstrated executed actions, realized robot responses, and sparse visual transitions. A flow-matching model predicts Effects from current image+instruction+state, then a state-conditioned decoder outputs action chunks.
Why it matters
Action supervision alone does not specify what commands achieve. AFCE replaces the intermediate target with an action-decodable, future-grounded representation tied to realized robot change and world transition, aiming to improve dexterous manipulation—especially coordinated bimanual tasks—under an interface that uses future evidence only for training.
Method
- Offline tokenizer constructs temporally ordered Effect tokens from synchronized demonstrated actions, realized robot-response sequences, and sparse visual transition evidence.
- Stage I trains the Effect tokenizer with reconstruction objectives (action plus auxiliary state and visual losses) and freezes it to cache normalized Effect targets.
- Stage II uses a conditional flow-matching VLA to predict normalized Effects from current images, instruction, and robot state; a state-conditioned decoder converts predicted Effects to action chunks for closed-loop execution.
Limitation
The authors report that while AFCE improves aggregate success, “some fine-control tasks remain challenging,” including precision-related single-arm tasks where overall gain “does not extend uniformly to fine-grained single-arm control.”
Abstract (from arXiv)
Actions specify what a robot should execute, but do not describe what those commands will achieve. Vision-language-action (VLA) policies nevertheless learn primarily by predicting action chunks from images, language, and state. We introduce Action--Future Coupled Effect Learning (AFCE), which makes a future-grounded physical Effect the policy's primary prediction target. An offline tokenizer constructs temporally ordered Effect tokens from demonstrated actions, realized robot responses, and sparse visual transitions. Reconstruction objectives encourage the representation to retain executable detail and observed physical change. A flow-matching VLA predicts Effects from current images, instruction, and robot state, while a state-conditioned decoder converts them into action chunks. Embodiment-specific interfaces allow single- and dual-arm tasks to share the Effect model. On DexJoCo, AFCE achieves $50.97\%$ overall success, improving over $\pi_{0.5}$ and UVT by 11.82 and 15.21 percentage points, respectively. It leads on all five bimanual tasks, highlighting the value of Effect supervision for coordinated dexterous manipulation. The project page is available at https://afceproject.github.io/AFCE-Web/.
Related papers
- Vision-Language-Delta-Action: Learning Compact Deltas as Actionable Transition Interfaces for Robotic Manipulation
- ViDAL: A Visual Dynamics-Grounded Action Latent Space for Vision-Language-Action Models
- WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction
- MotionWeave: Learning Motion-Centered Future Dynamics for Vision-Language-Action Policies
- What Should Robot Policies Learn to Predict? Rethinking Supervision Targets in Vision-Language-Action Learning
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-09 · Indexed by Pub-AI: 2026-10-12 UTC
Beyond Actions: Learning Future Operation Targets for Vision-Language-Action Models
Preprint posted: 2026-10-09 · Original record ↗
- arxiv
- 2610.13212
Explicit source versions (1)
- v2 · 2026-10-12
- Preprint available
Source date: 2026-10-09 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
- Explicit source version detected · v2
Source date: 2026-10-12 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
Abstract · arxiv
Actions specify what a robot should execute, but do not describe what those commands will achieve. Vision-language-action (VLA) policies nevertheless learn primarily by predicting action chunks from images, language, and state. We introduce Action--Future Coupled Effect Learning (AFCE), which makes a future-grounded physical Effect the policy's primary prediction target. An offline tokenizer constructs temporally ordered Effect tokens from demonstrated actions, realized robot responses, and sparse visual transitions. Reconstruction objectives encourage the representation to retain executable detail and observed physical change. A flow-matching VLA predicts Effects from current images, instruction, and robot state, while a state-conditioned decoder converts them into action chunks. Embodiment-specific interfaces allow single- and dual-arm tasks to share the Effect model. On DexJoCo, AFCE achieves $50.97\%$ overall success, improving over $π_{0.5}$ and UVT by 11.82 and 15.21 percentage points, respectively. It leads on all five bimanual tasks, highlighting the value of Effect supervision for coordinated dexterous manipulation. The project page is available at https://afceproject.github.io/AFCE-Web/.