Latent Dynamics · 2026-10-08
Event-Aligned Visual Action Reasoning for World Action Models
Xiaomeng Yang, Yushu Wu, Yi Gao, Yuhao Lei, Xuan Zhang, Pu Zhao, Yanzhi Wang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors introduce an event-aligned visual action reasoning framework for World-Action Models (WAMs). Instead of using fixed temporal horizons, they build visual-action chunks from interaction events and their evidence/support phases. They also add an execution validity head to skip padded/repeated action slots during chunked inference. On DOMINO, they report a 10.26 percentage-point DOMINO success-rate gain over LingBot-VA.
Why it matters
WAMs use future visual prediction as intermediate reasoning for action generation, but prior methods structure visual imagination on uniform temporal intervals. By aligning imagined visual rollouts to task-critical interaction events (with finer granularity for evidence phases), the method aims to improve action-relevant foresight and reduce drift. The execution validity head addresses redundant/padded actions caused by fixed-length chunking, aiming for smoother action execution and better manipulation performance.
Method
- Event-aligned visual-action supervision: segment demonstrations into interaction vs transition events; within interactions, protect evidence-phase resolution and compress/resample support-phase frames to form visual-action chunks that do not cross event boundaries.
- Execution validity head: during training/inference, predict per-frame validity for each generated action chunk position and execute only the action groups whose validity probability exceeds a threshold, avoiding stall/padded actions.
- Train an autoregressive latent-diffusion WAM on the event-aligned chunks using flow-matching losses for visual (dyn) and action (inv) streams, plus a weighted validity loss.
Limitation
The cross-level transfer to DOMINO Level 3 shows low absolute success rate, indicating that task completion under those dynamics remains challenging.
Abstract (from arXiv)
World-Action Models (WAMs) utilize future visual prediction as an intermediate reasoning process to guide action generation. However, existing WAMs typically structure visual imagination according to predefined temporal intervals, without explicitly accounting for the different roles of task-critical interactions and connecting transitions. We argue that effective visual foresight should align directly with task-relevant interactions and their corresponding reasoning demands. To this end, we introduce an event-aligned visual action reasoning framework that organizes visual-action prediction around interaction events. Through event-aligned visual-action supervision, WAM learns to generate event-aligned visual context in each imagined rollout, placing greater emphasis on critical state changes that inform action generation. This shapes the visual reasoning granularity according to the underlying interaction dynamics, with detailed reasoning around task-critical events and coarser progression through connecting transitions. Furthermore, we introduce an execution validity head that identifies the valid portion of each predicted action sequence, avoiding redundant actions during chunked inference. Experiments demonstrate a 10.26 percentage point improvement in DOMINO success rate over baseline and competitive performance on RoboTwin 2.0. It also transfers from DOMINO Level 1 to Levels 2 and 3 without target-level adaptation.
Related papers
- Achieve What You Imagined: Learning to Align Actions with Visual Plans
- Rethinking World-Action Model for Compositional and In-Context Robotic Manipulation
- RM-WAM: Reliable Memory-Augmented World Action Models for Long-Horizon Robot Manipulation
- Magic-W0: A Structured World-Action Foundation Model for Physical Intelligence
- Agentic World Action Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-07 · Indexed by Pub-AI: 2026-10-08 UTC
Event-Aligned Visual Action Reasoning for World Action Models
Preprint posted: 2026-10-07 · Original record ↗
- arxiv
- 2610.09427
Explicit source versions (1)
- v1 · 2026-10-07
- Preprint available
Source date: 2026-10-07 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World-Action Models (WAMs) utilize future visual prediction as an intermediate reasoning process to guide action generation. However, existing WAMs typically structure visual imagination according to predefined temporal intervals, without explicitly accounting for the different roles of task-critical interactions and connecting transitions. We argue that effective visual foresight should align directly with task-relevant interactions and their corresponding reasoning demands. To this end, we introduce an event-aligned visual action reasoning framework that organizes visual-action prediction around interaction events. Through event-aligned visual-action supervision, WAM learns to generate event-aligned visual context in each imagined rollout, placing greater emphasis on critical state changes that inform action generation. This shapes the visual reasoning granularity according to the underlying interaction dynamics, with detailed reasoning around task-critical events and coarser progression through connecting transitions. Furthermore, we introduce an execution validity head that identifies the valid portion of each predicted action sequence, avoiding redundant actions during chunked inference. Experiments demonstrate a 10.26 percentage point improvement in DOMINO success rate over baseline and competitive performance on RoboTwin 2.0. It also transfers from DOMINO Level 1 to Levels 2 and 3 without target-level adaptation.