Driving · 2026-11-04
Cycle WAM: Coupling Action and Video through Cycle Consistency
Qinglin Yang, Jiepeng Wang, Yuxing Huang, Haibin Huang, Kai Xu, Chi Zhang, Ruizhen Hu, Xuelong Li
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
Cycle WAM is a unified training framework for world action models that couples video and actions using bidirectional conversion and cycle-consistency. Starting from video–action co-training, it trains video→action (inverse dynamics) and action→video (forward dynamics), then adds a differentiable cycle-reconstruction objective to better align actions with their visual consequences and exploit pretrained video priors. The authors report improved control and dynamics on LIBERO-Pro and DROID.
Why it matters
Video priors can help action-conditioned world models, but existing training may not ensure consistency between predicted actions and the visual changes they cause. Cycle WAM targets this correspondence directly via bidirectional conversion plus cycle reconstruction, supporting forward dynamics, inverse dynamics, and policy learning in a single model. The authors evaluate improvements on LIBERO-Pro control and DreamZero on DROID, and provide forward/inverse dynamics comparisons with UWM.
Method
- Video–action co-training: jointly predicts future video and robot actions from context using a shared WAM with native flow-matching losses for video and action.
- Bidirectional Conversion: trains video→action and action→video mappings on paired data to recover inverse and forward dynamics while maintaining joint generation.
- Cycle Reconstruction: converts video→action→video (and back) with reconstruction loss and gradients through the converted action to couple action prediction with visual consequences.
Limitation
The authors state that unpaired ablations and incomplete training-budget records limit Cycle attribution on DROID.
Abstract (from arXiv)
Recent advances in video modeling have driven substantial progress in world action models (WAMs). Existing approaches exploit pretrained video priors by decoding actions from latent representations or inferring them from generated videos. The underlying link is causal: robot actions produce changes in the scene, and video observations of these changes provide evidence for inferring the actions that produced them. However, existing training formulations do not by themselves ensure that actions and their predicted visual consequences are consistent, potentially limiting the use of pretrained capabilities. We treat video and action as paired modalities linked by this causal relationship and learn their correspondence through a unified training framework designed to better exploit pretrained video priors. Cycle WAM first augments video-action co-training with bidirectional conversion to reinforce cross-modal correspondence and recover forward and inverse dynamics. It then introduces a differentiable cycle-reconstruction objective to further couple the two mappings. The resulting model supports both forward and inverse dynamics, with the potential to improve performance through more effective use of pretrained video priors. Cycle WAM achieves state-of-the-art performance among comparable methods on LIBERO-Pro and substantially improves DreamZero on DROID. Comparisons with UWM characterize its inverse-dynamics performance and show sharper visual detail in the displayed forward-dynamics predictions.
Related papers
- OpenWAM: An Open Framework for Composable World-Action Models
- Native Action-Prior Learning from Videos for World Action Models
- EVO-WAM: Evolving World Action Models through Video-Action Verification
- WorldBridge: Bridging Video and Action for Task Generalization in World Action Models
- {\Delta}WAM: Distilling Action Tangent Fields into World Action Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-11-04 · Indexed by Pub-AI: 2026-11-04 UTC
Cycle WAM: Coupling Action and Video through Cycle Consistency
First announced by arXiv: 2026-11-04 · Original record ↗
- arxiv
- 2611.04711
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-11-04 · First detected: 2026-11-04 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.04711v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
Recent advances in video modeling have driven substantial progress in world action models (WAMs). Existing approaches exploit pretrained video priors by decoding actions from latent representations or inferring them from generated videos. The underlying link is causal: robot actions produce changes in the scene, and video observations of these changes provide evidence for inferring the actions that produced them. However, existing training formulations do not by themselves ensure that actions and their predicted visual consequences are consistent, potentially limiting the use of pretrained capabilities. We treat video and action as paired modalities linked by this causal relationship and learn their correspondence through a unified training framework designed to better exploit pretrained video priors. Cycle WAM first augments video-action co-training with bidirectional conversion to reinforce cross-modal correspondence and recover forward and inverse dynamics. It then introduces a differentiable cycle-reconstruction objective to further couple the two mappings. The resulting model supports both forward and inverse dynamics, with the potential to improve performance through more effective use of pretrained video priors. Cycle WAM achieves state-of-the-art performance among comparable methods on LIBERO-Pro and substantially improves DreamZero on DROID. Comparisons with UWM characterize its inverse-dynamics performance and show sharper visual detail in the displayed forward-dynamics predictions.
