Driving · 2026-11-04

Cycle WAM: Coupling Action and Video through Cycle Consistency

Qinglin Yang, Jiepeng Wang, Yuxing Huang, Haibin Huang, Kai Xu, Chi Zhang, Ruizhen Hu, Xuelong Li

arXiv:2611.04711PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

Cycle WAM is a unified training framework for world action models that couples video and actions using bidirectional conversion and cycle-consistency. Starting from video–action co-training, it trains video→action (inverse dynamics) and action→video (forward dynamics), then adds a differentiable cycle-reconstruction objective to better align actions with their visual consequences and exploit pretrained video priors. The authors report improved control and dynamics on LIBERO-Pro and DROID.

Why it matters

Video priors can help action-conditioned world models, but existing training may not ensure consistency between predicted actions and the visual changes they cause. Cycle WAM targets this correspondence directly via bidirectional conversion plus cycle reconstruction, supporting forward dynamics, inverse dynamics, and policy learning in a single model. The authors evaluate improvements on LIBERO-Pro control and DreamZero on DROID, and provide forward/inverse dynamics comparisons with UWM.

Method

  • Video–action co-training: jointly predicts future video and robot actions from context using a shared WAM with native flow-matching losses for video and action.
  • Bidirectional Conversion: trains video→action and action→video mappings on paired data to recover inverse and forward dynamics while maintaining joint generation.
  • Cycle Reconstruction: converts video→action→video (and back) with reconstruction loss and gradients through the converted action to couple action prediction with visual consequences.

Limitation

The authors state that unpaired ablations and incomplete training-budget records limit Cycle attribution on DROID.

Abstract (from arXiv)

Recent advances in video modeling have driven substantial progress in world action models (WAMs). Existing approaches exploit pretrained video priors by decoding actions from latent representations or inferring them from generated videos. The underlying link is causal: robot actions produce changes in the scene, and video observations of these changes provide evidence for inferring the actions that produced them. However, existing training formulations do not by themselves ensure that actions and their predicted visual consequences are consistent, potentially limiting the use of pretrained capabilities. We treat video and action as paired modalities linked by this causal relationship and learn their correspondence through a unified training framework designed to better exploit pretrained video priors. Cycle WAM first augments video-action co-training with bidirectional conversion to reinforce cross-modal correspondence and recover forward and inverse dynamics. It then introduces a differentiable cycle-reconstruction objective to further couple the two mappings. The resulting model supports both forward and inverse dynamics, with the potential to improve performance through more effective use of pretrained video priors. Cycle WAM achieves state-of-the-art performance among comparable methods on LIBERO-Pro and substantially improves DreamZero on DROID. Comparisons with UWM characterize its inverse-dynamics performance and show sharper visual detail in the displayed forward-dynamics predictions.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-04 · Indexed by Pub-AI: 2026-11-04 UTC

preprint

Cycle WAM: Coupling Action and Video through Cycle Consistency

Qinglin Yang, Jiepeng Wang, Yuxing Huang, Haibin Huang, Kai Xu, Chi Zhang, Ruizhen Hu, Xuelong Li

First announced by arXiv: 2026-11-04 · Original record ↗

arxiv
2611.04711
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-04 · First detected: 2026-11-04 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2611.04711v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

Recent advances in video modeling have driven substantial progress in world action models (WAMs). Existing approaches exploit pretrained video priors by decoding actions from latent representations or inferring them from generated videos. The underlying link is causal: robot actions produce changes in the scene, and video observations of these changes provide evidence for inferring the actions that produced them. However, existing training formulations do not by themselves ensure that actions and their predicted visual consequences are consistent, potentially limiting the use of pretrained capabilities. We treat video and action as paired modalities linked by this causal relationship and learn their correspondence through a unified training framework designed to better exploit pretrained video priors. Cycle WAM first augments video-action co-training with bidirectional conversion to reinforce cross-modal correspondence and recover forward and inverse dynamics. It then introduces a differentiable cycle-reconstruction objective to further couple the two mappings. The resulting model supports both forward and inverse dynamics, with the potential to improve performance through more effective use of pretrained video priors. Cycle WAM achieves state-of-the-art performance among comparable methods on LIBERO-Pro and substantially improves DreamZero on DROID. Comparisons with UWM characterize its inverse-dynamics performance and show sharper visual detail in the displayed forward-dynamics predictions.