Video World Models · 2026-10-20

Self-Improving Video-to-Action Models

Hyunwoo Ryu, Eric Ming Chen, Max Wilcoxson, Vincent Sitzmann

arXiv:2610.24177PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

Mirror is a self-improving video-to-action (V2A) method that learns to execute video plans through environment interaction, without prior action data or hand-crafted rewards. It alternates planner-guided rollouts (with exploration noise) and V2A training on the resulting video-action pairs, iteratively following longer portions of the plan.

Why it matters

The authors aim to remove the scalability bottleneck of inverse dynamics training that typically requires large, human video-action datasets. Mirror’s results claim strong performance across Minecraft and decision-making benchmarks, including self-improvement without rewards or expert action labels, and a new offline state of the art for diamond acquisition.

Method

  • Uses an expert video planner to generate video plans conditioned on current observations; a video-to-action model predicts actions to match achieved transitions rather than planned ones.
  • Self-improves by alternating environment interaction (with stochastic V2A exploration noise and re-planning) and training V2A on collected video-action pairs.
  • Makes the learned video planner practical for online closed-loop control using a block-causal design with causal KV-caching and task-aware guidance.

Limitation

The authors note that for the diamond task they apply video post-processing due to the Wan2.2 VAE losing small, high-precision details (e.g., text and cursor), and that with oracle VP (which does not go through the VAE) the agent learns without this post-processing.

Abstract (from arXiv)

Video generative models are an emerging tool for embodied agents: they can serve as scalable visual planners, generating videos that depict how to complete previously unseen tasks. However, executing these plans typically requires a video-to-action or "inverse dynamics" model trained on large, human-collected video-action datasets, severely limiting scalability. We introduce Mirror, a self-supervised approach that learns video-to-action mappings through environment interactions, without prior action data or hand-crafted rewards. Mirror learns by attempting to follow a visual plan by executing its best guess of the underlying actions in the environment. Mirror then trains on the resulting video-action pairs, incrementally self-improving and following the visual planner more closely. Across Atari, OGBench, and DeepMind Control, Mirror outperforms leading learning-from-observation methods with learned video plans, and achieves near-expert performance with oracle plans. On Minecraft BASALT, it outperforms methods with access to expert actions or hand-engineered task knowledge. In Minecraft survival, it is the first method to acquire an iron pickaxe through self-improvement alone, without rewards or expert action labels. In the offline setting, with expert action labels but no environment interaction, it sets a new state of the art in obtaining diamonds. Our results suggest that advances in video models can translate into increasingly capable agents without a corresponding need for hand-crafted rewards or large collections of expert action data.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-16 · Indexed by Pub-AI: 2026-10-20 UTC

preprint

Self-Improving Video-to-Action Models

Hyunwoo Ryu, Eric Ming Chen, Max Wilcoxson, Vincent Sitzmann

Preprint posted: 2026-10-16 · Original record ↗

arxiv
2610.24177
Explicit source versions (1)
  • v1 · 2026-10-16
  1. Preprint available

    Source date: 2026-10-16 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

Video generative models are an emerging tool for embodied agents: they can serve as scalable visual planners, generating videos that depict how to complete previously unseen tasks. However, executing these plans typically requires a video-to-action or "inverse dynamics" model trained on large, human-collected video-action datasets, severely limiting scalability. We introduce Mirror, a self-supervised approach that learns video-to-action mappings through environment interactions, without prior action data or hand-crafted rewards. Mirror learns by attempting to follow a visual plan by executing its best guess of the underlying actions in the environment. Mirror then trains on the resulting video-action pairs, incrementally self-improving and following the visual planner more closely. Across Atari, OGBench, and DeepMind Control, Mirror outperforms leading learning-from-observation methods with learned video plans, and achieves near-expert performance with oracle plans. On Minecraft BASALT, it outperforms methods with access to expert actions or hand-engineered task knowledge. In Minecraft survival, it is the first method to acquire an iron pickaxe through self-improvement alone, without rewards or expert action labels. In the offline setting, with expert action labels but no environment interaction, it sets a new state of the art in obtaining diamonds. Our results suggest that advances in video models can translate into increasingly capable agents without a corresponding need for hand-crafted rewards or large collections of expert action data.