Video World Models · 2026-10-20
Self-Improving Video-to-Action Models
Hyunwoo Ryu, Eric Ming Chen, Max Wilcoxson, Vincent Sitzmann
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
Mirror is a self-improving video-to-action (V2A) method that learns to execute video plans through environment interaction, without prior action data or hand-crafted rewards. It alternates planner-guided rollouts (with exploration noise) and V2A training on the resulting video-action pairs, iteratively following longer portions of the plan.
Why it matters
The authors aim to remove the scalability bottleneck of inverse dynamics training that typically requires large, human video-action datasets. Mirror’s results claim strong performance across Minecraft and decision-making benchmarks, including self-improvement without rewards or expert action labels, and a new offline state of the art for diamond acquisition.
Method
- Uses an expert video planner to generate video plans conditioned on current observations; a video-to-action model predicts actions to match achieved transitions rather than planned ones.
- Self-improves by alternating environment interaction (with stochastic V2A exploration noise and re-planning) and training V2A on collected video-action pairs.
- Makes the learned video planner practical for online closed-loop control using a block-causal design with causal KV-caching and task-aware guidance.
Limitation
The authors note that for the diamond task they apply video post-processing due to the Wan2.2 VAE losing small, high-precision details (e.g., text and cursor), and that with oracle VP (which does not go through the VAE) the agent learns without this post-processing.
Abstract (from arXiv)
Video generative models are an emerging tool for embodied agents: they can serve as scalable visual planners, generating videos that depict how to complete previously unseen tasks. However, executing these plans typically requires a video-to-action or "inverse dynamics" model trained on large, human-collected video-action datasets, severely limiting scalability. We introduce Mirror, a self-supervised approach that learns video-to-action mappings through environment interactions, without prior action data or hand-crafted rewards. Mirror learns by attempting to follow a visual plan by executing its best guess of the underlying actions in the environment. Mirror then trains on the resulting video-action pairs, incrementally self-improving and following the visual planner more closely. Across Atari, OGBench, and DeepMind Control, Mirror outperforms leading learning-from-observation methods with learned video plans, and achieves near-expert performance with oracle plans. On Minecraft BASALT, it outperforms methods with access to expert actions or hand-engineered task knowledge. In Minecraft survival, it is the first method to acquire an iron pickaxe through self-improvement alone, without rewards or expert action labels. In the offline setting, with expert action labels but no environment interaction, it sets a new state of the art in obtaining diamonds. Our results suggest that advances in video models can translate into increasingly capable agents without a corresponding need for hand-crafted rewards or large collections of expert action data.
Related papers
- PAWA: Panoramic Action-conditioned World Modeling for Multiple Agents
- EmbodiedWorld-200K: Toward Embodied World Agents via Embodied-Planning Dataset and Interactive World Models
- Video Generation Meets Agent Harness: Agentic Video Generation, Agentic World Modeling and Recursive Self-Improvement
- AdaRSI: On-Demand Recursive Self-Improvement for Embodied Agents
- StateCraft: Augmenting Video World Models with Abstract-State Reasoning
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-16 · Indexed by Pub-AI: 2026-10-20 UTC
Self-Improving Video-to-Action Models
Preprint posted: 2026-10-16 · Original record ↗
- arxiv
- 2610.24177
Explicit source versions (1)
- v1 · 2026-10-16
- Preprint available
Source date: 2026-10-16 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Video generative models are an emerging tool for embodied agents: they can serve as scalable visual planners, generating videos that depict how to complete previously unseen tasks. However, executing these plans typically requires a video-to-action or "inverse dynamics" model trained on large, human-collected video-action datasets, severely limiting scalability. We introduce Mirror, a self-supervised approach that learns video-to-action mappings through environment interactions, without prior action data or hand-crafted rewards. Mirror learns by attempting to follow a visual plan by executing its best guess of the underlying actions in the environment. Mirror then trains on the resulting video-action pairs, incrementally self-improving and following the visual planner more closely. Across Atari, OGBench, and DeepMind Control, Mirror outperforms leading learning-from-observation methods with learned video plans, and achieves near-expert performance with oracle plans. On Minecraft BASALT, it outperforms methods with access to expert actions or hand-engineered task knowledge. In Minecraft survival, it is the first method to acquire an iron pickaxe through self-improvement alone, without rewards or expert action labels. In the offline setting, with expert action labels but no environment interaction, it sets a new state of the art in obtaining diamonds. Our results suggest that advances in video models can translate into increasingly capable agents without a corresponding need for hand-crafted rewards or large collections of expert action data.
