Video World Models · 2026-11-10
WanderBench: Can World Models Explore Dynamic Worlds from Video?
Yuhang Chen, Jiheng Li, Jiale Wang, Zai Wang, Guangyue Liu
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors introduce WanderBench, a benchmark for exploring dynamic worlds from a recorded source video with camera routes given as keyboard actions. Using 745 single-shot clips and 24 sub-metrics over six dimensions, they report that longer horizons degrade camera control and source-dynamics fidelity and that no model keeps revisited places consistent. They propose WanderLand, a training-free conditioning framework, reporting gains up to +8.3 points in source-dynamics fidelity.
Why it matters
Video-conditioned exploration requires outputs to preserve the recorded event progression and scene consistency when returning to a location, not just visual realism or camera following. WanderBench provides time-synchronized, event-focused evaluation via a source-video reference, enabling the authors to identify specific long-horizon failure modes (lost camera control/source dynamics and inconsistent revisits) and to show that evidence-ordered conditioning can improve those targeted dimensions.
Method
- WanderBench pairs 745 single-shot source videos (real and AI-generated) with camera routes specified by discrete keyboard actions, evaluates 24 sub-metrics across six dimensions using time-synchronized source references.
- The evaluation uses per-frame source matching: each output frame is compared to the source frame depicting the same instant, with scores rescaled to [0,100].
- WanderLand is a training-free enhancement that wraps a frozen video world model and rebuilds camera/source render evidence in a fixed order; on revisits it fills source-uncovered regions using the model’s earlier static content.
Limitation
The authors report that no model keeps revisited places consistent, and that models limited to a few seconds of generation cannot even reach the longer tiers.
Abstract (from arXiv)
Video world models conditioned on an input video make it possible to explore real, recorded dynamic scenes, enabling re-shooting of footage, free-viewpoint replay, and real-to-sim environments for embodied agents. Such uses require the generated rollout to stay faithful to the recorded world, yet existing benchmarks start from a single image or a text prompt, and video-conditioned generation is evaluated mainly on perceptual quality and camera following. These evaluations do not directly assess whether exploration preserves subject identity, event progression, and scene consistency upon revisiting a location. We introduce WanderBench, a benchmark for exploring dynamic worlds from video. It comprises 745 single-shot source videos of real and AI-generated content in three difficulty tiers (3-25 s), paired with camera routes given as keyboard actions, and evaluates six dimensions with 24 human-validated sub-metrics. Benchmarking eight models, we find that as clips grow longer, models keep the static scene and visual quality but lose camera control and the source dynamics, and that none keeps revisited places consistent. We further propose WanderLand, a training-free framework that conditions each frame of a frozen model on the same-instant source and, on revisits, on its own earlier static content. WanderLand improves five models on every dimension, by up to 8.3 points in source-dynamics fidelity.
Related papers
- WorldProbe: When Video World Models Fall Apart under Long-Horizon Spatial Exploration
- Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos
- StateCraft: Augmenting Video World Models with Abstract-State Reasoning
- WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models
- AgiWorld: An Agentic Video World Model for Building Persistent Worlds
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-11-09 · Indexed by Pub-AI: 2026-11-10 UTC
WanderBench: Can World Models Explore Dynamic Worlds from Video?
Preprint posted: 2026-11-09 · Original record ↗
- arxiv
- 2611.12557
Explicit source versions (1)
- v1 · 2026-11-09
- Preprint available
Source date: 2026-11-09 · First detected: 2026-11-10 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.12557v1; arxiv:announce_type=new; item pubDate (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Video world models conditioned on an input video make it possible to explore real, recorded dynamic scenes, enabling re-shooting of footage, free-viewpoint replay, and real-to-sim environments for embodied agents. Such uses require the generated rollout to stay faithful to the recorded world, yet existing benchmarks start from a single image or a text prompt, and video-conditioned generation is evaluated mainly on perceptual quality and camera following. These evaluations do not directly assess whether exploration preserves subject identity, event progression, and scene consistency upon revisiting a location. We introduce WanderBench, a benchmark for exploring dynamic worlds from video. It comprises 745 single-shot source videos of real and AI-generated content in three difficulty tiers (3-25 s), paired with camera routes given as keyboard actions, and evaluates six dimensions with 24 human-validated sub-metrics. Benchmarking eight models, we find that as clips grow longer, models keep the static scene and visual quality but lose camera control and the source dynamics, and that none keeps revisited places consistent. We further propose WanderLand, a training-free framework that conditions each frame of a frozen model on the same-instant source and, on revisits, on its own earlier static content. WanderLand improves five models on every dimension, by up to 8.3 points in source-dynamics fidelity.
