Video World Models · 2026-10-16
WorldCast: Exploring Intelligent Visual Generation in World State Prediction
Wenhao Yan, Haodong Li, Xiaotong Li, Yaxin Luo, Zhengrong Yue, Jialv Zou, Xu Huang, Haojin Yang, Nan Jia, Yan Feng, Peng Pei, Xunliang Cai
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
WorldCast is a benchmark for multi-step world state prediction from multimodal observations and diverse references, targeting L3–L4 capabilities. It evaluates 12 image/video generation models on 320 manually designed cases (1,352 requested states) using sequence-level keyframes and LMM-as-a-judge scoring (TC/CP/VC). The authors report GPT-Image-2.5 System scoring 72.8, while gaps remain large even for best open-source (20.1).
Why it matters
The authors argue that higher-level world state prediction—forecasting successive visual world states while reasoning about how references take effect—remains underexplored. WorldCast aims to quantify how well multimodal generative models externalize this L3–L4 intelligence, and to diagnose factors behind performance gaps (e.g., language-side enhancement, interleaved generation, and agentic/coding tools).
Method
- Define world state prediction as generating an ordered sequence of future visual world states from multimodal context and references; represent model outputs as ordered sequences of keyframes.
- Evaluate sequences on three dimensions: Task Completion (TC), Causal Plausibility (CP), and Visual Consistency (VC), aggregated into a 100-point score.
- Use an agentic video adapter (GPT-6 Astra via Codex) to extract task-aligned keyframes from video outputs and use GPT-5.5 as LMM-as-a-judge with human-annotated judge hints.
Limitation
Because WorldCast evaluates image outputs as ordered keyframes under a unified state-sequence protocol, the agentic video adapter is used only to determine whether the video contains required key states; it does not assess the plausibility of the full video sequence or its visual quality.
Abstract (from arXiv)
Humans interpret multimodal observations of the world and forecast how it will evolve. Recent multimodal generative systems have demonstrated similar capabilities: they can integrate multimodal context and synthesize visual outputs that depict changes in the world. To characterize this emerging trajectory, we organize model capabilities into a four-level hierarchy and discover that higher-level tasks remain relatively underexplored despite model progress. To bridge this gap, we explore model performance on multi-step world state prediction, a high-level task that requires models to forecast successive future world states from multimodal observations and diverse references. Such a task requires models to possess entangled understanding and generation capabilities, as well as extensive world knowledge to translate human action sequences, physical transformations, and abstract visual reasoning into coherent visual outputs. We introduce WorldCast, a benchmark comprising 320 manually designed cases across eight categories, and evaluate 12 representative models. The conducted experiments yield three key insights: (a) advances in frontier models are rapidly pushing the boundaries of intelligent world modeling, with the GPT-Image-2.5 System scoring 72.8 and demonstrating early-stage visual state prediction capabilities on real-world tasks; (b) WorldCast reveals substantial gaps between models: even Nano Banana Pro reaches only 42.8 and the best open-source model scores 20.1, with deficits in handling complex context, entangled references, and reasoning-heavy tasks; (c) strong generative models effectively draw on the capabilities of their underlying language models. These findings highlight the need for multimodal generative models that are capable of dynamic planning and reasoning while also being deeply unified to faithfully simulate real-world states.
Related papers
- Reading, Writing, Sharing, and Interacting with the World through Video
- StateCraft: Augmenting Video World Models with Abstract-State Reasoning
- WOVEN: Weaving Visual World Modeling into Multimodal LLMs
- PrismWAM: Decoupling State Representations in World Action Models
- What Matters for World Modeling in Vision-Language-Action Models?
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-14 · Indexed by Pub-AI: 2026-10-16 UTC
WorldCast: Exploring Intelligent Visual Generation in World State Prediction
Preprint posted: 2026-10-14 · Original record ↗
- arxiv
- 2610.20731
Explicit source versions (1)
- v2 · 2026-10-22
- Explicit source version detected · v2
Source date: 2026-10-22 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
- Preprint available
Source date: 2026-10-14 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Humans interpret multimodal observations of the world and forecast how it will evolve. Recent multimodal generative systems have demonstrated similar capabilities: they can integrate multimodal context and synthesize visual outputs that depict changes in the world. To characterize this emerging trajectory, we organize model capabilities into a four-level hierarchy and discover that higher-level tasks remain relatively underexplored despite model progress. To bridge this gap, we explore model performance on multi-step world state prediction, a high-level task that requires models to forecast successive future world states from multimodal observations and diverse references. Such a task requires models to possess entangled understanding and generation capabilities, as well as extensive world knowledge to translate human action sequences, physical transformations, and abstract visual reasoning into coherent visual outputs. We introduce WorldCast, a benchmark comprising 320 manually designed cases across eight categories, and evaluate 12 representative models. The conducted experiments yield three key insights: (a) advances in frontier models are rapidly pushing the boundaries of intelligent world modeling, with the GPT-Image-2.5 System scoring 72.8 and demonstrating early-stage visual state prediction capabilities on real-world tasks; (b) WorldCast reveals substantial gaps between models: even Nano Banana Pro reaches only 42.8 and the best open-source model scores 20.1, with deficits in handling complex context, entangled references, and reasoning-heavy tasks; (c) strong generative models effectively draw on the capabilities of their underlying language models. These findings highlight the need for multimodal generative models that are capable of dynamic planning and reasoning while also being deeply unified to faithfully simulate real-world states.