Video World Models · 2026-10-16
CoWorld Reasoner: Agentic Multimodal Reasoning with the Evolving Video World Model-Driven Harness
Dongjie Cheng, Ruifeng Yuan, Ran Wei, Ning Song, Yongqi Li, Wenjie Li
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
CoWorld Reasoner is an agentic multimodal reasoning framework that uses a Video World Model (VWM) as a unified engine for visual imagination. A Video World Model-Driven Harness exposes view change, interpolation, continuation, and intervention through tools/skills and guides a VLM to decide when and how to imagine and interpret generated video. The authors add VLM-VWM Harness Co-Evolution that refines tools/skills/execution code via feedback while keeping model weights fixed. It improves over direct VLM reasoning on seven benchmarks.
Why it matters
The paper targets multimodal spatiotemporal reasoning tasks where the needed evidence may be unobserved and requires different kinds of visual imagination. By coupling a VWM’s operations to a tool/skill-based harness and co-evolving the harness using execution feedback, the authors aim to generate task-relevant visual evidence and interpret it for improved benchmark accuracy.
Method
- Use a Video World Model-Driven Harness with VWM-driven tools and skills to guide a VLM in deciding whether to imagine, selecting a visual operation, constructing a generation request, and interpreting the generated video.
- Implement four-stage harness-guided execution (Capability Boundary Assessment, Request VWM Assistance, Video Interpretation, Evidence Integration and Reasoning) using a verifier and an execution layer.
- Apply VLM-VWM Harness Co-Evolution: a supervisor revises the harness interface (tools, skills, execution code) based on execution feedback while keeping VLM and VWM weights fixed, rerunning stages from the same interaction histories.
Abstract (from arXiv)
As Vision-Language Models (VLMs) advance, they are increasingly expected to perform spatiotemporal reasoning over motion, actions, and view changes beyond what the input images show. Video World Models (VWMs) provide a natural unified engine for visual imagination by generating task-relevant visual content as additional evidence. However, existing approaches typically invoke a VWM through fixed pipelines centered on a predefined form of visual imagination, such as novel-view synthesis. To support flexible visual imagination for diverse reasoning tasks, we introduce CoWorld Reasoner, an agentic framework that uses a VWM as a unified visual reasoning engine. Its Video World Model-Driven Harness exposes multiple visual operations through tools and uses skills to guide the VLM in deciding whether to imagine, selecting an operation, constructing a generation request, and interpreting the resulting visual evidence. We further introduce VLM-VWM Harness Co-Evolution, which jointly revises tools, skills, and execution code based on execution feedback while keeping both model weights fixed. Across seven benchmarks spanning spatial intelligence, temporal and causal reasoning, and perceptual reasoning, CoWorld Reasoner outperforms direct reasoning with the same VLM. The strongest variant improves on all seven benchmarks, including relative accuracy gains of 11.8% on MMSI and 13.0% on ERQA, and outperforms all five evolving baselines using the same backbone across five benchmarks.
Related papers
- WOVEN: Weaving Visual World Modeling into Multimodal LLMs
- AgentVQA: A Unified Benchmark for Agentic Visual Understanding
- Event-Aligned Visual Action Reasoning for World Action Models
- LAW: Unifying Language, Action, and World Modeling for Agentic Physical Intelligence
- WorldCast: Exploring Intelligent Visual Generation in World State Prediction
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-15 · Indexed by Pub-AI: 2026-10-16 UTC
CoWorld Reasoner: Agentic Multimodal Reasoning with the Evolving Video World Model-Driven Harness
Preprint posted: 2026-10-15 · Original record ↗
- arxiv
- 2610.21910
Explicit source versions (1)
- v1 · 2026-10-15
- Preprint available
Source date: 2026-10-15 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
As Vision-Language Models (VLMs) advance, they are increasingly expected to perform spatiotemporal reasoning over motion, actions, and view changes beyond what the input images show. Video World Models (VWMs) provide a natural unified engine for visual imagination by generating task-relevant visual content as additional evidence. However, existing approaches typically invoke a VWM through fixed pipelines centered on a predefined form of visual imagination, such as novel-view synthesis. To support flexible visual imagination for diverse reasoning tasks, we introduce CoWorld Reasoner, an agentic framework that uses a VWM as a unified visual reasoning engine. Its Video World Model-Driven Harness exposes multiple visual operations through tools and uses skills to guide the VLM in deciding whether to imagine, selecting an operation, constructing a generation request, and interpreting the resulting visual evidence. We further introduce VLM-VWM Harness Co-Evolution, which jointly revises tools, skills, and execution code based on execution feedback while keeping both model weights fixed. Across seven benchmarks spanning spatial intelligence, temporal and causal reasoning, and perceptual reasoning, CoWorld Reasoner outperforms direct reasoning with the same VLM. The strongest variant improves on all seven benchmarks, including relative accuracy gains of 11.8% on MMSI and 13.0% on ERQA, and outperforms all five evolving baselines using the same backbone across five benchmarks.