Video World Models · 2026-11-13
WorldDraft: If Code Builds the World, Can Video Models Just Render It?
Zhiyu Hou, Wen-Jie Shu, Harold Haodong Chen, Weilin Ruan, Chenfei Liao, Liuzhou Zhang, Disen Lan, Xiaogang Xu, Qi Chen, Ser-Nam Lim
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
WorldDraft studies whether video models can act as draft-to-video generative renderers when 4D world dynamics and camera paths are provided by executable code. It introduces a benchmark (432 strictly paired cases across 48 4D world families) to jointly measure world preservation and visual realization over varying draft granularity and appearance alignment. The authors propose WorldDraft-Primer to prompt frozen video renderers for better adherence.
Why it matters
The authors argue that relying on text-to-video alone often trades visual quality for physical/spatiotemporal reliability. By decoupling simulation drafts from rendering, WorldDraft pinpoints where rendering fails: preserving prescribed kinematics while binding target attributes persistently over time (and separating from conflicting source cues). WorldDraft-Primer then offers a test-time way to improve both objectives across frozen models.
Method
- Introduce WorldDraft: 432 strictly paired draft-to-video test cases spanning 48 executable 4D world families in six challenge categories, with a 3×3 factorial design over draft granularity and appearance alignment.
- Evaluate renderers using three disentangled dimensions: world preservation (entity/layout, kinematics & events, camera), rendering fidelity (attribute QA), and appearance persistence (correctness-gated persistence).
- Propose WorldDraft-Primer: plug-and-play first-frame visual prompting using object emphasis (corner brackets) and attribute cueing (localized color cues), leaving remaining frames untouched.
Abstract (from arXiv)
Multimodal coding agents decouple 4D world construction from visual generation, repurposing video models from world generators toward generative renderers. This division of labor raises a fundamental question: if code builds the world, can video models just render it? A reliable generative renderer hinges on two complementary criteria: (i) world preservation, retaining prescribed 4D kinematics and camera paths; and (ii) visual realization, persistently binding text-specified target attributes over time. To evaluate this capability, we introduce WorldDraft, a benchmark of $432$ strictly paired test cases across $48$ executable 4D world families spanning six dynamic challenge categories. Under a $3 \times 3$ factorial design, WorldDraft orthogonally crosses draft granularity (abstract proxy to previs) and appearance alignment (neutral to conflicting) under identical dynamics. Extensive evaluations across $12$ representative models reveal that: (I) world preservation and visual realization remain in stark operational tension; (II) richer draft scaffolding anchors structure but paradoxically impairs appearance fidelity; and (III) conflicting source cues trigger severe source leakage (up to $64\%$), exposing fragile cross-modal disentanglement. To mitigate these bottlenecks, we propose WorldDraft-Primer, a plug-and-play visual prompting strategy that consistently improves both objectives across frozen video renderers. Together, WorldDraft and WorldDraft-Primer lay a foundation for bridging programmatic 4D simulation with video generation, advancing physically grounded generative rendering.
Related papers
- Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos
- WorldFoundry: Towards Unified Infrastructure for World Intelligence
- WorldProbe: When Video World Models Fall Apart under Long-Horizon Spatial Exploration
- WorldCast: Exploring Intelligent Visual Generation in World State Prediction
- BoxWorld: Let World Engines Control and Video Generators Render
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-11-12 · Indexed by Pub-AI: 2026-11-13 UTC
WorldDraft: If Code Builds the World, Can Video Models Just Render It?
Preprint posted: 2026-11-12 · Original record ↗
- arxiv
- 2611.16217
Explicit source versions (1)
- v1 · 2026-11-12
- Preprint available
Source date: 2026-11-12 · First detected: 2026-11-13 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.16217v1; arxiv:announce_type=new; item pubDate (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Multimodal coding agents decouple 4D world construction from visual generation, repurposing video models from world generators toward generative renderers. This division of labor raises a fundamental question: if code builds the world, can video models just render it? A reliable generative renderer hinges on two complementary criteria: (i) world preservation, retaining prescribed 4D kinematics and camera paths; and (ii) visual realization, persistently binding text-specified target attributes over time. To evaluate this capability, we introduce WorldDraft, a benchmark of $432$ strictly paired test cases across $48$ executable 4D world families spanning six dynamic challenge categories. Under a $3 \times 3$ factorial design, WorldDraft orthogonally crosses draft granularity (abstract proxy to previs) and appearance alignment (neutral to conflicting) under identical dynamics. Extensive evaluations across $12$ representative models reveal that: (I) world preservation and visual realization remain in stark operational tension; (II) richer draft scaffolding anchors structure but paradoxically impairs appearance fidelity; and (III) conflicting source cues trigger severe source leakage (up to $64\%$), exposing fragile cross-modal disentanglement. To mitigate these bottlenecks, we propose WorldDraft-Primer, a plug-and-play visual prompting strategy that consistently improves both objectives across frozen video renderers. Together, WorldDraft and WorldDraft-Primer lay a foundation for bridging programmatic 4D simulation with video generation, advancing physically grounded generative rendering.
