Video World Models · 2026-10-02
ROWBench: Do Video Models Render What the Program Specifies?
Zheng-Hui Huang, Guixu Lin, Yu-Ju Tsai, Jian-Kai Zhu, Fengbo Lan, Yu-Lun Liu, Yung-Yu Chuang, Kaipeng Zhang, Zhixiang Wang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors introduce PROWBench, a benchmark for evaluating whether programmable video world models visually realize fine-grained, program-specified world events. PROWBench builds 170 programmatically constructed episodes (600 proxy videos) with logged entity states and timestamped engine events (including off-camera), and uses replayable world records to render synchronized views/proxies. It evaluates entity control, long-horizon memory, and introduces Logic-Render Alignment (timeline) and Interaction Success Rate (event execution).
Why it matters
The authors argue existing benchmarks rarely test fidelity to fine-grained, program-specified world events because they lack replayable entity-state and timestamped-event records. PROWBench’s engine-recorded world logs let generated videos be checked against observable consequences of program execution, aiming to measure visual realization of prescribed states, rules, and interactions rather than only plausibility.
Method
- Build PROWBench from a programmable world generation pipeline that records multi-entity states and timestamped interaction events (including off-camera), enabling independent replay and rendering into synchronized views/proxies.
- Evaluate entity control and long-horizon memory using engine-state references, and add VLM-based Logic-Render Alignment (timeline segment action visibility) and Interaction Success Rate (whether recorded events’ action and expected end state are judged present).
- Compare multiple video generation/world-model methods under different proxy/first-frame/camera conditions across main and challenge tracks, including multi-view consistency and long-horizon (30s) tests.
Limitation
The proxy itself matters: outputs sometimes follow the input geometry too closely and inherit coarse shapes of low-poly proxies, and designing a good proxy remains an open problem.
Abstract (from arXiv)
Programmable world models separate executable dynamics from visual generation, offering a promising foundation for next-generation game engines. However, their visual adherence to explicit rules and interactions remains insufficiently evaluated. Existing benchmarks assess visual quality, controllability, and instruction or physical adherence, but rarely test fidelity to fine-grained, program-specified world events. We introduce PROWBench, comprising 170 programmatically constructed episodes and 600 proxy videos covering diverse scenes and interactions. PROWBench logs entity states and timestamped events, including those outside the camera's field of view, as replayable world records, from which it renders synchronized views and proxy representations. This enables generated videos to be checked against the observable consequences of program execution. An extensible framework constructs scenes, controls behaviors, and can render each camera view in different representations, such as coarse 3D, and bounding boxes. The benchmark covers first- and third-person perspectives, with synchronized multi-view observations available for a subset of episodes. Grounded in these records, PROWBench evaluates entity control, long-horizon memory, and, with two VLM-based metrics, Logic-Render Alignment and Interaction Success Rate, adherence to the prescribed timeline and the visual realization of timestamped engine-recorded events.
Related papers
- StateCraft: Augmenting Video World Models with Abstract-State Reasoning
- WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models
- State World Models: Execute the Rules, Generate the Pixels
- ActionRoPE: action as coordinates for spatially consistent control in planar-camera game world models
- MultiWorldBench: Do Independently Controlled Views Describe One Shared World?
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-01 · Indexed by Pub-AI: 2026-10-02 UTC
PROWBench: Do Video Models Render What the Program Specifies?
Preprint posted: 2026-10-01 · Original record ↗
- arxiv
- 2610.02205
Explicit source versions (1)
- v2 · 2026-10-05
- Preprint available
Source date: 2026-10-01 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
- Explicit source version detected · v2
Source date: 2026-10-05 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
Abstract · arxiv
Programmable world models separate executable dynamics from visual generation, offering a promising foundation for next-generation game engines. However, their visual adherence to explicit rules and interactions remains insufficiently evaluated. Existing benchmarks assess visual quality, controllability, and instruction or physical adherence, but rarely test fidelity to fine-grained, program-specified world events. We introduce PROWBench, comprising 170 programmatically constructed episodes and 600 proxy videos covering diverse scenes and interactions. PROWBench logs entity states and timestamped events, including those outside the camera's field of view, as replayable world records, from which it renders synchronized views and proxy representations. This enables generated videos to be checked against the observable consequences of program execution. An extensible framework constructs scenes, controls behaviors, and can render each camera view in different representations, such as coarse 3D, and bounding boxes. The benchmark covers first- and third-person perspectives, with synchronized multi-view observations available for a subset of episodes. Grounded in these records, PROWBench evaluates entity control, long-horizon memory, and, with two VLM-based metrics, Logic-Render Alignment and Interaction Success Rate, adherence to the prescribed timeline and the visual realization of timestamped engine-recorded events.