Video World Models · 2026-10-07
World Models' Last Exam in Physics
Mingju Gao, Qingle Liu, Yuzhao Peng, Xinjie Lin, Ziming Qin, Zheng Jiang, Wenyi Li, Calvin Xiao, Youjie Zheng, Kaisen Yang, Qinhuai Na
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors introduce World Models’ Last Exam in Physics, a measurement-based benchmark for physical consistency in video world models. It uses 40 controlled tasks across nine physical domains, with an evaluator that gates physics scoring by temporal consistency/observability and otherwise applies task-specific quantitative measurements from each video. Across eight video generation models on 1,280 videos, the best overall score is 57.76/100 and physical inconsistencies persist.
Why it matters
Video world models may look plausible while violating physical laws, undermining prediction and planning for embodied AI. The benchmark targets observable physical relationships directly via predefined measurement criteria (no reference videos or simulator ground truth needed) and provides an interpretable diagnosis of where models fail and how evaluation agreement compares to a vision-language baseline.
Method
- Benchmark: 40 controlled image–prompt video generation tasks across nine physical categories; each task defines assumptions, an initial frame, and measurable physical criteria.
- Evaluator: run temporal-consistency/semantic-coherence and task-observability screening, then extract task-specific quantities (e.g., trajectories, periods, angles, liquid levels) using tracking/segmentation/temporal analysis and compare to predefined physical criteria.
- Composite scoring: a gated physics score contributes only when the automatic consistency score C(V)≥80; passing the screen does not guarantee physical correctness.
Limitation
Passing the temporal-consistency/observability screen neither establishes physical correctness nor guarantees recovery of every required quantity; the evaluation is restricted to these observable relationships under stated experimental assumptions, with unmeasured physical properties unverified.
Abstract (from arXiv)
Video world models can produce visually convincing yet physically inconsistent sequences, raising concerns about their reliability for prediction and planning in embodied AI systems. Existing evaluations often rely on model-based judgments or reference videos, while direct physical tests largely focus on mechanics. We introduce World Models' Last Exam in Physics, a measurement-based benchmark for evaluating physical consistency in video world models. The benchmark comprises 40 controlled tasks spanning mechanics, optics, fluids, thermal and phase-change phenomena, electromagnetism, and surface tension. Each task pairs an initial image and a generation prompt with predefined physical criteria, enabling interpretable tests of observable physical relationships without requiring reference videos. Its evaluator combines task-observability screening with task-specific quantitative physical measurements. Experiments on eight video generation models across 1,280 videos reveal persistent physical inconsistencies and substantial variation across tasks, with the best model achieving an overall score of 57.76 out of 100. Evaluation on synthetic videos with known physical relationships provides evidence for the validity of the measurement module under controlled conditions. The evaluator also achieves higher agreement with human judgments than a direct vision-language model baseline in both within-task rankings and pairwise comparisons. By combining coverage across physical domains with scores grounded in measurable evidence and explicit measurement limitations, the benchmark provides an interpretable basis for diagnosing physical inconsistencies and tracking progress toward physically consistent video world models.
Related papers
- PhysicsLENS: Diagnosing Physical Property Blindness in Video Generation Models
- PVBench: Quantifying Physical Plausibility in Video Generation with Object-Level Physical Measurements
- Robotic Video World Models: A Survey of Applications, Research Challenges, Future Directions
- PhysiCalWorld: In-Context Physics Calibration for World Models
- World Models Need a Physical Registry: Learning Internal Predictive States for Physically Plausible World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-06 · Indexed by Pub-AI: 2026-10-07 UTC
World Models' Last Exam in Physics
Preprint posted: 2026-10-06 · Original record ↗
- arxiv
- 2610.08791
Explicit source versions (1)
- v1 · 2026-10-06
- Preprint available
Source date: 2026-10-06 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Video world models can produce visually convincing yet physically inconsistent sequences, raising concerns about their reliability for prediction and planning in embodied AI systems. Existing evaluations often rely on model-based judgments or reference videos, while direct physical tests largely focus on mechanics. We introduce World Models' Last Exam in Physics, a measurement-based benchmark for evaluating physical consistency in video world models. The benchmark comprises 40 controlled tasks spanning mechanics, optics, fluids, thermal and phase-change phenomena, electromagnetism, and surface tension. Each task pairs an initial image and a generation prompt with predefined physical criteria, enabling interpretable tests of observable physical relationships without requiring reference videos. Its evaluator combines task-observability screening with task-specific quantitative physical measurements. Experiments on eight video generation models across 1,280 videos reveal persistent physical inconsistencies and substantial variation across tasks, with the best model achieving an overall score of 57.76 out of 100. Evaluation on synthetic videos with known physical relationships provides evidence for the validity of the measurement module under controlled conditions. The evaluator also achieves higher agreement with human judgments than a direct vision-language model baseline in both within-task rankings and pairwise comparisons. By combining coverage across physical domains with scores grounded in measurable evidence and explicit measurement limitations, the benchmark provides an interpretable basis for diagnosing physical inconsistencies and tracking progress toward physically consistent video world models.