Video World Models · 2026-10-20

PVBench: Quantifying Physical Plausibility in Video Generation with Object-Level Physical Measurements

Kaiwei Chen, Moran Li, Zhenye Gan, Yabiao Wang, Chengjie Wang, Kexin Zhang, Yong Liu, Jiangning Zhang

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

PVBench is a benchmark for physical plausibility in video generation. It evaluates 46 dimensions across seven domains and three layers (semantic, perceptual, physical) using specialized vision models that extract object-level physical quantities (e.g., 6D pose, metric scale, surface normals, kinematics, conservation-law residuals), with MLLM scoring where quantities are hard to compute. The authors release code, prompts, and a calibration protocol.

Why it matters

The authors argue that existing video benchmarks cannot measure whether generated videos obey physical laws. PVBench makes physical plausibility numerically testable by pairing evaluation dimensions with computable physical quantities and physical-law consistency criteria, supporting both text-to-video and image-to-video. The paper also validates agreement between automatic physical-layer metrics and human judgments.

Method

Defines PVBench’s evaluation taxonomy: 46 dimensions across 7 domains and 3 layers (semantic, perceptual, physical), with the physical layer containing 25 dimensions grouped into geometric/spatial, material/fields, kinematics/persistence, and mechanical/conservation.
Constructs 600 prompt groups with paired T2V/I2V variants sharing the same initial scene; 30% include embodied-agent scenarios, with observability constraints tied to target physical dimensions.
Evaluates physical dimensions via specialized vision models that output numerical physical quantities and check them against physical laws; maps physical values to 1–5 scores using manually set thresholds, validated with human annotations.

Abstract (from arXiv)

Recent video generation models produce visually compelling outputs, yet their physical plausibility remains poorly measured. Physical plausibility asks whether generated objects obey geometry, kinematics, dynamics, and conservation laws. Existing benchmarks rely on coarse visual-language model judgments or target embodied world models rather than video generation itself. We introduce PVBench, a physical video benchmark that quantitatively measures the plausibility of generated videos across 46 dimensions organized into seven domains, spanning: i) instruction and semantic fidelity, ii) behavioral and causal consistency, iii) perceptual and temporal quality, iv) geometric and spatial consistency, v) material properties and field states, vi) kinematic and spatiotemporal persistence, vii) and mechanical plausibility with conservation laws. PVBench comprises 600 carefully crafted prompts, of which 30% involve physical embodied agents such as robot arms, humanoids, quadrupeds, and multi-agent systems. For challenging physically tractable dimensions, it pairs each with specialized vision models that derive verifiable, physically grounded measurements, such as object-level 6D pose, metric scale, surface normals, kinematics, and numerical conservation-law residuals. It thus avoids relying solely on subjective scoring, while supporting both text-to-video and image-to-video generation. Finally, we validate the agreement between automatic metrics and human judgments through human evaluation. Meanwhile, we evaluate mainstream open-source and closed-source models and present several insights. By releasing the benchmark, prompts, and calibration protocol, we aim to foster physically grounded video generation.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-19 · Indexed by Pub-AI: 2026-10-20 UTC

preprint