Video World Models · 2026-10-02
PhysicsLENS: Diagnosing Physical Property Blindness in Video Generation Models
Isaiah Milkey, Som Sagar, Aditya Taparia, Xinyuan Liu, Jiqing Wen, Ransalu Senanayake
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
PhysicsLENS is a robotics-grounded video generation dataset/benchmark that evaluates both physical plausibility and “physical-property grounding” for hidden properties. Using matched scenario pairs (same conditioning frame and task, varying the latent property in the prompt), the authors find that plausible-looking videos often ignore the stated property: 34 of 47 plausible unobservable videos do not adhere to the property, and stating the property lowers plausibility only slightly (0.28 points; not significant).
Why it matters
The authors argue that existing video-generation benchmarks often evaluate visual realism rather than whether generated interactions respect text-specified physical properties that are not inferable from appearance. PhysicsLENS directly probes whether models translate prompt-stated physical properties (e.g., viscosity, friction conditions) into physically grounded motion outcomes, which is relevant for reliable use of video world models in robot learning, planning, and evaluation.
Method
- Construct matched observable/unobservable scenario pairs: same conditioning frame and task, but the unobservable prompt states a latent physical property that conflicts with what the frame suggests.
- Curate 110 prompts (80 observable + 30 matched unobservable variants) from public robot video sources; generate 439 videos using four models; annotate each clip with physical plausibility, task completion, and (for unobservable) hidden-property adherence.
- Evaluate with human ratings and an automated multi-stage diagnostic pipeline that combines motion measurements with targeted VLM questions (including a physics-error question and an unobservable hidden-property question).
Limitation
The dataset does not support an estimate of inter-annotator reliability because each clip received one final annotation rather than independent ratings from multiple annotators.
Abstract (from arXiv)
Reliable video world models could provide scalable predictive environments for robot learning, planning, and evaluation. However, generated robot videos can violate physical principles and complete tasks through physically implausible behavior, limiting their reliability for robot learning and planning. Current video-generation benchmarks exclude physics that are inherently hidden by visuals (e.g., weight, viscosity, friction). Due to this, video models are evaluated on the fidelity of physics, not the underlying accuracy of physics. We introduce PhysicsLENS, a dataset and benchmark for evaluating plausibility of physical properties grounded in robotics. PhysicsLENS uses matched scenario pairs that hold the same conditioning frame and task, while varying underlying physics in the scene description. Scenarios are curated from public robot video sources and annotated across seven physical domains: collision, gravity, momentum, friction, deformation, fluid, and causality. We evaluate across four video generation models, producing over 400 human-annotated labels. Results show that plausible-looking videos often ignore the stated property (34 of 47), and that stating the property lowers plausibility only slightly and not significantly.
Related papers
- World Models' Last Exam in Physics
- PVBench: Quantifying Physical Plausibility in Video Generation with Object-Level Physical Measurements
- Robotic Video World Models: A Survey of Applications, Research Challenges, Future Directions
- Grounded-Dreamer: Robot Learning from World Model Synthetic Data with Physical Grounding
- PhysiCalWorld: In-Context Physics Calibration for World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-01 · Indexed by Pub-AI: 2026-10-02 UTC
PhysicsLENS: Diagnosing Physical Property Blindness in Video Generation Models
Preprint posted: 2026-10-01 · Original record ↗
- arxiv
- 2610.01162
Explicit source versions (1)
- v1 · 2026-10-01
- Preprint available
Source date: 2026-10-01 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Reliable video world models could provide scalable predictive environments for robot learning, planning, and evaluation. However, generated robot videos can violate physical principles and complete tasks through physically implausible behavior, limiting their reliability for robot learning and planning. Current video-generation benchmarks exclude physics that are inherently hidden by visuals (e.g., weight, viscosity, friction). Due to this, video models are evaluated on the fidelity of physics, not the underlying accuracy of physics. We introduce PhysicsLENS, a dataset and benchmark for evaluating plausibility of physical properties grounded in robotics. PhysicsLENS uses matched scenario pairs that hold the same conditioning frame and task, while varying underlying physics in the scene description. Scenarios are curated from public robot video sources and annotated across seven physical domains: collision, gravity, momentum, friction, deformation, fluid, and causality. We evaluate across four video generation models, producing over 400 human-annotated labels. Results show that plausible-looking videos often ignore the stated property (34 of 47), and that stating the property lowers plausibility only slightly and not significantly.