Video World Models · 2026-10-01

MindWorldBench: Evaluating Mental-State-to-Behavior Reasoning in Image-to-Video Generation

Ruiqi Li, Xuanyi Liu, Sijia Li, Haofeng Wang, Yuxin Liu, Feng Xie, Songchao Tan, Shiqi Wang, Hanwei Zhu, Yizong Wang, Chuanmin Jia, Siwei Ma

arXiv:2609.39147 PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

MindWorldBench evaluates whether image-to-video models generate mental-state-conditioned behavior (belief/desire/perception) using Zero-Action Prompting and counterfactual prompt pairs (744 prompts over 372 images). The authors report that across 11 models, visual quality and commonsense plausibility stay relatively strong, but intention accuracy and world-state maintenance are much weaker, revealing frequent misalignment between latent mental states and generated actions (e.g., Omniscient Bias).

Why it matters

The paper targets a gap in video generation evaluation: generating actions from latent mental states rather than from explicit action prompts or physical continuation. By isolating causal effects of mental states with counterfactual prompting and automated LVM-based scoring, MindWorldBench provides evidence that current image-to-video systems often preserve physical/common-sense realism without tracking subjective belief and other mental variables.

Method

Defines mental-state-to-behavior reasoning using BDI-P latent variables (perception, belief, desire) to generate video actions without explicit action prompts.
Uses Zero-Action Prompting plus counterfactual prompt pairs that keep the image fixed while minimally varying a mental condition to test causal sensitivity.
Evaluates generated videos with an automated LVM pipeline (Gemini 3.1 Pro) using five metrics: Visual Quality, Commonsense Plausibility, Action Occurrence, Intention Accuracy, and World-State Maintenance.

Limitation

The paper does not state a specific limitation.

Abstract (from arXiv)

Current image-to-video models achieve visual realism and physical plausibility, but reasoning about mental states remains unexplored. Actions are driven by belief, desire, and perception, requiring inference beyond explicit instructions. We introduce MindWorldBench to evaluate mental-state-conditioned video generation. We formalize this as mental-state-to-behavior reasoning, where models generate actions from a world state and latent variables without explicit action prompts. MindWorldBench utilizes Zero-Action Prompting and a counterfactual design with 744 prompts to isolate the causal effects of mental states. An automated pipeline evaluates video quality, commonsense plausibility, and mental-state consistency. Evaluations of 11 models show that despite visual fidelity and physical reasoning, models fail to align behaviors with latent mental states. We identify a failure mode, termed Omniscient Bias, where models default to the objective world state rather than human's subjective belief. These results demonstrate a disconnect between visual generation and cognitive reasoning, suggesting a need for explicit mental-state modeling in video generation systems. Project website: https://richard2049-lee.github.io/MindWorldBench/

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-09-30 · Indexed by Pub-AI: 2026-10-01 UTC

preprint