Latent Dynamics · 2026-10-23

Dual Exam: Controlled Evaluation of Multimodal Agents and World Models

Zhefan Rao, Meng Chu, Xuan Zhang, Jingfan Chen, Guohao Cai, Runyi Li, Shiyi Du, Ning Lu, Xu Huang, Zhixuan Chen, Feng Nie, Harold Haodong Chen, Hanwen Wang, Wen-Jie Shu, Yajing Bai, Xianfeng Wu, Xuanhua He, Shuhan Guo, Ziyue Ma, Ruichuan An, Long Bai, Miaosen Chai, Wenhao Chai, Jingye Chen, Qi Chen, Xuhang Chen, Wei Chow, Minghao Fu, Yunhao Gou, Haokun Gui, Dong Huang, Wei Huang, Zi Cheng Huang, Haibo Jin, Lingdong Kong, Leung Chun Ho, Tianqin Li, Yanheng Li, Jiayuan Liu, Jiyang Liu, Mingyu Liu, Shaoteng Liu, Xiaocheng Lu, Yawen Luo, Zeyu Qin, Tiancheng Shen, Yueqi Song, Zhaolong Su, Hengkai Tan, Tianhao Tang, Yiran Tao, Juncheng Wang, Qinghe Wang, Shangshang Wang, Yingying Wang, Fang Wu, Tiange Xiang, Qiushi Yang, Weicai Ye, Weichen Yu, Jiarui Zhang, Jize Zhang, Liuzhou Zhang, Renrui Zhang, Sheng Zhang, Wenhu Zhang, Junru Zhao, Lingting Zhu, Yuxin Zuo, Yutong Bai, Long Chen, Yang Deng, Ziqi Huang, Yeying Jin, Weijia Li, Kevin Qinghong Lin, Quande Liu, See-Kiong Ng, Hongliang Ren, Hongru Wang, Shujun Wang, Qifeng Chen, Haoxuan Che

arXiv:2610.31066 PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors introduce Dual Exam, a framework that evaluates Multimodal Agents and World Models separately under controlled conditions: Agents with a fixed environment response mechanism (action selection/task completion) and World Models with fixed action interventions (state evolution/preservation). They report an Agent track (10 configurations, 1,088 items, 15 categories) and a World Model track (400 cases, 8 scenarios). Scores are track-specific and not directly comparable.

Why it matters

Dual Exam is designed to diagnose failures in coupled MA–WM interaction by separating (1) action selection/control from (2) action-conditioned state evolution. The authors argue that coupled outcomes alone can be insufficient for understanding why an interaction fails, and that reliable reciprocal improvement would require both components to remain reliable over repeated interaction.

Method

Agent Examination: evaluate candidate agents in a controlled environment with a fixed reference environment response mechanism; the agent chooses actions and task verifiers score outcomes against goals/constraints.
World Model Examination: evaluate candidate world models by fixing specification-defined intervention sequences (actions) and scoring whether generated observations realize requested effects while preserving task-relevant state/identities and obeying rules.
Use specification/verification-based evidence for both tracks; tracks need not use matched one-to-one episodes, and are meant for component-level gap characterization rather than causal attribution in coupled rollouts.

Limitation

The authors state that whether Dual Exam profiles predict coupled success or sustained reciprocal improvement remains to be tested; the framework does not by itself establish the cause of a particular coupled failure or whether joint training will yield sustained reciprocal improvement.

Abstract (from arXiv)

World Models (WMs) can support Multimodal Agents (MAs) through simulated experience, while Agent trajectories can inform World Model refinement. Evaluating this interaction requires separate evidence about action selection and state evolution. We introduce Dual Exam, which tests Agents under fixed environment response mechanisms and World Models under fixed action interventions across physical, digital, social, and scientific domains. The Agent track compares 10 configurations on 1,088 items in 15 reporting categories. The World Model track contains 400 cases across eight scenarios; 15 configurations have full coverage and one covers six scenarios. GPT 6 Astra leads the case-weighted Agent Index at 31.3, and GPT-Image-2 leads the full-coverage WM Overall at 21.7. These scores use different tasks and metrics and are not directly comparable. A 150-case pilot of one coupled pairing records zero task successes and motivates component-level diagnosis. Results reveal uneven task performance and failures to preserve task-relevant state. Their value for predicting coupled success or sustained reciprocal learning remains to be established.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-22 · Indexed by Pub-AI: 2026-10-23 UTC

preprint