acceptodds
Under review as a conference paper at ICLR 2027

Dual Exam: Controlled Evaluation of Multimodal Agents and World Models

Abstract

World Models (WMs) can support Multimodal Agents (MAs) through simulated experience and action-conditioned predictions, while MA interactions can provide feedback for refining WMs. This potential reciprocal relationship motivates assessing both components. A pilot study of one MA–WM pairing across ten scenarios and 150 cases illustrates a diagnostic limitation: coupled task outcomes alone cannot identify the source of failure. We introduce Dual Exam, a unified framework that evaluates the two components separately under controlled conditions, assessing MAs under a fixed environment response mechanism and WMs under fixed action interventions. Across four representative domains including physical, digital, social, and scientific, we evaluate ten MA configurations on 1,088 cases spanning 15 reporting categories and 16 WMs across eight scenarios, with 15 evaluated on the full set of 400 cases. The highest MA Index and WM overall scores are 31.3 and 21.7 on their respective 0–100 scales, revealing substantial capability gaps under the evaluated task requirements. These findings position Dual Exam as a diagnostic framework for assessing capabilities relevant to general-purpose multimodal co-evolution.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.