Mindmaster: Benchmarking and Analyzing Mental State Reasoning via An Immersive Social Interaction Game

Social explanation benchmarks typically evaluate what an observer believes happened, rather than how people explain decisions while participating in the interaction itself. In this work, we introduce Mindmaster Roleplay, a participant-grounded resource that links largely nonverbal social interaction to the judgments, intentions, actions, and reasons reported by the people shaping that interaction. Across 223 players, 388 trajectories, and 6,089 steps, these records preserve each explanation together with the local interaction history in which it arose. We use this resource to ask whether third-party observers recover participants' reported judgments and whether model explanations organize social evidence in the same way as Human accounts. Observer labels show limited correspondence with participants' action and intention reports. Across four models, explanation graphs systematically differ from the Human reference in their use of absence evidence, while differences in intermediate reasoning support persist under answer- and graph-size-matched analyses. A blinded human audit finds over 95% of sampled graph representations faithful to their source text. Together, these results show that “human-like explanation” is not a single target: it depends on whose interpretation is taken as the reference and whether similarity is measured in stated content or in the support structure connecting evidence, reasoning, and answers.