MENTIS-OCULI-3D: A VERIFIED BENCHMARK FOR 3D STATE TRACKING UNDER PRESCRIBED ACTIONS

Evaluating spatial prediction requires both a correct terminal label and evidence that the supplied observations determine it. We introduce MENTISOCULI-3D, a procedural benchmark for tracking 3D state under prescribed actions. Five task families cover articulated markers, hidden token identities, support release, counterfactual branches, and combined root and carousel rotations. Each sampled root yields two questions with identical text and different verified answers. The generator checks initial-view evidence, marker visibility, independent kinematic execution, and numerical stability of physical outcomes. Matched inputs vary access to intermediate images, exact states, and terminal coordinates. The artifact also contains 4,800 distribution-shift and long-horizon questions, diagnostic banks, and an interactive human-review interface that records visual assistance. The core contains 4,000 questions from 2,000 paired roots. Across 6 full-core open-model evaluations, accuracy ranges from 22.35% to 33.40% under a shared 2,048-token response budget. In two matched 800-question panels, terminal-coordinate readout scores 95.38% to 98.75%. Neither feedback-minus-repeated-image accuracy interval excludes zero. Further two-model experiments compare state trajectories, frame rewrites, output budgets, voting, and model-estimated-state tools. Separate guided-JSON probes and first-valid voting controls distinguish response completeness from geometric correctness. These results support reporting information controls and response validity alongside accuracy. They characterize behavior on the generated tasks, without establishing an internal simulation mechanism.