ButtonsBench: A Controllable Visual Reasoning Benchmark with Co-Origin Verification

Visual puzzles offer an ideal testbed for visual reasoning due to their minimal knowledge priors and essential reliance on visual information . However, current evaluation practice on such puzzles exhibits two fundamental flaws: endpoint-only scoring inflates performance by masking unfaithful shortcuts behind illusory competence, while intuitive, non-parametric difficulty grading obscures models' concrete collapse thresholds and combinatorial limits. We present four contributions toward reliable evaluation: (1) ButtonsBench-1, a diagnostic benchmark spanning 9 visual puzzle tasks parameterized by explicit difficulty knobs (“buttons”) with discrete levels, transforming difficulty from a subjective tier into a measurable, programmatically grounded property; (2) Co-Origin Probing Methodology, an evaluation protocol that pairs base queries with decoupled intermediate behavioral checkpoints under majority consensus, formalizing verified joint accuracy ($Ac{c}_{\text{joint}}$), falsification rate ($FR$), and structural crash points ($CP$); (3) ButtonsBench Platform, a modular three-stage system for on-demand programmatic generation of tailored visual materials, base questions, probes and final benchmark with ground-truth metadata to eliminate benchmark saturation and contamination; and (4) an audit of four frontier MLLMs across $1,594$ instances. We find that endpoint accuracy severely inflates perceived reasoning: probe auditing reveals widespread falsification rates ($26.1\%$ to $72.4\%$), overturning the apparent performance ranking by elevating qwen-3.8-max ($46.1\%$) over gpt-5.5 ($45.1\%$). Furthermore, mapping multi-factor crash frontiers reveals a stark decoupling: while frontier models display high endurance along isolated 1D sweeps ($>L^{\max }$ on up to 11 slices), concurrent multi-factor scaling triggers catastrophic compositional collapse (${\text{CP}}_{\text{all}}\le 5$). The data of ButtonsBench-1 benchmark and the source code of ButtonsBench platform will be fully open-sourced upon publication.