acceptodds
Under review as a conference paper at ICLR 2027

The Ruler Decides: Two Standard Reasoning Diagnostics Measure Other Than Claimed

Abstract

Same-weight dual-mode models such as Qwen3 offer a controlled pairing: one weight set, one prompt, and only an `enable_thinking` switch. We audit two standard reasoning diagnostics on this pairing: the leading variance share of the cross-problem difference matrix , and binary agreement read as metrics of divergence. In this paper, we ask what each actually measures. On MATH500 (0.6B–32B) the leading direction is mainly the question-invariant (mean) direction: **at L8** across the transition (; 0.6B–14B; question-disjoint split-half). The familiar curve is therefore a **question-specific energy-share curve**, not a measure of rank structure; a non-thinking control shows the same drop, so its existence is not evidence of a phase transition. We build a content-graded ladder of five controlled continuations. Its crossing position orders consistently with the contrast's real content, but monotonicity does not survive a family-wise joint test. That position shows no monotone scaling across six sizes, and a pre-registered difficulty axis fails outright.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.