What Can a Success-Set Flip Tell Us? Independent Response Validation in Language Model Evaluation

We ask which claims about items selected by a finite-budget success-set flip survive responses withheld from selection, larger discovery budgets, and later checkpoints. Among 151 Llama export-comparison flips selected at eight attempts, independent $V_{128}$ responses support 81 original directions; a separate $W_{512}$ pool supports 96, favors 21 opposite directions, and leaves 34 unresolved. Nine reversals survive Fisher/Holm correction over both directions of all selected items, with a median signed effect of $-9.18$ percentage points. Across saved $W$ request-index prefixes, corrected determinate directions rise from 56 at depth 128 to 91 at 512. Increasing the discovery budget to 64 makes 120 selected items solved at both endpoints, including 57 with independent directional support. A separate Qwen3-4B checkpoint study follows 95 items with independently confirmed improvements. Their fixed-denominator score declines by $12.25$ points after $t_1$, with a model-conditional posterior interval of $[-12.94,-11.17]$, although 60 items retain support relative to baseline. An unmatched reference supplies descriptive context. Together, these fixed-item measurements connect a selected label to its later directional evidence under frozen machine scores. Training provenance and response dependence limit broader interpretation. The anonymous supplement provides analysis code and item-level results, which we will also release publicly.