What Drives Early-Stopping Gains? Threshold Policies and an MRI Case Study

Early-stopping systems use intermediate scores to decide when further computation or measurement is unnecessary. Yet greater resource savings can reflect better within-stage ranking, changes in cross-stage numerical scale, more flexible threshold policies, or a different fitting constraint, obscuring what improved. We study these dependencies for threshold-based stopping along fixed sequences, using repeated MRI as a case study. Sharing one threshold couples decisions across stages, so adding checkpoints can reduce attainable savings. Stage-specific thresholds can reproduce any shared-threshold policy that uses only a subset of checkpoints; with zero training failures, a greedy construction stops every training case no later than any feasible policy in this class. Strictly increasing stage-wise score transformations preserve stopping decisions when thresholds are transformed accordingly, distinguishing within-stage ranking from cross-stage numerical scale. Mechanism analyses in repeated prostate MRI show that these dependencies can change method comparisons: rescaling scores preserves within-stage ranking yet can reverse mean savings comparisons between scoring methods, while relaxing the training failure budget largely removes the advantage of fewer checkpoints. Brain MRI also exhibits scale sensitivity. The choice of baseline further changes the interpretation: in a separate test of prespecified policies, a strong fixed-budget control captures nearly all the primary adaptive policy’s mean saving, leaving its additional advantage unresolved. Early-stopping savings thus reflect the score, stopping policy, and fitting procedure jointly; comparisons require explicit policy classes, matched fitting constraints, and strong fixed-budget controls.