The Resolution of Noisy Validation : Model Selection From Repeated Measurements

More proxy labels can estimate an observed distribution ever more precisely while leaving the best predictor unidentified. We characterize this obstacle for repeated measurements of a positive target. In a fixed finite panel allowing dependent reader errors, two admissible worlds share the complete observation law yet reverse the risks of the same nonconstant candidate pair. Matching bounds give a cubic resolution floor from second-order information, a quartic floor plus interface error from marginal third-order or matched signed information, and a sampling-limited frontier under exact transport. Candidate-disagreement directions map channel uncertainty to risk comparisons; one simultaneous matrix certifies exact selection, tolerance and an entire curve of minimum-improvement promises. Boundary-uniform moment geometry makes the information operational at finite noise. A pre-specified paid comparison tests the same-cost value of acquiring and exploiting signed information; same-information controls isolate processing gains. A stochastic-target control preserves every candidate risk difference and calibration interval while increasing evaluation variance. Together, these results identify what measurement information resolves a comparison, how strongly the available data can support an improvement, and which remaining uncertainty blocks the decision