Sharp Selective-Risk Envelopes under Normalized Density-Ratio Shift
Abstract
Selective prediction controls error by abstaining on uncertain inputs, but an operating point calibrated on source data can lose its guarantee under distribution shift. We characterize the exact worst-case selective risk under a normalized joint density-ratio bound, . For binary loss, it depends only on the source coverage and the accepted error rate. The familiar error-odds envelope is sharp precisely when rejected examples can absorb the normalization imbalance; at high coverage, it can discard valid operating points. We derive a closed-form, coverage-dependent error target and use it in , which combines independent score-only coverage estimation with exact binomial tests and Holm correction. Its certificate remains valid after threshold selection, its certified set contains those of the matched odds-envelope and Bonferroni rules, and it recovers every threshold that clears an explicit estimation margin. Independent linear programs match the population formulas to within on 2,220 instances. In a controlled numerical study with fully specified score–loss distributions, the matched sharp and odds rules coincide in a low-coverage regime, whereas in a high-coverage regime the sharp rule raises mean worst-case coverage from to with no observed risk violations. These results identify when normalization materially changes calibration and when the simpler odds envelope suffices.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.