Conformal Prediction under Distribution Shift: False Coverage Proportion Guarantees

Conformal Prediction (CP) is a distribution-free and model-agnostic framework for uncertainty quantification that constructs prediction sets with rigorous finite-sample coverage guarantees. Despite these advantages, the classical CP guarantee is marginal and therefore does not directly control the realized proportion of miscovered observations across a test sample. This has motivated the study of the False Coverage Proportion (FCP) in the literature, which provides a stronger form of coverage assessment over multiple predictions. Nevertheless, both classical CP and existing FCP theory typically rely on exchangeability between calibration and test data, an assumption that may fail under calibration-to-test distribution shift. Recent works have proposed Weighted CP to address this issue for classical CP by reweighting calibration observations to account for such distribution shifts. However, existing FCP theory does not directly extend to this weighted setting and has largely remained restricted to exchangeable data, leaving FCP control under distribution shift substantially unexplored. Therefore, in this work, we establish finite-sample FCP guarantees for Weighted CP under calibration-to-test distribution shift. We further study the inverse FCP problem of selecting a miscoverage level to achieve a prescribed target FCP. For practical implementation, we also develop data-driven procedures for estimating the quantities appearing in our bounds and establish their convergence. Empirically, we validate our theoretical guarantees across multiple classification and regression datasets under both classical Covariate Shift and more challenging one-phase shifts. We further investigate the large-sample behavior of the proposed bounds through asymptotic studies.