Difficulty-Calibrated Conformal Prediction for Transfer Learning

State-of-the-art machine learning models, including foundation models, are predominantly non-Bayesian, and reliable methods for quantifying their predictive uncertainty remain underdeveloped. This challenge becomes particularly important in transfer learning, where models are adapted to downstream tasks with limited data. Conformal prediction (CP) provides model-agnostic uncertainty quantification with finite-sample coverage guarantees, but conventional CP does not exploit uncertainty-related information acquired during pretraining. We introduce Difficulty-Calibrated Conformal Prediction (DC-CP), a framework that transfers source-derived task difficulty to target-task calibration. DC-CP requires neither paired source–target observations nor a shared output space and learns only a one-dimensional link between source difficulty and target prediction error from limited target data. With exact covariate-shift weights, DC-CP combined with weighted CP retains finite-sample marginal coverage under distribution shift while adapting uncertainty estimates to input difficulty. Experiments on two open-domain question-answering benchmarks for large language models and a simulation-to-real polymer science benchmark demonstrate effective difficulty transfer across tasks and domains. Our results establish a model-agnostic framework for transferring not only predictive knowledge but also uncertainty-related information to downstream tasks.