Prior-Aware Transductive Conformal Prediction for Vision–Language Models under Label Shift

Conformal prediction gives zero-shot vision–language models prediction sets with a guaranteed probability of containing the true label. Transductive methods shrink these sets by adapting scores to an unlabeled batch of query images, preserving coverage when calibration and query points are exchangeable and treated symmetrically. Conf-OT jointly fits optimal transport to calibration and query scores while fixing the pooled average class probabilities to the calibration label proportions. Compared with split conformal prediction, which calibrates fixed scores, Conf-OT reports smaller sets when both subsets share a distribution. We examine its coverage under label shift, where query class proportions change while the image distribution within each class stays fixed. We show that enforcing the calibration proportions drives the fitted class bias against classes that become more frequent in the query batch. Standard prior correction upweights these same classes. In a class-symmetric population model, we prove that the resulting coverage shortfall grows quadratically with small mismatches in class proportions. Under the strongest tested shift, Conf-OT averages 69.3% coverage versus 89.7% for split conformal prediction at a 90% target. To address this mechanism, we fit transport on half of the calibration set alone and estimate query class proportions from the other half and the query predictions. Holding the map fixed removes most of the shortfall, and prior correction reduces the remaining shortfall when class proportions can be reliably estimated from predictions. Under the same shift, our predictor attains 91.0% mean coverage with 3.30 labels per set versus 4.35 for split conformal prediction. These results link the coverage loss to pooled transport fitting and support prior-aware correction for compact prediction sets under label shift.