Multiclass Projected Calibration: Estimation Limits and Cross-Entropy Stationarity

When training linear classification heads on frozen feature representations, a vanishing cross-entropy gradient signals optimization progress but does not, by itself, guarantee calibrated probabilities. We identify exactly when a zero population cross-entropy gradient guarantees multiclass calibration for finite-support input distributions. We show that this guarantee holds for every conditional label distribution if and only if linear combinations of the features can isolate each group of inputs assigned the same probability vector. If they cannot, even a global population minimizer can remain miscalibrated. We then move from exact stationarity to finite training and derive a high-probability certificate for projected smooth calibration, a multiclass calibration metric that tests differences between labels and predicted probabilities using bounded Lipschitz functions of one-dimensional probability projections. The certificate separates optimization error, representation error from calibration tests that the frozen features cannot accurately represent, and finite-sample uncertainty. We also derive a computable upper bound on this representation error using only the model predictions and frozen features. Finally, we characterize the finite-sample uncertainty incurred when calibration is evaluated on held-out data and construct an exactly calibrated predictor whose empirical score matches the scaling of our upper bound up to logarithmic factors. Synthetic controls and frozen CNN features on CIFAR-10 illustrate how feature representation affects calibration, the finite-training certificate, and the statistical limits of empirical calibration auditing.