Reconciling robustness and finite-sample efficiency in Gaussian process regression

Robust fitting reduces the influence of outliers but can sacrifice finite-sample predictive efficiency when the data follow the assumed model. In this work, we quantify how robust fitting changes the finite-sample predictive gain from Studentization, which accounts for uncertainty in an estimated variance scale rather than treating its estimate as known. Under the assumed Gaussian process (GP) model with a shared variance scale, fixed kernel parameters, and more than two training observations, Studentization yields a strictly positive finite-sample gain in expected log predictive density over plug-in Gaussian prediction, and this gain is available in closed form. We combine Studentization with density power divergence (DPD) fitting and quantify the predictive cost of robustification. Our analysis relates this cost to model complexity and the influence of the training responses on test predictions. It gives a sufficient condition for retaining a prescribed fraction of the exact Studentization gain over ordinary Gaussian prediction. Controlled synthetic experiments support the analysis and show predominantly positive gains from Studentization in prediction. Beyond the theoretical assumptions, applying Studentization to existing robust GP methods mostly improves joint predictive log likelihood, with only small decreases otherwise. Finally, we derive a distribution-free bound on the possible decrease in log predictive density when Studentization is applied to an already fitted model. The code is available at https://anonym ous.4open.science/r/rsgpr-0E17/.