When and Why Does Test-Time Adaptation Improve Label Differentially Private Models?

Label differential privacy (LabelDP) protects sensitive training labels but often degrades accuracy. We study when test-time adaptation (TTA) can recover part of this lost utility. Since TTA uses only the released model and unlabeled inputs, it incurs no additional training-label privacy cost. We first show that symmetric randomized-response (RR) noise contracts the population label posterior toward uniform without changing the Bayes decision, biasing training toward a uniform-label objective with a privacy-dependent excess clean risk. We then characterize empirically when adaptation improves a label-private model. On clean CIFAR-10, TENT lowers the error of every LabelDP model, with gains growing from 0.87 to 3.62 points as privacy strengthens, while it increases the error of a matched non-private model. These improvements come with lower calibration error and vanish when the source model is near chance. Under common corruptions, the robustness benefits of TTA transfer to LabelDP-trained models, mostly through re-estimating batch-normalization (BN) statistics: across four TTA methods, entropy-based updates help only when combined with target-batch BN statistics. We evaluate on CIFAR-10/100, their corruption benchmarks, and ImageNet64 subsets, and identify small batches and label-correlated streams as failure conditions. With sufficiently large test batches that are not strongly label-skewed, TTA is a practical way to keep label-private models useful with or without distribution shift.