Hyperspherical Prototype-Driven Feature Adaptation for Test-Time Adaptation under Long-Tailed Pretraining

Test-Time Adaptation (TTA) techniques aim to adapt a pretrained model to an unlabeled target domain during inference, and existing methods are typically designed under the assumption of relatively balanced source training data. When the source dataset is long-tailed or highly-imbalanced, the classifier bias inherited from source training can cause conventional TTA to further favor majority classes and degrade minority-class recognition, even when the target distribution is balanced. To address this technical gap, we formulate TTA under long-tailed source pretraining as a prototype-driven feature adaptation problem, in which target feature representations should be adapted to better fit the decision boundaries learned from the long-tailed source data. Accordingly, we propose Hyperspherical Prototype-Driven Feature Adaptation TTA (HyPFA-TTA). HyPFA-TTA uses the normalized source classifier weights as fixed class prototypes and estimates the class tendency of each incoming target batch from the distance-based associations between target features and fixed source prototypes on a unit hypersphere. The estimated tendency is then used to correct source-biased predictions before pseudo-label generation. These corrected pseudo-labels provide prototype-based supervision for the cross-entropy and prototype-alignment objectives, through which the trainable parameters of the batch normalization layers are updated during adaptation. Experiments on ImageNet-LT$\to$ImageNet-C and the real-world Whitebait-31$\to$Whitebait-31Ex dataset demonstrate that HyPFA-TTA consistently outperforms existing TTA methods across different corruption types and source class-frequency groups, with improvements extending to both head and non-head classes.