- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper considers test-time adaptation (TTA) when the source model was trained on long-tailed data. The authors propose HyPFA-TTA, which treats the normalized source classifier weights as fixed class prototypes. The adaptation procedure consists of: 1) projecting the target features and source class prototypes onto a unit hypersphere; 2) estimating the class tendency of the current target batch from feature-to-prototype associations computed using a Gaussian RBF; 3) adjusting the prototype-based logits using the estimated class tendency and generating pseudo-labels; and 4) updating the batch-normalization parameters using a cross-entropy loss (L_CE) and a prototype-alignment loss (L_align). A short warm-up stage is applied before feature normalization and class-tendency correction. Experiments on ImageNet-LT->ImageNet-C and Whitebait-31->Whitebait-31Ex show improvements over existing TTA methods.
Critical strengths
- The paper provides strong empirical support. The Whitebait results show that conventional TTA may improve overall accuracy mainly through head classes while degrading many-shot and medium-shot classes. HyPFA-TTA instead achieves the best overall result and improves both many-shot and medium-shot accuracy. It also obtains the best results across all 15 ImageNet-C corruptions.
- The paper is well motivated and clearly organized. The relationship between long-tailed pretraining, biased pseudo-labels, and prototype-driven adaptation is presented clearly. The overview, algorithm, and ablations make the method relatively easy to understand.
Critical weaknesses
The confidence-filtering mechanism after warm-up is unclear. After warm-up, both the target features and class prototypes are normalized, so the cosine logits would be bounded between -1 and 1. With the confidence threshold \tau=0.5, it is unclear whether any target sample can satisfy the selection criterion. The paper should clarify the proportion of samples selected after warm-up.
Questions for authors
After warm-up, how many samples satisfy \gamma_i > \tau (=0.5)? Since the features and prototypes are normalized, the logits are bounded cosine similarities. Is an additional temperature or logit scale used in the implementation but omitted from the paper?
[Rebuttal, Part-3] Roles of Batch-level Class Tendency, Logit Correction, and Prototype-guided Feature Alignment
We thank Reviewers jMgB, nnPs, and i6uR for their questions regarding batch-level class tendency, logit correction, and the contributions of individual components. We here clarify that the core design of HyPFA-TTA is not simply to adjust classification scores through logit correction. Instead, it combines batch-level tendency-based logit correction with prototype-guided feature alignment to perform TTA of a long-tailed pretrained model.
The main idea of our method is to model long-tailed TTA as a feature alignment/re-orientation task in which target features are aligned with source class prototypes. Furthermore, minority collapse caused by long-tailed pretraining may cause the angular relationships among source class prototypes to deviate from the equiangular structure described by neural collapse. Hence, in addition to adjusting feature directions, we employ logit adjustment to correct the classification scores computed from feature-to-prototype inner products, thereby addressing classification biases inherited from long-tailed pretraining.
To clarify this design, we show again the Table 3(B) in our manuscript and describe the relationships among the individual components.
Table 3(B). Component ablation of HyPFA-TTA on Whitebait-31 → Whitebait-31Ex. All results are classification accuracy (%). LC denotes logit correction, Nm denotes vector normalization, PU denotes prototype update, and Wrm denotes warm-up. As stated in Rebuttal Part-1, the CE loss is zero in configurations without warm-up, even when the CE component is enabled.
(1) Normalization eliminates prototype norm variation caused by long-tailed pretraining
Long-tailed source pretraining causes the classifier weights (prototypes) of different classes to have different vector norms. Hence, classification scores are affected not only by feature-to-prototype directional similarity but also by prototype magnitudes. We therefore apply normalization to eliminate prototype norm variation, so that classification decisions depend primarily on the directional relationships between features and prototypes.
Configuration-(1) in Table 3(B) shows that normalization alone increases the overall classification accuracy from 25.73% for the source model to 40.26%. Head accuracy increases from 44.51% to 68.64%, Many-shot accuracy from 11.26% to 14.96%, and Medium-shot accuracy from 6.60% to 18.70%.
These results demonstrate that eliminating prototype norm variation inherited from long-tailed pretraining can substantially improve classification performance. However, normalization only addresses differences in prototype magnitudes. It does not directly adjust target feature directions or fully resolve the classification biases caused by long-tailed pretraining.
(2) Feature alignment alone can become trapped by head classes
The central research idea of HyPFA-TTA is to perform feature re-orientation by aligning target features with their corresponding source class prototypes, thereby addressing feature-to-prototype directional misalignment caused by domain shift.
However, when feature alignment is applied alone, the adaptation process can become trapped by head classes. Since a long-tailed source model is inherently biased toward head classes, directly performing feature-to-prototype alignment may cause the re-orientation of target features to be dominated by head classes, further exacerbating head-class dominance.
This phenomenon can be observed in Configuration-(2) of Table 3(B). Compared with Configuration-(1), which uses normalization alone, adding the alignment loss maintains the overall accuracy at 40.26%, but Head accuracy increases from 68.64% to 79.00%. Meanwhile, Many-shot accuracy decreases from 14.96% to 5.55%, and Medium-shot accuracy decreases from 18.70% to 5.18%.
These results indicate that relying solely on feature alignment may cause adaptation to become excessively biased toward head classes, preventing effective improvements in long-tailed TTA performance.
Therefore, we introduce logit adjustment to mitigate the influence of head-class dominance on the feature re-orientation process.