acceptodds
Under review as a conference paper at ICLR 2027

Hyperspherical Prototype-Driven Feature Adaptation for Test-Time Adaptation under Long-Tailed Pretraining

Abstract

Test-Time Adaptation (TTA) techniques aim to adapt a pretrained model to an unlabeled target domain during inference, and existing methods are typically designed under the assumption of relatively balanced source training data. When the source dataset is long-tailed or highly-imbalanced, the classifier bias inherited from source training can cause conventional TTA to further favor majority classes and degrade minority-class recognition, even when the target distribution is balanced. To address this technical gap, we formulate TTA under long-tailed source pretraining as a prototype-driven feature adaptation problem, in which target feature representations should be adapted to better fit the decision boundaries learned from the long-tailed source data. Accordingly, we propose Hyperspherical Prototype-Driven Feature Adaptation TTA (HyPFA-TTA). HyPFA-TTA uses the normalized source classifier weights as fixed class prototypes and estimates the class tendency of each incoming target batch from the distance-based associations between target features and fixed source prototypes on a unit hypersphere. The estimated tendency is then used to correct source-biased predictions before pseudo-label generation. These corrected pseudo-labels provide prototype-based supervision for the cross-entropy and prototype-alignment objectives, through which the trainable parameters of the batch normalization layers are updated during adaptation. Experiments on ImageNet-LTImageNet-C and the real-world Whitebait-31Whitebait-31Ex dataset demonstrate that HyPFA-TTA consistently outperforms existing TTA methods across different corruption types and source class-frequency groups, with improvements extending to both head and non-head classes.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
TTA, long-tailed classification, Prototype Re-orientation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_AundOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper considers test-time adaptation (TTA) when the source model was trained on long-tailed data. The authors propose HyPFA-TTA, which treats the normalized source classifier weights as fixed class prototypes. The adaptation procedure consists of: 1) projecting the target features and source class prototypes onto a unit hypersphere; 2) estimating the class tendency of the current target batch from feature-to-prototype associations computed using a Gaussian RBF; 3) adjusting the prototype-based logits using the estimated class tendency and generating pseudo-labels; and 4) updating the batch-normalization parameters using a cross-entropy loss (L_CE) and a prototype-alignment loss (L_align). A short warm-up stage is applied before feature normalization and class-tendency correction. Experiments on ImageNet-LT->ImageNet-C and Whitebait-31->Whitebait-31Ex show improvements over existing TTA methods.

Critical strengths
  1. The paper provides strong empirical support. The Whitebait results show that conventional TTA may improve overall accuracy mainly through head classes while degrading many-shot and medium-shot classes. HyPFA-TTA instead achieves the best overall result and improves both many-shot and medium-shot accuracy. It also obtains the best results across all 15 ImageNet-C corruptions.
  2. The paper is well motivated and clearly organized. The relationship between long-tailed pretraining, biased pseudo-labels, and prototype-driven adaptation is presented clearly. The overview, algorithm, and ablations make the method relatively easy to understand.
Critical weaknesses

The confidence-filtering mechanism after warm-up is unclear. After warm-up, both the target features and class prototypes are normalized, so the cosine logits would be bounded between -1 and 1. With the confidence threshold \tau=0.5, it is unclear whether any target sample can satisfy the selection criterion. The paper should clarify the proportion of samples selected after warm-up.

Questions for authors

After warm-up, how many samples satisfy \gamma_i > \tau (=0.5)? Since the features and prototypes are normalized, the logits are bounded cosine similarities. Is an additional temperature or logit scale used in the implementation but omitted from the paper?

Reviewer_nnPsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses test-time adaptation (TTA) for models pretrained on long-tailed data, where class bias inherited from source training can degrade pseudo-label quality and steer adaptation toward majority classes. The authors propose HyPFA-TTA, which treats the normalized source classifier weights as fixed class prototypes and projects target features onto the unit hypersphere. The distance between features and prototypes are averaged within each batch to get a class tendency on the batch level, which is utilized to adjust the prediction logits before generating pseudo-labels. The method is benchmarked on ImageNet-LT to ImageNet-C and on Whitebait-31to Whitebait-31Ex, with additional experiments on CIFAR-100-LT to CIFAR-100-C.

Critical strengths

The paper studies TTA under long-tailed source pretraining which is a realistic setting and is less explored than target-side imbalance. TENT achieves higher overall accuracy but many-shot accuracy decreases, which shows that standard TTA can magnify inherited bias.

Critical weaknesses
  1. The correction penalizes classes that receive more predictions, which may also affect classes that are genuinely frequent in the target distribution. It would be helpful to see the method evaluated under long-tailed or non-i.i.d. target streams.
  2. The authors should consider including BN-statistics-only adaptation (AdaBN) and a training-free variant (normalized prototypes with batch-prior correction, without gradient updates) to clarify where the gains come from. The CE-only variant already improves Whitebait accuracy from 25.73 to 40.26 so we need to understand how much of the improvement is due to normalization or BN adaptation versus the proposed correction and alignment objectives.
  3. L-TTA [1] is a relevant baseline the authors should discuss. In addition, LAME [2] is evaluated only on Whitebait, SAR [3] does not appear in the class-group comparison in Table 2, and T3A [4] and Label Shift Adapter [5] are discussed but not compared.
  4. The paper states that the method consistently outperforms existing approaches which should be checked. At batch size 64, SAR performs better on 9 of 15 ImageNet-C corruptions, and RoTTA does so on 3 at batch size 16. Few-shot accuracy also remains low (1.4 - 4.1% on the reported ImageNet-C corruptions and 0.50% on Whitebait).
  5. A sensitivity study for alpha, tau, lambda_align and T_w should be added alongwith an explanation on how these values were selected without target labels. The component ablations are currently limited to Whitebait. One result I found difficult to interpret is that adding logit correction to CE alone reduces accuracy from 40.26 to 29.38 (Table 3(b)), while combining it with alignment improves performance. Some discussion of this interaction, and of the motivation for the warm-up stage beyond empirical results, would be helpful.
  6. The ImageNet and Whitebait experiments use ResNet-101, the CIFAR backbone is not specified, and only severity 5 is reported. Results on additional backbones and severity levels are needed for ViTs and vision-language models which would help the method's generalizability.

References: [1] Long-Tailed Test-Time Adaptation for Vision-Language Models. ICLR 2026. [2] Parameter-Free Online Test-Time Adaptation. CVPR 2022. [3] Towards Stable Test-Time Adaptation in Dynamic Wild World. ICLR 2023. [4] Test-Time Classifier Adjustment Module for Model-Agnostic Domain Generalization. NeurIPS 2021. [5] Label Shift Adapter for Test-Time Adaptation under Covariate and Label Shifts. ICCV 2023.

Questions for authors

Please refer to the weaknesses listed.

Reviewer_i6uROpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper studies test-time adaptation when the source model was trained on long-tailed data. HyPFA-TTA normalizes the source classifier weights into fixed class prototypes, estimates class tendencies from target batches, adjusts pseudo-labels, and updates batch-normalization parameters using cross-entropy and prototype-alignment losses.

Critical strengths

On the main ImageNet-LT to ImageNet-C benchmark, HyPFA-TTA reports a mean accuracy of 20.43% across 15 corruption types, compared with 17.89% for RoTTA, the strongest listed baseline by mean accuracy. The paper also reports results by source-class frequency and tests several batch sizes.

Critical weaknesses
  1. The stated confidence rule is inconsistent with the stated equations. After warm-up, both features and prototypes are unit-normalized. Using the paper’s association formula and alpha = 0.1, the adjusted logits of any two classes can differ by at most 2.4. The resulting maximum softmax confidence is below the stated threshold of 0.5 for the 31-, 100-, and 1000-class experiments. Thus, under the equations as written, the set of samples selected for cross-entropy is empty after warm-up. The paper does not specify an additional logit scale or explain how the loss is handled when this set is empty.
  2. The primary public benchmark lacks controls needed to establish which components drive the gain. Most component ablations are reported on Whitebait. On ImageNet-C, it would be useful to distinguish the benefit of normalization alone from the effects of logit adjustment, warm-up, and parameter updates. This matters because classifier-weight normalization has already been studied in long-tailed recognition.
  3. The performance advantage does not carry over uniformly to the additional public benchmark. In the five CIFAR-100-LT to CIFAR-100-C corruption settings reported in the appendix, HyPFA-TTA does not achieve the highest overall accuracy in any setting. This limits the breadth of the paper’s performance claim.
Questions for authors

Regarding weakness 1: Does the implementation use a logit scale, temperature, or confidence calculation not specified in the paper? What proportion of samples passes the 0.5 threshold after warm-up, and how is the cross-entropy loss handled when none does? Please clarify how the published equations correspond to the implementation.

Regarding weakness 2: Are results available on ImageNet-C for normalization without adaptation, logit adjustment without adaptation, and warm-up without the subsequent full procedure? These comparisons would help establish the contribution of each step on the main benchmark.

Regarding weakness 3: Why does the method fail to rank first on the five reported CIFAR-100-C corruptions despite leading on all 15 ImageNet-C corruptions? Which differences between the two settings account for this change?

Reviewer_jMgBOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper considers test-time adaptation (TTA) for models trained on long-tailed data. In this setting, the source classifier can remain biased toward head classes even when the target stream is balanced. The proposed method, HyPFA-TTA, uses ℓ2\ell_2-normalised classifier weights as fixed class prototypes. It estimates a batch-level class tendency, πbatch\pi^{\text{batch}}, from soft assignments ai,k∝exp⁡(−∥f^i−Ck∥22)a_{i,k}\propto\exp(-\|\hat{\mathbf f}_i-\mathbf C_k\|_2^2), then adjusts logits by subtracting αlog⁡πkbatch\alpha\log\pi^{\text{batch}}_k before generating pseudo-labels. After a 10-batch warm-up, it updates the BN affine parameters using a thresholded cross-entropy loss and a cosine prototype-alignment loss.

The experiments cover ImageNet-LT → ImageNet-C with ResNet-101 at corruption severity 5, a new real-world dataset pair (Whitebait-31 → 31Ex), and CIFAR-100-LT → CIFAR-100-C in the appendix. The main comparisons include TENT, SAR, RoTTA, DeYO, and ROID, with LAME and Tsallis-entropy also evaluated on Whitebait. The paper reports the highest average accuracy on ImageNet-C and Whitebait.

Critical strengths
  1. The paper addresses a worthwhile limitation of existing TTA work, which generally assumes a balanced source dataset. The Whitebait results in Table 3(a) illustrate the problem clearly: TENT improves overall accuracy from 25.7 to 36.3, but many-shot accuracy drops from 11.3 to 0.8. HyPFA-TTA is the only method in this comparison to improve head, many-shot, and medium-shot accuracy together. I found this a compelling motivation for the approach.

  2. The release of Whitebait-31/31Ex is a useful contribution. It provides a more realistic testbed than synthetic corruptions alone.

Critical weaknesses
  • The baseline configurations are not clear enough to assess the comparisons. Section 4.1 (lines 340–342) specifies SGD with a learning rate of 10−310^{-3} and momentum 0.9, with a batch size of 32 on ImageNet-C. However, I could not determine whether these settings also apply to the baselines. Neither the main paper nor the appendix provides per-method hyperparameters or explains departures from the published configurations. This matters because those configurations differ substantially:
MethodPublished ImageNet-C setting (ResNet-50)Source
TENTSGD with momentum; lr 2.5e-4, BS 64; lr reduced with batch sizeWang et al. (2021), Sec. 4
SARSGD with SAM (ρ=0.05), momentum 0.9; lr 2.5e-4, BS 64Niu et al. (2023), Sec. 5
DeYOSGD, momentum 0.9; lr 2.5e-4, BS 64; τEnt=0.5ln⁡C\tau_{\text{Ent}}=0.5\ln C, τPLPD=0.3\tau_{\text{PLPD}}=0.3Lee et al. (2024), App. E.3
ROIDSame as TENT (lr 2.5e-4, BS 64); α=0.99, τ=1/3Marsden et al. (2024), “Implementation details”
RoTTAAdam, lr 1e-3, BS 64; not evaluated on ImageNet-CYuan et al. (2023), Sec. 4.1

If the baselines were also run with lr 1e-3 at BS 32, this choice needs justification. It could help explain why some methods perform worse than the source model; Marsden et al., Appendix A.2, Figure 6, is relevant here. A shared configuration also does not remove the need to explain how HyPFA-TTA's learning rate, α\alpha, τ\tau, λalign\lambda_{\text{align}}, and TwT_w were selected. I would like to see how sensitive the reported gains are to these choices.

  • The summary averages do not agree with the per-corruption results. When I recompute the 15-corruption means from Tables 1 and 4, I obtain different values from those in Table 7. The largest discrepancy is SAR at BS=64: its mean is 19.46, rather than 14.99. The latter is ROID's mean, which suggests that some rows may have been shifted. At BS=64, SAR matches or exceeds HyPFA-TTA on 9 of the 15 corruptions. Based on Table 4B, HyPFA-TTA's average advantage over SAR is about 2.3 points (21.78 vs. 19.46), whereas Table 7 suggests about 6.5 points (21.51 vs. 14.99). There are also discrepancies for Source (9.44 vs. 10.80) and Ours at BS=32 (21.05 vs. 20.43). These differences affect the size of the claimed improvement and need to be resolved.

  • The CIFAR results do not support the claim of consistent improvement. On CIFAR-100-LT → CIFAR-100-C (Table 10), HyPFA-TTA is not the best method on any of the five corruptions, while ROID is best on three. This seems at odds with the abstract's claim that the method “consistently outperforms” the alternatives. The paper should discuss these results and qualify the claim accordingly.

  • The description of pseudo-labelling appears to be missing a logit scale. After warm-up, the logits are described as cosine similarities in [−1,1][-1,1], with no temperature specified. For logits in this range, the maximum softmax confidence is e2/(e2+K−1)e^2/(e^2+K-1), or approximately 0.007 for K=1000K=1000 and 0.20 for K=31K=31. Both are well below τ=0.5\tau=0.5, so the thresholded LCE\mathcal L_{\text{CE}} would never be active under this interpretation. The limited scale of the soft assignments also raises a question about how informative πbatch\pi^{\text{batch}} is. I may be missing an implementation detail, but the paper needs to specify the scaling and the confidence calculation, including how logit adjustment enters it. Until this is clear, I find the ablation in Table 3(b) difficult to interpret.

  • Additional controls are needed to identify where the gains come from. Two comparisons would be particularly useful: (1) normalised classifier weights without adaptation (τ-norm, Kang et al., 2020), since normalisation is the most influential component in Table 3(b); and (2) batch-prior logit adjustment without BN updates (cf. Saerens et al., 2002). The evaluation also uses a single seed without reporting variance, considers only severity 5, and uses only i.i.d. streams. The last point is especially relevant because batch-prior estimation may be sensitive to non-i.i.d. streams. Finally, the target class distribution of Whitebait-31Ex is not reported.

Taken together, these issues make it difficult to judge how much of the reported gain comes from the proposed method and how much depends on the evaluation choices.

Questions for authors
  1. Could you provide the optimizer, learning rate, and method-specific hyperparameters for every baseline, and explain any departures from the published settings above? How do the baselines perform with those settings, with learning rates scaled linearly to your batch size where applicable (approximately 1.25e-4 at BS 32)?

  2. How were the learning rate, α\alpha, τ\tau, λalign\lambda_{\text{align}}, and TwT_w selected? Did you use a held-out stream? Does HyPFA-TTA retain its advantage across a reasonable range of these parameters, particularly learning rates from 10−610^{-6} to 10−310^{-3}?

  3. Is a logit scale or temperature used after warm-up? Please clarify how confidence is calculated and report the fraction of samples that exceed τ\tau.

  4. Could you reconcile Table 7 with Tables 1 and 4, particularly the SAR result at BS=64 (19.46 vs. 14.99)?

  5. What accuracy do you obtain with normalized classifier weights alone, and with logit adjustment but no BN updates?

  6. What explains the weaker CIFAR-100-LT results? Please also report the Whitebait-31Ex target class distribution and discuss the method's behaviour on non-i.i.d. streams.

References for the hyperparameter table: Wang et al., Tent, ICLR 2021 (arXiv:2006.10726); Niu et al., SAR, ICLR 2023 (arXiv:2302.12400); Lee et al., DeYO, ICLR 2024 (arXiv:2403.07366); Marsden et al., ROID, WACV 2024 (arXiv:2306.00650); Yuan et al., RoTTA, CVPR 2023 (arXiv:2303.13899).

Comments on the paper

5 comments
AuthorsOpenReview ↗

[Rebuttal, Part-3] Roles of Batch-level Class Tendency, Logit Correction, and Prototype-guided Feature Alignment

We thank Reviewers jMgB, nnPs, and i6uR for their questions regarding batch-level class tendency, logit correction, and the contributions of individual components. We here clarify that the core design of HyPFA-TTA is not simply to adjust classification scores through logit correction. Instead, it combines batch-level tendency-based logit correction with prototype-guided feature alignment to perform TTA of a long-tailed pretrained model.

The main idea of our method is to model long-tailed TTA as a feature alignment/re-orientation task in which target features are aligned with source class prototypes. Furthermore, minority collapse caused by long-tailed pretraining may cause the angular relationships among source class prototypes to deviate from the equiangular structure described by neural collapse. Hence, in addition to adjusting feature directions, we employ logit adjustment to correct the classification scores computed from feature-to-prototype inner products, thereby addressing classification biases inherited from long-tailed pretraining.

To clarify this design, we show again the Table 3(B) in our manuscript and describe the relationships among the individual components.

Table 3(B). Component ablation of HyPFA-TTA on Whitebait-31 → Whitebait-31Ex. All results are classification accuracy (%). LC denotes logit correction, Nm denotes vector normalization, PU denotes prototype update, and Wrm denotes warm-up. As stated in Rebuttal Part-1, the CE loss is zero in configurations without warm-up, even when the CE component is enabled.

ConfigurationCEAlignLCNmPUWrmAllHeadManyMed.Few
Source––––––25.7344.5111.266.600.00
(1)✓––✓––40.2668.6414.9618.700.63
(2)✓✓–✓––40.2679.005.555.180.00
(3)✓–✓✓––29.3853.697.748.720.38
(4)✓✓✓✓––41.9267.0722.5719.140.63
(5)✓✓✓–––40.5377.412.6717.990.00
(6)✓✓–✓✓–39.4667.0214.8518.810.50
(7)✓✓✓✓✓–41.0965.3922.5718.920.88
Full model✓✓✓✓–✓42.9470.4619.8720.770.50

(1) Normalization eliminates prototype norm variation caused by long-tailed pretraining

Long-tailed source pretraining causes the classifier weights (prototypes) of different classes to have different vector norms. Hence, classification scores are affected not only by feature-to-prototype directional similarity but also by prototype magnitudes. We therefore apply normalization to eliminate prototype norm variation, so that classification decisions depend primarily on the directional relationships between features and prototypes.

Configuration-(1) in Table 3(B) shows that normalization alone increases the overall classification accuracy from 25.73% for the source model to 40.26%. Head accuracy increases from 44.51% to 68.64%, Many-shot accuracy from 11.26% to 14.96%, and Medium-shot accuracy from 6.60% to 18.70%.

These results demonstrate that eliminating prototype norm variation inherited from long-tailed pretraining can substantially improve classification performance. However, normalization only addresses differences in prototype magnitudes. It does not directly adjust target feature directions or fully resolve the classification biases caused by long-tailed pretraining.

(2) Feature alignment alone can become trapped by head classes

The central research idea of HyPFA-TTA is to perform feature re-orientation by aligning target features with their corresponding source class prototypes, thereby addressing feature-to-prototype directional misalignment caused by domain shift.

However, when feature alignment is applied alone, the adaptation process can become trapped by head classes. Since a long-tailed source model is inherently biased toward head classes, directly performing feature-to-prototype alignment may cause the re-orientation of target features to be dominated by head classes, further exacerbating head-class dominance.

This phenomenon can be observed in Configuration-(2) of Table 3(B). Compared with Configuration-(1), which uses normalization alone, adding the alignment loss maintains the overall accuracy at 40.26%, but Head accuracy increases from 68.64% to 79.00%. Meanwhile, Many-shot accuracy decreases from 14.96% to 5.55%, and Medium-shot accuracy decreases from 18.70% to 5.18%.

These results indicate that relying solely on feature alignment may cause adaptation to become excessively biased toward head classes, preventing effective improvements in long-tailed TTA performance.

Therefore, we introduce logit adjustment to mitigate the influence of head-class dominance on the feature re-orientation process.

AuthorsOpenReview ↗

[Rebuttal Part-3 (2/4, continued)] Roles of Batch-level Class Tendency, Logit Correction, and Prototype-guided Feature Alignment

(3) Logit correction alone cannot replace feature re-orientation

Conversely, applying logit correction without performing feature alignment is also insufficient to address the long-tailed TTA problem.

This can be explained by a straightforward mathematical observation. For a model employing a fully connected layer as its final classifier, classification logits are essentially computed through the inner products between feature vectors and class prototypes:

$ z_{i,k}=f_i^\top C_k. $

Logit adjustment modifies the classification scores after these inner products have been computed:

$ \tilde z_{i,k} =z_{i,k}-\alpha\log(\pi_k^{\mathrm{batch}}+\epsilon). $

However, modifying logit scores alone does not change the directions of feature vectors and therefore cannot directly correct the directional misalignment between target features and source class prototypes. This mathematical relationship follows from the basic operation of a linear classifier (inner product accomplished by the final fully connected layer) and does not depend on any particular TTA network architecture.

Configuration-(3) in Table 3(B) shows that adding logit correction to normalization without feature alignment substantially decreases the overall accuracy from 40.26% to 29.38%. Many-shot accuracy also decreases from 14.96% to 7.74%.

These results demonstrate that adjusting classification scores through logit correction alone is insufficient to address long-tailed TTA. Effective feature adaptation still requires feature re-orientation.

(4) Feature alignment and logit adjustment must work together

The preceding two experiments reveal the most important technical consideration underlying the design of HyPFA-TTA.

On the one hand, feature alignment alone is susceptible to head-class dominance, causing feature re-orientation to become trapped by head classes. On the other hand, logit correction alone only modifies classification scores and cannot directly adjust feature-to-prototype geometry. Therefore, feature alignment and logit adjustment must work together to address both target feature directions and classification biases inherited from the long-tailed source classifier.

This is also why we adopt the concept of logit adjustment from MDCS (ICCV 2023, "MDCS: More Diverse Experts with Consistency Self-distillation for Long-tailed Recognition"). MDCS improves long-tailed recognition by reducing the logit scores of many-shot classes while compensating for those of medium-shot and few-shot classes. HyPFA-TTA extends this concept to test-time adaptation. Rather than directly using predefined head-, medium-, or few-shot class groups, we estimate batch-level class tendency through soft RBF associations between target features and fixed source prototypes in the current target batch:

$ \pi_k^{\mathrm{batch}} =\frac{1}{B}\sum_{i=1}^{B}a_{i,k}. $

This tendency reflects the relative degree of association received by each source prototype within the current batch and is used for logit adjustment. In other words, we use the characteristics of the current target batch to determine which prototypes exhibit higher or lower class tendencies and adjust their classification scores accordingly, rather than directly relying on the class proportions of the source training distribution.

Under this design, logit adjustment and feature alignment are not two independent classification correction procedures. Logit adjustment mitigates head-class dominance, while feature alignment re-orients target features toward their corresponding source prototypes.

Configuration-(4) in Table 3(B) provides direct experimental support for this design. When normalization, logit correction, and feature alignment are combined, the overall accuracy reaches 41.92%, with a Many-shot accuracy of 22.57% and a Medium-shot accuracy of 19.14%.

Compared with Configuration-(2), which combines normalization and alignment without logit correction, Configuration-(4) reduces Head accuracy from 79.00% to 67.07%, while substantially improving Many-shot accuracy from 5.55% to 22.57% and Medium-shot accuracy from 5.18% to 19.14%. The overall accuracy also increases from 40.26% to 41.92%.

Furthermore, compared with Configuration-(3), which combines normalization and logit correction without alignment, Configuration-(4) improves overall accuracy from 29.38% to 41.92% and Many-shot accuracy from 7.74% to 22.57%.

These results demonstrate that combining logit adjustment with feature-to-prototype alignment effectively mitigates the head-class trap and improves long-tailed TTA performance through feature re-orientation. Accordingly, Configuration-(4) achieves the highest overall accuracy among all ablation configurations without warm-up (note that CE is 0 in this setting and thus can be considered as disabled. ).

AuthorsOpenReview ↗

[Rebuttal Part-3 (3/4, continued)] Roles of Batch-level Class Tendency, Logit Correction, and Prototype-guided Feature Alignment

(5) Removing normalization further highlights the importance of feature re-orientation

Configuration-(5) in Table 3(B) removes normalization while retaining logit correction and the alignment loss, achieving an overall accuracy of 40.53%.

Compared with Configuration-(2) and Configuration-(3), this result indicates that even without the complete normalization design, adjusting feature directions through the alignment loss can still yield considerable overall classification performance.

This observation further supports our research perspective of formulating long-tailed TTA as a feature-to-prototype alignment/re-orientation problem.

Nevertheless, Configuration-(5) achieves a Head accuracy of 77.41%, while its Many-shot accuracy is only 2.67%. Compared with the Many-shot accuracy of 22.57% achieved by Configuration-(4), this substantial difference further demonstrates that normalization plays an important role in reducing prototype norm variation caused by long-tailed pretraining and improving the recognition of non-head classes.

Therefore, although feature re-orientation is the core of adaptation, normalization and logit adjustment are also essential components for enabling effective adaptation under long-tailed classification bias.

(6) The relationship between prototype update and logit adjustment

Configurations-(6) and -(7) in Table 3(B) further evaluate the effect of prototype update (PU).

When normalization and alignment are combined, Configuration-(6) achieves an overall accuracy of 39.46%, even when class prototypes are allowed to be updated during adaptation. Adding logit correction further improves the overall accuracy to 41.09% in Configuration-(7), while Many-shot accuracy increases from 14.85% to 22.57%.

From the mathematical operation of the classifier, prototype update and logit adjustment have somewhat similar effects. Since classification logits are computed from the inner products between feature vectors and class prototypes, changing prototype directions also changes/corrects the resulting logit scores.

However, the two mechanisms operate differently. Prototype update directly modifies the directions of class prototypes, whereas the logit adjustment in HyPFA-TTA corrects the computed logits according to the current batch-level class tendency.

Our design choice is to keep source class prototypes fixed as reference directions for target feature re-orientation, while using batch-level tendency-based logit adjustment to address classification score biases, rather than continuously modifying source prototypes during adaptation.

This choice is also supported by the experimental results. Configuration-(4), which uses fixed prototypes, achieves an overall accuracy of 41.92%, outperforming Configuration-(7), which allows prototype updates (41.09%). These results indicate that when normalization, logit adjustment, and feature alignment are combined, fixed source prototypes can provide effective reference directions for adaptation, without requiring additional prototype updates to achieve strong classification performance.

(7) The warm-up stage further improves the performance of the full model

Finally, we would like to connect these observations to the mathematical analysis of the CE loss and warm-up stage presented in Part-1.

The former seven configurations in Table 3(B) do not employ warm-up. According to the confidence upper bound established in Part-1, the CE loss is identically zero under these configurations. Therefore, even when CE is checked in the table, this does not imply that the CE objective contributes to adaptation.

This distinction is important because it establishes that the differences among these ablation configurations do not arise from the CE loss. Instead, they reflect the effects of normalization, logit correction, prototype alignment, and prototype update.

With normalization, logit correction, and feature alignment, Configuration (4) already achieves an overall accuracy of 41.92%. Adding the warm-up stage further improves the overall accuracy of the full HyPFA-TTA model to 42.94%, while Medium-shot accuracy increases from 19.14% to 20.77%.

These results indicate that the warm-up stage provides an additional improvement in overall TTA performance on top of the prototype-guided feature adaptation mechanism. However, as explained in Part-1, adaptation after warm-up is still driven by the prototype-alignment loss rather than continued model updates through the CE loss.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.