acceptodds
Withdrawn from ICLR 2027

Evaluate Before You Commit: Label-Free Candidate-Update Utility for Test-Time Adaptation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Test-time adaptation (TTA) can improve performance under distribution shift, yet deciding where to adapt does not determine whether the resulting parameter update is worth retaining. We formulate candidate-update utility: after a base adaptation operator produces a tentative state, should that realized update enter persistent model state? We introduce a label-free evaluate-before-commit controller that allocates candidate-generation compute using forward-only screening, evaluates each tentative update on independently sampled probe views using held-out predictive consistency and drift from a frozen pretrained reference, and transactionally rolls back low-utility candidates. Across CLIP and DINOv2 under synthetic corruption, natural domain shift, and a cross-hospital stress test, the proposed utility estimator improves harmful-update AUROC from 0.589 to 0.750 and AUPRC from 0.305 to 0.492 relative to predictive entropy. At the principal operating point, Utility-Aware control reduces the harmful committed-update rate (HCUR) from 27.0% to 11.0% while increasing the accuracy gain from +2.20 to +2.40 percentage points, halving candidate-update backward passes, and reducing adaptation-specific wall-clock time by 28±3%. Post-proposal verification also complements strong pre-update selection: adding Utility-Aware verification after Cross-Augmentation Similarity (CAS) reduces HCUR from 15.2% to 8.2%, while the proposed score achieves next-window harmful-update AUROC of 0.738 versus 0.641 for CAS. These results support pre-update selection and post-proposal candidate commitment as distinct and complementary control levels for TTA.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Test-Time Adaptation, Distribution Shift, Selective Adaptation, Candidate-Update Utility, Label-Free Adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_9owkOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies the acceptance of candidate updates in test-time adaptation (TTA): even when a test input warrants an adaptation attempt, the resulting parameter update may not be worth retaining. The authors propose a four-stage pipeline that screens inputs, generates candidate updates, evaluates them, and either commits or rolls them back. First, predictive entropy and quota rules are used to allocate the budget for generating candidate updates. Next, one step of entropy minimization is performed on the update views. Finally, independently augmented views of the same batch of original images are used to compare prediction consistency before and after the update, with an additional penalty for prediction drift relative to a frozen pretrained model. The candidate parameters and optimizer state for the current batch are committed only if the acceptance score exceeds a threshold; otherwise, the pre-adaptation state is restored. The paper reports experiments on datasets including ImageNet-C and Office-Home, using models such as CLIP and DINOv2. The improvements over the baselines are limited.

Critical strengths

The paper addresses a practically important problem. TTA updates can degrade model performance or even cause model collapse, making it necessary to evaluate the updates themselves. Compared with the pre-update sample filtering commonly used in prior TTA methods, this paper introduces a mechanism for accepting or rejecting candidate updates. This is a general strategy that could potentially be integrated into existing TTA methods to improve their robustness.

Critical weaknesses
  1. The experimental results do not provide sufficient evidence that the proposed method achieves state-of-the-art performance. The most recent baseline is SAR, published in 2023, and the average improvement over SAR is less than 0.20.2%. These gains are marginal, and comparisons with more recent methods are missing.

  2. The paper frequently uses defensive formulations such as "rather than," making the writing difficult to follow and disrupting its flow. The main figure consists almost entirely of text, making it difficult to understand the rationale behind the method's design.

  3. The proposed gating mechanism has limited novelty. Multi-view consistency across augmented views and regularization that penalizes drift from a pretrained model are common components of TTA methods. Although the paper explicitly acknowledges that consistency alone does not guarantee correctness, it still uses consistency on held-out views as the basis for gating, without providing a theoretical justification for this choice.

  4. The experimental assessment of harmful updates considers only samples in the current batch. It does not account for the long-term effects of TTA updates on predictions for subsequent samples. Therefore, the reported reduction in harmful updates does not directly establish the effectiveness of the gating mechanism.

Questions for authors
  1. Could you include comparisons with more recent baselines, particularly the latest TTA methods, to better demonstrate the effectiveness of the proposed method?

  2. Why do you adopt the gating mechanism despite explicitly acknowledging that it does not guarantee correctness? Could you provide theoretical or experimental evidence to justify this choice?

  3. Equation (6) implies that when the number of probe views is M = 1, inter-view disagreement should always be zero. Consequently, all candidate updates would be rejected, and the model should remain in its original state. This is inconsistent with the gain of 2.04 reported in Appendix Table 12. Could you explain this discrepancy?

Reviewer_xvYVOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper addresses the stateful risks of Test-Time Adaptation (TTA), where retaining a low-quality parameter update can degrade all subsequent predictions and updates across a test trajectory. It formalizes a conceptual separation between two distinct control decisions: Where to adapt (pre-update screening) and Whether the update should persist (post-proposal verification). It also proposes label-free utility score to balance held-out predictive consistency gains against candidate drift away from a frozen pretrained reference. The method is evaluated on CLIP ViT-B/16 and DINOv2 ViT-B/14 across ImageNet-C, Office-Home, and Camelyon17-WILDS using mechanism-matched controls, pre-update CAS baselines, and retrospective oracle headroom analysis.

Critical strengths
  • The idea of separate deciding whether an observation should trigger adaptation from deciding whether the particular state sounds novel.
  • The paper carefully designs mechanism-matched controls to further investigate the same base candidate generator, target stream, adaptable parameters, optimizer, learning rate, and update objective.
  • The statistical protocol is designed fairly which uses 10,000 paired bootstrap replicates with hierarchical resampling across shift units.
Critical weaknesses
  • The claim on computational cost and memory reduction should be explicitly stated that the reason is because it is restricted to LayerNorm parameters, and might not scale easily to full fine-tuning without techniques like LoRA.
  • The framework relies heavily on predictive entropy and consistency across views. Under severe label shift (e.g., real-world streams), entropy minimization often collapses the model into predicting the majority class confidently. Will the frozen anchor save the model, or will it penalize valid adaptation?
  • The framework is tested exclusively on a single base operator: one AdamW step of entropy minimization over LayerNorm affine parameters. It remains unproven whether candidate verification behaves similarly for multi-step gradient updates, sharpness-aware optimizers (e.g., SAR), or prompt-tuning mechanisms.
  • While the related work discusses reset-based continual TTA methods such as ASR (Lim et al., 2026) and DO-ALL (Jang et al., 2026), these methods are omitted from empirical comparisons.
Questions for authors

Please see weaknesses

Reviewer_q47hOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes an evaluate-before-commit controller for test-time adaptation (TTA). An entropy score first decides which windows receive a candidate update. Each candidate is then scored on held-out augmented views by its gain in prediction consistency minus its added drift from the frozen pretrained model, and it is kept only if the score exceeds a margin; otherwise the model and optimizer states are rolled back. Experiments use CLIP and DINOv2 on ImageNet-C, Office-Home and Camelyon17-WILDS with a one-step LayerNorm entropy-minimization update, and report a lower rate of harmful committed updates (HCUR) than always adapting, with a small accuracy gain.

Critical strengths

I cannot find any.

Critical weaknesses
  1. The key evidence is unreliable. The component ablation in Table 9 is inconsistent with the rest of the paper: "Screening only" should be identical to Entropy Gate but reports different numbers, and the gap between "+ Frozen Anchor" and "Full" cannot be explained by any remaining term of the method. The main detection result (Table 1) is also not a fair comparison. The proposed score looks at the model after the update, while all baselines only see the model before it, so it is expected to predict harm better. No post-update baseline is given.

  2. The contribution is thin and the gains are marginal. The method is a single thresholded score, and Proposition 1 follows directly from its own assumption. Accuracy improves by at most 0.2 pp over Always Adapt and SAR, with no variance or absolute accuracy reported, and the large improvements appear only in HCUR, the metric the gate is built to reduce. Recent CLIP TTA methods and the closest rollback methods (ASR, DO-ALL) are not compared. Much of the 9-page main text and 30-page appendix restates the same caveats, which makes the actual contribution hard to find.

  3. Presentation is poor. The paper keeps repeating meaningless sentences such as "xxx, not as a correctness oracle", and is hardly readable. The paper is apparently below ICLR standard.

Questions for authors
  1. Which configuration does each row of Table 9 correspond to, and why does "Screening only" differ from Entropy Gate?
  2. How does the proposed score compare with simple post-update baselines, such as the entropy change on the probe views?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.