- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
Sparse Delta Learning (SDL) learns which parameter groups to update for each unlabeled target batch in online test-time adaptation. Group Split-LBI first identifies a budget-constrained sparse support, and the base adaptation optimizer then refines only the selected parameters. On VisDA-C with DeiT-S/16 and SHOT, SDL raises final mean per-class accuracy by 14.29 percentage points over the strongest budget-matched sparse baseline while updating at most 0.0213% of the model parameters per batch. These gains, however, depend on the adaptation objective, and support discovery incurs substantial computational overhead.
Critical strengths
-
A principled formulation of where to adapt. The paper casts the adaptation support as a variable learned from each incoming target batch under an explicit group budget. Its two-stage design decouples support discovery, performed by Group Split-LBI, from the subsequent parameter update, while preserving the underlying TTA objective. The same formulation extends to fully connected, convolutional, and Transformer parameters through architecture-specific grouping. The result is a coherent framework for studying how limited update capacity should be allocated—one that gives the contribution methodological reach across architectures.
-
Strong empirical evidence for the value of learned update locations. On VisDA-C with DeiT-S/16 and SHOT, SDL reaches 61.50% final mean per-class accuracy, exceeding the strongest budget-matched sparse baseline by 14.29 percentage points while updating at most 0.0213% of the model parameters per batch (Table 2). It also surpasses candidate-dense adaptation by a further 8.56 points and comes within 1.08 points of full-dense adaptation. Together these results indicate that learning the update support makes a small parameter budget considerably more effective, and they directly support the paper's central claim.
Critical weaknesses
-
The comparisons do not fully isolate the benefit of support selection. The reported gains are confounded with differences in optimization settings. For instance, the DeiT + SHOT controls use a learning rate of , whereas SDL's second stage runs at on VisDA-C with and also inherits the parameter increment produced by the first stage (Appendices A.4 and A.6). Absent controls that hold these settings fixed, the sizeable improvement cannot be confidently attributed to the quality of the learned support. This weakens the empirical case for the paper's central methodological claim.
-
Support discovery incurs substantial computational cost in online adaptation. In the Office-31 efficiency benchmark, SDL requires approximately 7.05 seconds per batch, making it 150–166 times slower than the sparse baselines; support discovery accounts for 99.22% of adaptation time (Appendix A.3). Its peak memory allocation also exceeds that of candidate-dense adaptation and of the other sparse methods. These costs considerably undercut the practical value of the small update footprint. Comparisons under matched compute or latency budgets are therefore needed to determine whether the accuracy gains justify the added online processing.
-
Outdated and limited baselines. The comparison set omits strong work from the past two years; the most recent competitive baseline, COME, is from ICLR two years ago, so the empirical positioning of the method is unclear.
-
Missing a standard benchmark. The evaluation does not include ImageNet-C, a widely used benchmark in this area, which prevents direct comparison with a large body of prior work.
-
No released code. No code is provided in the appendix or as supplementary material, which limits reproducibility and makes the reported gains difficult to verify.
Questions for authors
-
Clarification: Hyperparameter selection. Appendix A.6 states that configurations were selected according to target-domain FO accuracy. Were they chosen on an independent validation set, or on the same target data used to report the final results? Which target labels were available during selection, and did all compared methods receive equivalent tuning?
-
Evidence needed for assessment: Attribution of the accuracy gains. How much of SDL's improvement can be attributed to support selection, rather than to the first-stage parameter increment or to the different second-stage learning rates? Could you provide ablations or matched optimization controls that isolate these effects, particularly for the main VisDA-C + DeiT + SHOT result?
-
Evidence needed for assessment: Accuracy under comparable computational budgets. Given the reported 150–166× runtime overhead, does SDL retain its accuracy advantage when methods are given comparable adaptation time or gradient-evaluation budgets per batch? Such a comparison would materially inform the assessment of its practical value for online adaptation.