- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studied frozen model, backpropagation-free test-time adaptation. In this paper, the authors argue that global target statistics and class-specific target statistics differ in their sample requirements, which it calls estimation support asymmetry. The paper reports a reliability reversal. A branch using the global logit statistics is more accuracte with few target samples, while a prototype-based branch becomes better as samples accumulate. EXACT swithes from to once the sample count reaches which is the coupon-collector expectation under uniform classes. It achieves the best average on seven benchmarks across ViT/CLIP and online/transductive settings, and improves 35 additional backbones.
Critical strengths
-
The paper is well-written and the motivation is clear. Also, the method is simple, trainging-free, and nearly hyperparameter-free.
-
The evaluation is broad, covering seven benchmarks, two protocols, and 35 extra backbones, with consistent gains.
-
The diagnostics in Figures 3 and 4 are informative, and the authors are candid that is not an exact crossover.
Critical weaknesses
W1. The analysis compares the estimation variance of each statistic around its own population value and never relates it to accuracy. A “variance only” argument can’t produce a crossover, since it favors the global statistic at every . The large advantage of requires a bias term, such as the global centering being unable to correct class-conditional shifts, and no such term is given.
W2. In the transductive setting always holds, so EXACT reduces to . On the online experiments, the switch occurs early in the stream, at about on CIFAR-100-C and on ImageNet-C, which means that most prediction come from . On ImageNet-C, most of the gain comes from alone. Figure 12 confounded in the same way, since replacing early predictions of any method with a stronger predictor would improve it.
Questions for authors
Q1. The paper attributes the small failure of to limited estimation support, but Figure 4 shows only that absent class weight and prototype error decrease together with . Could the authors run two interventions at fixed , (a) building from oracle-label prototypes and (b) restricting to classes with non-negligible soft count, and report whether the collapse persists?
Q2. Two components could cause the small collapse independently of support. The weight normalization in Eq. (14) turns every absent-class prototype into a full-magnitude average of other classes. The balanced Sinkhorn in Eqs. (12) and (16) enforces a uniform marginal even when . Could the authors report Always accuracy versus , as in Figure 3, with the Sinkhorn steps replaced by softmax and with prototypes initialized from the classifier weights as in T3A?
Q3. TDA (Karmanov et al., CVPR 2024) builds class-wise feature caches. It is training-free, backpropagation-free, and source-free, so it appears to meet the baseline criteria stated in App. A.3, and it is cited in the paper. The reviewer would like it to be added to Table 1.