- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes Drift-TTA, a test-time adaptation framework that uses "hidden trajectories," the sequence of mean-pooled intermediate representations across layers, as an adaptation signal beyond output entropy. Motivated by Chain-of-Embedding, it trains zero-initialized LoRA adapters on a frozen ViT/CLIP backbone offline to align corrupted trajectories with cleaner targets, in a source-dependent setting (clean targets) and a source-free setting (the input is further corrupted with the same corruption family, and the less corrupted view serves as the target). An online variant adapts only scalar gates per adapter via entropy to calibrate the correction strength. A MAP interpretation and KL-based bounds are provided. Gains are clear on CIFAR-100-C, but on ImageNet-C with ViT-B/16 the method (60.2/60.7%) is below FOA, CMF, and PEA (66.1/65.9/64.5%), and it only ties BITTA with CLIP (Table 6).
Critical strengths
- A new adaptation signal. Aligning multi-layer hidden trajectories goes beyond output entropy and single-layer feature statistics, and Fig. 1(b) shows that corruption systematically displaces these trajectories.
- Separation of direction and strength. The correction direction is learned offline, while only 24-36 scalar gates are adapted online. This keeps test-time adaptation lightweight and stable, and it improves calibration over PEA and BITTA (Table 4).
Critical weaknesses
W1. The formulation assumes prior knowledge of the corruption process, which makes the comparison with TTA baselines unfair.
- The motivation relies on benchmark structure. The progressive severity-dependent drift (Fig. 1) reflects the five-level structure of the -C benchmarks. In real TTA, such as autonomous driving, the corruption type and severity are unknown.
- Training requires simulating the test corruption. The adapters are trained on corruptions synthesized at severities 1-5 from 10,000 clean training images, and this holds even for the "source-free" variant (Sec. 4.1). Its Worsen operator also requires a simulator of the same corruption family as the input (Sec. 3.2), so even source-free training depends on synthetic corruption operators.
- The baselines do not receive this supervision. TENT, SAR, FOA, PEA, and BITTA use only the unlabeled test stream. In the main tables, Drift-TTA also uses a separate adapter for each held-out family.
- Missing assumption-matched baselines. Fair comparisons would be input-level correction such as Decorruptor [1], or corruption-augmented training followed by standard TTA.
W2. The novelty is incremental, and the evidence is limited.
- Incremental novelty. The core idea transfers Chain-of-Embedding to TTA, and the remaining components (per-block LoRA, teacher-student alignment, entropy-tuned gates) are standard.
- Weak results on ImageNet-C. The method underperforms several baselines on ImageNet-C (Table 6).
- Single seed. All results use a single training seed.
- Table 2 does not compare methods. It only contrasts adaptation paradigms, so it does not replace an assumption-matched comparison with recent TTA methods.
In particular, the source-free variant learns only a relative, one-severity-step correction (from a worsened view back to the current corrupted view), which depends on the severity ordering of a corruption simulator and, as the authors note, only partially reverses the drift (Sec. 3.2). In contrast, methods such as Decorruptor [1] restore a corrupted input directly toward a clean one regardless of severity, and the paper does not demonstrate an advantage over such an approach.
[1] Oh et al. "Efficient diffusion-driven corruption editor for test-time adaptation.", ECCV 2024
Questions for authors
- Does the source-free variant need the test input's corruption family to apply Worsen? How would it handle shifts that cannot be simulated, such as ImageNet-R or ImageNet-V2?
- Does Drift-TTA still have an advantage over Decorruptor, or over AugMix-style training followed by TENT/SAR, when all methods are given the same synthetic-corruption supervision?
- In the single-adapter evaluation (Appendix D), how do baselines other than PEA perform?
2 comments
4. What the method contributes beyond its components (W2)
We disagree that the contribution is adequately described as transferring Chain of Embedding to TTA and adding standard components.
The contribution connects an observation about hidden trajectory drift to a correction objective and an adaptation procedure, while also providing theoretical analysis alongside it. The objective matches corresponding hidden states across network depth, preserves the reference behavior through the anchor term, and learns corrections while the backbone remains frozen (Section 3.2). We also propose an online version as well, in which adaptation then adjusts the strength of those learned corrections rather than learning their directions through entropy minimization (Section 3.3).
LoRA and entropy minimization are established tools. Their use does not remove the contribution of the formulation they implement. Also, our same proposed approach applies to both supervised ViTs and CLIP models, under source dependent and source-free preparation.
Figure 1 and Appendices F and H examine trajectory displacement and correction. Section 3.4 provides a probabilistic interpretation and analyzes the residual of the actual alignment objective. Together, these connect the motivating observation, the proposed algorithm, and the analysis. The theory’s scope and assumptions remain as explained in our response to Reviewer 7xrG, Point 6.
5. Decorruptor and corruption augmented baselines (W1, Question 2)
Decorruptor is a relevant comparison, but providing the same synthetic corruptions would not by itself match all assumptions. Decorruptor trains a separate diffusion editor using clean and corrupted image pairs (Oh et al., 2024, Sections 4.1, 4.2, and 5.1). Our source free objective uses no clean reference targets, and neither variant requires a separate generative model. ecva.net
The approaches also optimize different targets. Decorruptor learns to edit images toward clean targets. Our method learns to correct intermediate representations and decisions. A clean reconstruction target does not by itself establish an advantage in classification accuracy on unseen corruptions.
6. ImageNet accuracy and calibration (W2)
The supervised ImageNet C accuracy limitation is explicitly reported in Table 6. The source free and source dependent online variants achieve 60.2% and 60.7%, below FOA, CMF, and PEA. We agree that these results need discussion, but the assessment should also include the benefits demonstrated elsewhere.
On ImageNet C itself, the source dependent and source free online variants achieve mean ECEs of 13.03% and 14.72% for supervised ViT, compared with 33.44% for PEA, and 12.89% and 8.01% for CLIP, compared with 18.04% for BITTA (Table 12). The source free online variant also matches BITTA’s reported 31.9% mean CLIP accuracy.
As discussed in our response to Reviewer 7xrG, Point 5, calibration improvements deserve consideration alongside accuracy limitations. The contribution should be assessed on its methodological novelty, supporting observations, and demonstrated benefits. Outperforming every baseline on every benchmark is not a prerequisite for a meaningful contribution.
7. Baselines in Appendix D and statistical uncertainty (Question 3, W2)
Appendix D includes BITTA as well as PEA, alongside the unadapted models. Tables 7, 8, and 9 report the same adapter evaluated across four unseen corruption types.
On CIFAR 100 C, the source free online variant achieves 82.6% versus 79.5% for PEA with supervised ViT, and 54.5% versus 50.1% for BITTA with CLIP (Table 9).
On ImageNet C, the corresponding CLIP result is 39.8% versus 38.0% for BITTA. For supervised ViT, the source free and source dependent online variants achieve 67.2% and 68.4%, respectively, compared with 67.6% for PEA (Table 7).
The other adaptation baselines are not evaluated under this additional protocol. Their results cannot be inferred from their rankings on the standard corruption types.
Section 4.1 explicitly reports the single training seed. Appendix F separately provides confidence intervals for the association between trajectory drift and prediction error.
We appreciate the recognition of the hidden trajectory signal and the separation of correction direction from strength. The main point to clarify is that training on synthetic corruptions does not mean training on the corruption used for evaluation.
1. Training does not require simulating the test corruption (W1, Question 1)
The corruption type used for evaluation is excluded from adapter training. Section 4.1 trains each adapter on fourteen corruption types and evaluates it on the excluded fifteenth type. The simulator for that evaluated type is therefore not used to train that adapter.
The main tables aggregate separately trained adapters, as the reviewer notes. Furthermore, Appendix D additionally examines the deployment setting where one adapter is trained on fifteen types and that same adapter is applied to four unseen types, without selecting a different adapter for each test corruption.
These protocols directly test whether corrections learned from some corruption types transfer to others. Related corruption types can remain in training, so excluding an entire category would provide a stricter test, as discussed in our response to Reviewer CmqX, Point 2 and would be an interesting ablation. This distinction does not change the fact that the evaluated corruption type is excluded from training.
2. Source free training and unknown severity (W1, Question 1)
We use synthetic corruption operators during preparation, including to construct the corrupted training views in the benchmark experiments (Section 4.1). Source free refers to learning without clean reference targets or class labels, rather than claiming that the benchmark training views were collected from naturally corrupted data.
Section 3.2 constructs a further corrupted view using the same family as the current training image. The current image supplies the teacher trajectory and anchor. This construction does not require the simulator for the excluded evaluation corruption.
The description of this operation as a fixed “one severity step” correction is also too restrictive. Worsen produces a view with higher effective corruption; it is not defined as moving exactly one level along the benchmark’s numbered severity scale.
Neither the corruption family nor its severity is required during deployment. The learned adapters process the input directly, and online adaptation adjusts their strengths using prediction entropy (Section 3.3). It does not apply Worsen or construct reference pairs.
Similarly, the severity levels in Figure 1 provide a controlled way to study trajectory displacement. They are not inputs to the adaptation algorithm. Appendix F also examines the association between trajectory drift and prediction error after controlling for severity.
Applying the trained adapters to ImageNet R or ImageNet V2 would provide an interesting additional test beyond synthetic corruptions.
3. Prior data assumptions and the purpose of Table 2 (W1, W2)
The statement that all listed baselines use only the incoming test stream overlooks preparation in some of those methods. PEA computes and stores source feature statistics from training images before deployment (Ma et al., 2026, Section 4.1 and Appendix D.5.2). BITTA’s authors used subsets of training images to fit the relationship underlying their sample selection rule (Sun et al., 2026, Section 3.3 and Appendix E.1).
These procedures differ from training our adapters. The relevant distinction is how prior data are used, rather than whether any preparation occurs before deployment. Our training stage is explicitly disclosed, uses no class label supervision, and excludes evaluation images.
Table 2 compares adaptation paradigms and their data requirements. Its purpose is to explain the different variants and when their parameters are learned or adapted. It is not presented as an empirical comparison or as evidence that every baseline receives identical preparation.