acceptodds
Under review as a conference paper at ICLR 2027

Hidden Trajectories Don't Lie: Correcting Corruption Drift for Test-Time Adaptation

Abstract

Test-time adaptation (TTA) adapts pretrained models to distribution shifts encountered during deployment. However, most existing methods rely primarily on the weight representations of the final layer or signals derived from the model’s output probability distribution. In this work, we study hidden trajectories: the ordered sequence of intermediate representations produced across a deep neural network. We observe that clean and corrupted trajectories become more separable as corruption severity increases. Based on this observation, we introduce Drift-TTA, which adds zero-initialized low-rank adapters to a frozen backbone and trains them to align corrupted hidden trajectories with those from cleaner inputs. The proposed approach supports two variants: a source-dependent version trained with clean targets, and a source-free setting that learns from corrupted images at different severities. Building on these two variants, we further develop an online TTA approach that freezes the backbone and adapters while updating a scalar gate to control adaptation strength. We further cast trajectory alignment as MAP inference under a Gaussian observation model and derive a KL-based transfer bound characterizing test-time trajectory misalignment in terms of offline residual error, residual dimensionality, and distribution shift between offline and test corruptions. We evaluate Drift-TTA on CIFAR-10-C, CIFAR-100-C, and ImageNet-C using supervised ViTs and CLIP models. % On CIFAR-100-C at severity 5, our source-free online approach reaches 74.9% accuracy with ViT-B/16 and 47.4% with CLIP ViT-B/16, improving over no TTA by 13.3 and 11.6 percentage points, respectively. On CIFAR-100-C at severity 5, our source-free online approach improves accuracy over no TTA by 13.3 percentage points with ViT-B/16 and 11.6 percentage points with CLIP ViT-B/16, reaching 74.9% and 47.4%, respectively. Under the same setting, prior TTA baselines yield ECEs of 18.00% for ViT and 14.70% for CLIP, while our approach reduces them to 9.53% and 5.77%, respectively. These results demonstrate that hidden trajectory alignment provides an effective signal for both offline correction and online test-time adaptation.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
test-time adaptation, distribution shift, corruption robustness, intermediate representations, representation alignment, hidden trajectories, low-rank adaptation, source-free adaptation, online adaptation, model calibration

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_rP6AOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Drift-TTA, a test-time adaptation framework that uses "hidden trajectories," the sequence of mean-pooled intermediate representations across layers, as an adaptation signal beyond output entropy. Motivated by Chain-of-Embedding, it trains zero-initialized LoRA adapters on a frozen ViT/CLIP backbone offline to align corrupted trajectories with cleaner targets, in a source-dependent setting (clean targets) and a source-free setting (the input is further corrupted with the same corruption family, and the less corrupted view serves as the target). An online variant adapts only scalar gates per adapter via entropy to calibrate the correction strength. A MAP interpretation and KL-based bounds are provided. Gains are clear on CIFAR-100-C, but on ImageNet-C with ViT-B/16 the method (60.2/60.7%) is below FOA, CMF, and PEA (66.1/65.9/64.5%), and it only ties BITTA with CLIP (Table 6).

Critical strengths
  1. A new adaptation signal. Aligning multi-layer hidden trajectories goes beyond output entropy and single-layer feature statistics, and Fig. 1(b) shows that corruption systematically displaces these trajectories.
  2. Separation of direction and strength. The correction direction is learned offline, while only 24-36 scalar gates are adapted online. This keeps test-time adaptation lightweight and stable, and it improves calibration over PEA and BITTA (Table 4).
Critical weaknesses

W1. The formulation assumes prior knowledge of the corruption process, which makes the comparison with TTA baselines unfair.

  • The motivation relies on benchmark structure. The progressive severity-dependent drift (Fig. 1) reflects the five-level structure of the -C benchmarks. In real TTA, such as autonomous driving, the corruption type and severity are unknown.
  • Training requires simulating the test corruption. The adapters are trained on corruptions synthesized at severities 1-5 from 10,000 clean training images, and this holds even for the "source-free" variant (Sec. 4.1). Its Worsen operator also requires a simulator of the same corruption family as the input (Sec. 3.2), so even source-free training depends on synthetic corruption operators.
  • The baselines do not receive this supervision. TENT, SAR, FOA, PEA, and BITTA use only the unlabeled test stream. In the main tables, Drift-TTA also uses a separate adapter for each held-out family.
  • Missing assumption-matched baselines. Fair comparisons would be input-level correction such as Decorruptor [1], or corruption-augmented training followed by standard TTA.

W2. The novelty is incremental, and the evidence is limited.

  • Incremental novelty. The core idea transfers Chain-of-Embedding to TTA, and the remaining components (per-block LoRA, teacher-student alignment, entropy-tuned gates) are standard.
  • Weak results on ImageNet-C. The method underperforms several baselines on ImageNet-C (Table 6).
  • Single seed. All results use a single training seed.
  • Table 2 does not compare methods. It only contrasts adaptation paradigms, so it does not replace an assumption-matched comparison with recent TTA methods.

In particular, the source-free variant learns only a relative, one-severity-step correction (from a worsened view back to the current corrupted view), which depends on the severity ordering of a corruption simulator and, as the authors note, only partially reverses the drift (Sec. 3.2). In contrast, methods such as Decorruptor [1] restore a corrupted input directly toward a clean one regardless of severity, and the paper does not demonstrate an advantage over such an approach.

[1] Oh et al. "Efficient diffusion-driven corruption editor for test-time adaptation.", ECCV 2024

Questions for authors
  1. Does the source-free variant need the test input's corruption family to apply Worsen? How would it handle shifts that cannot be simulated, such as ImageNet-R or ImageNet-V2?
  2. Does Drift-TTA still have an advantage over Decorruptor, or over AugMix-style training followed by TENT/SAR, when all methods are given the same synthetic-corruption supervision?
  3. In the single-adapter evaluation (Appendix D), how do baselines other than PEA perform?
2 comments

We appreciate the recognition of the hidden trajectory signal and the separation of correction direction from strength. The main point to clarify is that training on synthetic corruptions does not mean training on the corruption used for evaluation.

1. Training does not require simulating the test corruption (W1, Question 1)

The corruption type used for evaluation is excluded from adapter training. Section 4.1 trains each adapter on fourteen corruption types and evaluates it on the excluded fifteenth type. The simulator for that evaluated type is therefore not used to train that adapter.

The main tables aggregate separately trained adapters, as the reviewer notes. Furthermore, Appendix D additionally examines the deployment setting where one adapter is trained on fifteen types and that same adapter is applied to four unseen types, without selecting a different adapter for each test corruption.

These protocols directly test whether corrections learned from some corruption types transfer to others. Related corruption types can remain in training, so excluding an entire category would provide a stricter test, as discussed in our response to Reviewer CmqX, Point 2 and would be an interesting ablation. This distinction does not change the fact that the evaluated corruption type is excluded from training.

2. Source free training and unknown severity (W1, Question 1)

We use synthetic corruption operators during preparation, including to construct the corrupted training views in the benchmark experiments (Section 4.1). Source free refers to learning without clean reference targets or class labels, rather than claiming that the benchmark training views were collected from naturally corrupted data.

Section 3.2 constructs a further corrupted view using the same family as the current training image. The current image supplies the teacher trajectory and anchor. This construction does not require the simulator for the excluded evaluation corruption.

The description of this operation as a fixed “one severity step” correction is also too restrictive. Worsen produces a view with higher effective corruption; it is not defined as moving exactly one level along the benchmark’s numbered severity scale.

Neither the corruption family nor its severity is required during deployment. The learned adapters process the input directly, and online adaptation adjusts their strengths using prediction entropy (Section 3.3). It does not apply Worsen or construct reference pairs.

Similarly, the severity levels in Figure 1 provide a controlled way to study trajectory displacement. They are not inputs to the adaptation algorithm. Appendix F also examines the association between trajectory drift and prediction error after controlling for severity.

Applying the trained adapters to ImageNet R or ImageNet V2 would provide an interesting additional test beyond synthetic corruptions.

3. Prior data assumptions and the purpose of Table 2 (W1, W2)

The statement that all listed baselines use only the incoming test stream overlooks preparation in some of those methods. PEA computes and stores source feature statistics from training images before deployment (Ma et al., 2026, Section 4.1 and Appendix D.5.2). BITTA’s authors used subsets of training images to fit the relationship underlying their sample selection rule (Sun et al., 2026, Section 3.3 and Appendix E.1).

These procedures differ from training our adapters. The relevant distinction is how prior data are used, rather than whether any preparation occurs before deployment. Our training stage is explicitly disclosed, uses no class label supervision, and excludes evaluation images.

Table 2 compares adaptation paradigms and their data requirements. Its purpose is to explain the different variants and when their parameters are learned or adapted. It is not presented as an empirical comparison or as evidence that every baseline receives identical preparation.

4. What the method contributes beyond its components (W2)

We disagree that the contribution is adequately described as transferring Chain of Embedding to TTA and adding standard components.

The contribution connects an observation about hidden trajectory drift to a correction objective and an adaptation procedure, while also providing theoretical analysis alongside it. The objective matches corresponding hidden states across network depth, preserves the reference behavior through the anchor term, and learns corrections while the backbone remains frozen (Section 3.2). We also propose an online version as well, in which adaptation then adjusts the strength of those learned corrections rather than learning their directions through entropy minimization (Section 3.3).

LoRA and entropy minimization are established tools. Their use does not remove the contribution of the formulation they implement. Also, our same proposed approach applies to both supervised ViTs and CLIP models, under source dependent and source-free preparation.

Figure 1 and Appendices F and H examine trajectory displacement and correction. Section 3.4 provides a probabilistic interpretation and analyzes the residual of the actual alignment objective. Together, these connect the motivating observation, the proposed algorithm, and the analysis. The theory’s scope and assumptions remain as explained in our response to Reviewer 7xrG, Point 6.

5. Decorruptor and corruption augmented baselines (W1, Question 2)

Decorruptor is a relevant comparison, but providing the same synthetic corruptions would not by itself match all assumptions. Decorruptor trains a separate diffusion editor using clean and corrupted image pairs (Oh et al., 2024, Sections 4.1, 4.2, and 5.1). Our source free objective uses no clean reference targets, and neither variant requires a separate generative model. ecva.net

The approaches also optimize different targets. Decorruptor learns to edit images toward clean targets. Our method learns to correct intermediate representations and decisions. A clean reconstruction target does not by itself establish an advantage in classification accuracy on unseen corruptions.

6. ImageNet accuracy and calibration (W2)

The supervised ImageNet C accuracy limitation is explicitly reported in Table 6. The source free and source dependent online variants achieve 60.2% and 60.7%, below FOA, CMF, and PEA. We agree that these results need discussion, but the assessment should also include the benefits demonstrated elsewhere.

On ImageNet C itself, the source dependent and source free online variants achieve mean ECEs of 13.03% and 14.72% for supervised ViT, compared with 33.44% for PEA, and 12.89% and 8.01% for CLIP, compared with 18.04% for BITTA (Table 12). The source free online variant also matches BITTA’s reported 31.9% mean CLIP accuracy.

As discussed in our response to Reviewer 7xrG, Point 5, calibration improvements deserve consideration alongside accuracy limitations. The contribution should be assessed on its methodological novelty, supporting observations, and demonstrated benefits. Outperforming every baseline on every benchmark is not a prerequisite for a meaningful contribution.

7. Baselines in Appendix D and statistical uncertainty (Question 3, W2)

Appendix D includes BITTA as well as PEA, alongside the unadapted models. Tables 7, 8, and 9 report the same adapter evaluated across four unseen corruption types.

On CIFAR 100 C, the source free online variant achieves 82.6% versus 79.5% for PEA with supervised ViT, and 54.5% versus 50.1% for BITTA with CLIP (Table 9).

On ImageNet C, the corresponding CLIP result is 39.8% versus 38.0% for BITTA. For supervised ViT, the source free and source dependent online variants achieve 67.2% and 68.4%, respectively, compared with 67.6% for PEA (Table 7).

The other adaptation baselines are not evaluated under this additional protocol. Their results cannot be inferred from their rankings on the standard corruption types.

Section 4.1 explicitly reports the single training seed. Appendix F separately provides confidence intervals for the association between trajectory drift and prediction error.

Reviewer_CmqXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces a new TTA method, called Drift-TTA, based on chain of embeddings. The chain of embedding simply is the collection of embeddings at the different layers of the transformer, and previous work has shown that accuracy, and the angle/magnitude of the embeddings at sequential layers, are correlated. The TTA method adds LoRA adapters within transformer blocks, where the training objective is to have a student model's representations match the teacher model's representations at each layer, using an anchor objective as regularization.

The paper includes 4 variations of Drift-TTA, an offline/online variation, and source-free/-dependent variations of those. The online variations performs inference on the input batch, as soon as it receives it, whereas the offline variation does inference taking all test data into account. The source-free variation doesn't receive clean images, but corrupts already corrupted images even more, whereas the source-dependent variation receives clean images, without labels.

The methods show substantial improvements on CIFAR-100-C and CIFAR-10-C, compared to other state-of-the-art TTA methods, both in accuracy and calibration.

Critical strengths
  1. The paper uses a simple method to adapt to corrupted inputs at test-time. It seems like an interesting method to align embeddings between cleaner and corrupted data, though quite a few TTA methods already align embeddings during test-time.

  2. The paper offers some theoretical bounds and theoretical interpretations of their method. Though I wasn't able to check the correctness of proofs by hand.

  3. The improvements the method shows in CIFAR-C benchmarks are substantial and significantly exceed state-of-the-art in many cases.

Critical weaknesses
  1. The methods that are compared against mostly use different assumptions around what data is available during inference. Drift-TTA assumes having some data to train on before performing inference. To the best of my knowledge none or few of the TTA methods compared against, do this.

  2. The source-free method uses corrupted training data, and corrupts it even more using a function that is from the same family as the corrupted image. During evaluation, the corruption that is being evaluated is left out from the training set, using only the other 14 corruptions. But the other 14 corruptions will still contain very similar corruptions, such as different blur corruptions. There is no experiment showing how leaving out similar corruptions such as all blurs affects the results.

  3. There is no clear link between the angle and magnitude, as introduced in the beginning of the paper, and the main method. Ang(H) and Mag(H) are only mentioned in the introduction and defined in the beginning of section 3. But I do not see any mention of them in the experiments, results or theoretical analysis. This adds confusion and shows no strong link between the method and its motivation.

  4. The results in the appendix are weaker than the results in the main text. On ImageNet-C, Drift-TTA is one the weakest methods when used with a ViT. There is no explanation or analysis of this in the main text. Additionally, source-free Drift-TTA does better than source-dependent Drift-TTA on ImageNet-C with CLIP, which seems counter-intuitive, and isn't explained.

  5. There are no ablation experiments or further experiments analyzing the inner workings of the method, in the main text. There are no ablations about the hyper-parameters of the method, which layers are most important for increasing accuracies, or how embeddings change as a result of the LoRA adapters. Additionally, there are no confidence intervals on the main results.

Questions for authors

How does the corruption method work exactly that is used by the source-free version? How would it translate to non-synthetic corruptions?

How does this method transfer to non-synthetic corruptions?

Which layers are most needed for this method to work?

How do angle and magnitude correlate with accuracy and calibration?

How many samples are needed for training this method? Can it work on just a batch of training data?

2 comments

We would like to thank the reviewer for their thoughtful review and recommendations. We address the data assumptions, the connection between our observations and the method, and the additional analyses requested below.

1. Offline preparation and inference (Weakness 1)

First, the frozen variant does not require collecting or processing all test data before inference. Its adapters are trained beforehand on images from the training split, with no evaluation images used in training (Section 4.1). Once trained, it can process each test image directly. The online variant of our proposed method additionally adjusts correction strengths on each arriving batch (Section 3.3).

Furthermore, prior data use is also not unique to our method. PEA computes and stores source feature statistics from training images before deployment (Ma et al., 2026, Section 4.1 and Appendix D.5.2). BITTA’s authors used subsets of training images to fit the relationship underlying their sample selection rule (Sun et al., 2026, Section 3.3 and Appendix E.1). These examples show that information obtained from additional data before deployment can be part of a TTA method.

The methods differ in how they obtain and use this information. We do agree that these differences deserve consideration, but the existence of an offline stage alone does not invalidate online TTA. Our adapter training uses representation and decision matching without class label supervision, while online adaptation uses unlabeled test inputs. Table 2 explicitly distinguishes preparation from adaptation during deployment.

2. Unseen corruption types versus entire categories (Weakness 2)

Excluding an entire corruption category would provide a valuable ablation study. Our existing experiments examine transfer to corruption types excluded from training.

Section 4.1 describes training on fourteen types and evaluating on the excluded fifteenth type. Appendix D additionally trains one adapter on fifteen types and evaluates that same adapter on four unseen types (Tables 7, 8, and 9).

We do agree that the suggested experiments are useful extensions to the generalization questions examined by the existing experiments.

3. Connection between trajectory geometry and the method (Weakness 3)

Mag(H) and Ang(H) are used in Figure 1(b) and Appendix H, which show their distributions across corruption severities, models, datasets, and training variants, before and after correction. Their role is diagnostic: they reveal how corruption changes hidden trajectories and help visualize the resulting corrections.

Two scalar summaries cannot fully describe the sequence of hidden states. We therefore align the states themselves across layers, instead of optimizing those summaries. For two nonzero states uu and vv, with angle θ\theta between them, squared alignment distance satisfies

∥u−v∥2=(∥u∥−∥v∥)2+2∥u∥∥v∥(1−cos⁡θ).\lVert u−v\rVert^2 = (\lVert u\rVert−\lVert v\rVert)^2 + 2\lVert u\rVert\lVert v\rVert(1−\cos\theta).

Therefore, state matching penalizes the differences in both magnitude and direction. This explains the geometry of the objective without equating it with direct optimization of Mag(H) or Ang(H).

Also, Appendix F further quantifies the association between relative trajectory drift and prediction error. Across corruption and severity conditions, Pearson correlations are 0.73 on CIFAR 100 C and 0.78 on ImageNet C, and the association remains after controlling for severity. Calibration improvements are evaluated separately in Table 4 and Appendix E.

The connection is therefore from observing trajectory changes, to aligning hidden states, to measuring the resulting correction. Section 3.4 analyzes the residual of this same alignment objective. We do agree that this progression can be made clearer in the main text.

4. ImageNet results and the source free versus source dependent comparison (Weakness 4)

Table 6 reports the supervised ViT results on ImageNet C. The source free and source dependent online variants achieve 60.2% and 60.7%, compared with 55.5% without adaptation and 64.5% for PEA. They improve over the original model but fall below several adaptation baselines. We can make this tradeoff more prominent in the main text. The corresponding calibration benefits are discussed in our response to Reviewer 7xrG, Point 5, with results in Tables 4 and 12.

For CLIP on ImageNet C, source free online adaptation reaches 31.9%, compared with 31.5% for source dependent online adaptation (Table 6). Access to clean targets does not guarantee better accuracy on an unseen corruption type. The two variants learn from different reference trajectories and anchors. Their objectives optimize representation and decision matching, rather than classification error on the unseen test corruption. Source dependent training therefore does not always provide an upper bound on source free performance.

5. Representation analyses, ablations, and uncertainty (Weakness 5)

The paper already examines how the representations change. Figure 1 shows trajectory correction, Figure 3 shows class structure, Appendix G provides additional UMAP and t SNE visualizations, and Appendix H shows trajectory feature distributions before and after adaptation. Bringing more of this material into the main text would make the analysis easier to assess.

Our preliminary experiments consistently favored applying LoRA across all layers over restricting it to the final third. These results support our adapter placement, and including this comparison would provide further validation.

We agree that repeated runs would strengthen the statistical characterization of the reported improvements. Appendix F separately reports confidence intervals for the correlation analysis; these concern the association between trajectory drift and prediction error.

6. Source free training, natural shifts, and sample requirements (Questions for Authors)

In source free training, the current corrupted image supplies the teacher trajectory and anchor. We deliberately corrupt it further using the same corruption family used during training, then train the adapters to move its representations back toward those of the current image (Section 3.2). The objective uses neither a clean reference nor class labels. Section 4.1 explains that the benchmark training views are synthesized from images in the training split.

The worsening operator is used during training, not deployment. Applying a trained adapter to a new image requires neither identifying its corruption nor constructing a cleaner reference. Evaluating on natural shifts such as ImageNet R or ImageNet V2 would provide a valuable additional test of transfer beyond the synthetic corruption setting studied here.

Our reported experiments use 10,000 training images per dataset, separate from the evaluation images (Section 4.1 and Appendix A.2). This is the training budget, rather than a minimum requirement. We agree that comparing smaller subsets, including a single batch, would be a valuable addition to characterize the method’s data efficiency.

Reviewer_7xrGOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a new test-time adaptation method that learns LoRA adapters at different layers to align the hidden representation of the retrained model when subjected to distribution shifts. The proposed method; Drift-TTA has different variants depending whether the adaptation is done online/offline and whether data from the source domain is available or not. Experiments were carried out on CIFAR10/100 -C and ImageNet-C benchmarks.

Critical strengths

The main strengths of this work are:

  1. The problem this paper tackles is important and practical.

  2. Analysis and adaptation beyond the output logic; e.g. looking at representation space, is intuitive and interesting.

  3. The authors released an anonymized link to their codebase for reproducibility.

Critical weaknesses

Despite the paper's strengths, the paper has the following critical weaknesses:

  1. The assumptions under which the Drift-TTA is built can violate the TTA settings, dis-validating is from being one.

1a) The non-online version of the proposed method should not be called a TTA method, as it requires offline training of the LoRA parameters.

1b) Further, the proposed method requires knowing the corruption function C\mathcal C a-priori (line 307) violating the TTA setting. Not only that, the premise of TTA is to adapt the model to changing conditions that we cannot anticipate. If one knows the function CC, it is simpler to include it in the training objective, or treat the problem as a domain adaptation problem.

1c) The source free training setting replaces the function CC with a vague "Worsen" without detailing how is this function selected and what impact do different choices of this function have on the performance.

  1. Even if we go with DriftTTA-Online (SF) as the proposed TTA method, Its performance underperforms the reported baselines (74.9 vs 75.7 for PEA on CIFAR100). Further, the efficiency of the proposed method is not discussed, especially that it relies on teacher student schemes.

  2. The main paper reports mainly the results on CIFAR100 since one variant of Drift-TTA performs well. However, on the reported results on the more standard benchmark ImageNet-C shows that the proposed method significantly underperforms several baselines from the literature (I am referring to the results in Table-6 in the appendix). This demonstrates the ineffectiveness of the proposed method.

  3. The writing of the paper can be significantly improved along with the presentation of the experiments. Here are a few points in that regard:

4a) The notation can be significantly simplified. For instance L\mathcal L was overloaded as it was used in Equation (1) as a loss function, and in line 263 as the selected linear maps.

4b) Consider removing "subsection" from 2.1 as there is no 2.2 in the paper.

4c) Consider removing "Drift-TTA" from Table 1 (The offline version) as it is not a valid TTA method.

4d) The theoretical analysis in section 3.4 seem quite orthogonal with no impact on the design choices of the proposed method, nor on any of the experimental analysis. Consider moving them to the appendix.

4e) The experiments section in the paper should report the more significant results (e.g. experiments on ImageNet-C) first.

Questions for authors

Please refer to the Weaknesses section

3 comments

Thank you for recognizing the importance of the problem and the value of studying intermediate representations. Below, we clarify the adaptation setting, the evaluation on unseen corruptions, and the results relevant to each concern.

1. Offline training and the TTA setting (1a, 4c)

Drift-TTA separates learning a correction direction from adapting its strength. Both variants first train low-rank adapters through trajectory alignment. The frozen variant applies these corrections directly, while Drift-TTA-online adjusts their strengths using unlabeled test inputs.

We agree that the frozen variant performs offline correction, and its labeling should make this distinction clear. However, this does not invalidate the online formulation as TTA. As described in Section 3.3, it optimizes scalar gates on each arriving test batch and resets them afterward, following Definition 2.1. The backbone and adapter weights remain frozen. Training the correction directions beforehand does not change the fact that their strengths are adapted during deployment. Table 2 explicitly distinguishes these stages.

2. Knowledge of the test corruption (1b)

The corruption operator in Section 3.2 constructs offline training pairs. It does not mean that the corruption encountered during deployment is known beforehand.

We test generalization to unseen‾\underline{\textbf{unseen}} corruption families through two protocols:

  • Leave-one-family-out: Each adapter is trained on fourteen families and evaluated on the excluded fifteenth family (Section 4.1). The evaluated family is therefore absent from that adapter’s training.
  • One adapter trained, then tested on four unseen families: A single adapter is trained on the fifteen standard families and evaluated on four additional families: speckle noise, Gaussian blur, spatter, and saturate (Appendix D).

The second protocol also addresses the limitation that the leave-one-family-out tables combine separately trained adapters. Here, the same adapter handles all four unseen families, without selecting a different adapter for each test corruption.

On CIFAR-100-C, this single source-free online adapter achieves 82.6% accuracy versus 79.5% for PEA, and 54.5% versus 50.1% for BITTA, for supervised ViT and CLIP, respectively (Table 9). These results demonstrate transfer beyond the families used during training. They do not establish generalization to every possible shift, but they directly address the claim that the actual test corruption must be known in advance.

3. The source-free worsening operation (1c)

The idea is simple: we start with an already-corrupted image and deliberately make it worse. We then train the adapters to move its hidden representations back toward those of the original, less-corrupted image. Neither image needs to be clean, and also no class labels are required.

Section 3.2 describes this construction. We apply further corruption from the same family used during training, not the family on which we evaluate and report results. Whenever possible, we preserve the existing corruption pattern. For example, we increase the strength of a noise pattern rather than replace it with unrelated noise. The image with less corruption provides the teacher trajectory and anchor, and the image with more corruption is the student input. Appendix A.2 specifies worsening settings of severity 5 for CIFAR and severity 4 for ImageNet.

Important to also note that these pairs are used only during offline training. During deployment, the model applies the learned corrections, and online adaptation adjusts their strengths using prediction entropy. It requires neither a cleaner reference image nor access to the test corruption operator.

We agree that this construction could be explained more clearly and that alternative worsening choices deserve further evaluation. However, the operation itself is defined in the paper.

4. Accuracy, calibration, and applicability (2)

We’d like to note that our method provides a shared trajectory-alignment approach applicable to both supervised ViTs and‾\underline{\textbf{both supervised ViTs and}} CLIP-based vision-language models‾\underline{\textbf{CLIP-based vision-language models}}. The same correction mechanism and online gate-adaptation procedure support both settings. This applicability is part of the contribution, alongside the individual benchmark results. Recent ICLR work, such as BITTA (Sun et al., 2026), focuses specifically on vision-language models.

On supervised CIFAR-100-C, the source-free online variant achieves 74.9% accuracy versus 75.7% for PEA, a difference of 0.8 percentage points. It also reduces mean ECE from 18.00% to 9.53%, an improvement of 8.47 percentage points‾\underline{\textbf{an improvement of 8.47 percentage points}}.

For CLIP, the same approach improves accuracy from 42.6% to 47.4% over BITTA and reduces ECE from 14.70% to 5.77%. The source-dependent online variant‾\underline{\textbf{The source-dependent online variant}} exceeds both corresponding baselines‾\underline{\textbf{exceeds both corresponding baselines}} in accuracy (Tables 3–4).‾\underline{\textbf{in accuracy (Tables 3–4).}}

The review emphasizes the supervised‾\underline{\textbf{The review emphasizes the supervised}} source-free accuracy gap without discussing‾\underline{\textbf{source-free accuracy gap without discussing}} these corresponding benefits‾\underline{\textbf{these corresponding benefits}}. Applicability across both settings strengthens the contribution, and a limitation in one setting does not negate improvements established in the other. The assessment should consider applicability, accuracy, and calibration together fairly.

Regarding efficiency, teacher–student‾\underline{\textbf{Regarding efficiency, teacher–student}} trajectory matching occurs during‾\underline{\textbf{trajectory matching occurs during}} offline training‾\underline{\textbf{offline training}}. Deployment requires neither a teacher forward pass nor paired cleaner inputs. The frozen variant requires no test-time optimization. The online variant updates only 36 scalar gates for supervised ViT or 24 for CLIP, with optimization settings provided in Appendix A.2.

This establishes a small number of trainable test-time parameters, but does not by itself establish faster runtime. We agree online adaptation still requires differentiation, and the paper does not report wall-clock comparisons.

5. ImageNet-C results and the conclusion of ineffectiveness (3, 4e)

We acknowledge the accuracy limitation on ImageNet-C with the supervised ViT backbone. The source-free and source-dependent online variants achieve 60.2% and 60.7%, respectively, below several baselines (Table 6). These results are explicitly reported, and Section 4.2 directs readers to the full CIFAR-10-C and ImageNet-C results. We agree that this limitation could be more prominent in the main text.

However, it does not establish that the method is ineffective overall. For CLIP, the source-free online variant matches BITTA’s reported 31.9% mean accuracy which the review does not discuss.

Additionally, the ImageNet-C calibration results also deserve consideration. The source-dependent and source-free online variants achieve mean ECEs of:

  • Supervised ViT: 13.03% and 14.72%, compared with 33.44% for PEA.
  • CLIP: 12.89% and 8.01%, compared with 18.04% for BITTA.

Table 12 reports these calibration values. These results deserve consideration alongside the accuracy limitations. The contribution should be assessed‾\underline{\textbf{The contribution should be assessed}} on its methodological novelty,‾\underline{\textbf{on its methodological novelty,}} supporting observations, demonstrated benefits,‾\underline{\textbf{supporting observations, demonstrated benefits,}} and potential for future research.‾\underline{\textbf{and potential for future research.}} Outperforming every baseline on every benchmark‾\underline{\textbf{Outperforming every baseline on every benchmark}} is not a prerequisite for a meaningful‾\underline{\textbf{is not a prerequisite for a meaningful}} contribution, per ICLR guidelines.‾\underline{\textbf{contribution, per ICLR guidelines.}}

6. Relevance of the theoretical analysis (4d)

We strongly disagree that Section 3.4 is orthogonal to the method. It analyzes the actual training objective: the MAP formulation accounts for trajectory alignment, anchor preservation, and correction regularization. The transfer bound relates test residual error to offline residual error and the shift in the residual distribution, with an explicit Gaussian specialization.

The analysis explains the objective and the conditions under which alignment error can be controlled under distribution shift. It does not claim to prescribe the architecture or guarantee classification improvements, nor do we believe every theoretical analysis should. The absence of such guarantees does not make the analysis orthogonal to the method. The paper explicitly states that the bounds concern fixed adapters under additional assumptions and do not guarantee online adaptation gains.

Detailed proofs are already in Appendix B. We can make the connection clearer.

7. Notation and presentation (4a–4b)

We agree that the notation distinguishing the test-time loss from the selected linear-map sets can be clearer, and that the subsection structure should be simplified. These are addressable presentation issues. They should be considered separately from the validity of the trajectory-alignment method, its online adaptation procedure, and the benefits demonstrated in the experiments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.