acceptodds
Under review as a conference paper at ICLR 2027

Moment Transport Poisoning for Online Test-Time Adaptation

Abstract

Online test-time adaptation (TTA) updates a pretrained model on unlabeled test data, allowing poisoned inputs to influence benign predictions through shared normalization and model updates. We propose Moment Transport Poisoning (\method), a grey-box attack that targets pre-BatchNorm activation statistics using a fixed source model. MTP maximizes standardized shifts in channel means and log-variances and combines this moment objective with non-ground-truth entropy through gradient-normalized projected updates. Our analysis connects moment displacement to feature distortion under mixed-batch normalization and to adaptation-state changes, and derives sufficient conditions for increased benign risk. Experiments on CIFAR-10-C, CIFAR-100-C, and four ImageNet-C subsets show that MTP outperforms DIA and RTTDP baselines in mean benign error. Five-seed evaluations on both CIFAR-C datasets confirm a mean advantage at both perturbation budgets. Ablations show that moment transport contributes more than the entropy objective alone, while their combination achieves the highest benign error. Elevated error during subsequent benign-only adaptation and increased held-out probe loss further demonstrate effects beyond the poisoned batch.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
test-time adaptation, adversarial samples, online adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_cnSbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Moment Transport Poisoning (MTP), a grey-box poisoning attack against online test-time adaptation. MTP maximizes shifts in pre-BatchNorm activation means and log-variances using a fixed source model and combines this objective with non-ground-truth entropy (NHE) through gradient-normalized optimization. The paper provides theoretical analyses relating moment displacement to feature distortion, adaptation updates, and benign prediction risk. Experiments show improved attack performance over DIA and RTTDP baselines.

Critical strengths
  • The paper proposes a normalization-aware poisoning objective that directly targets pre-BatchNorm activation statistics to disrupt online TTA.

  • The experiments demonstrate improvements over existing TTA poisoning baselines across multiple datasets, corruption types, and adaptation methods, supported by five-seed evaluations.

  • The ablation studies demonstrate the effectiveness of the moment-based objective, while the recovery and held-out probe experiments show poisoning effects beyond the immediate adaptation batch.

Critical weaknesses
  • My main concern is the lack of evaluation against MedBN (CVPR 2024), an existing defense specifically designed to mitigate TTA poisoning through robust BatchNorm statistics. Since MTP explicitly maximizes shifts in pre-BatchNorm activation means and log-variances, MedBN's median-based normalization may directly weaken the proposed attack mechanism. However, the paper evaluates only conventional TTA methods without considering this relevant defense. It is therefore unclear whether MTP remains effective against existing countermeasures targeting the same vulnerability.

  • The research positioning and methodological novelty relative to existing TTA poisoning attacks remain unclear. The paper adopts the grey-box threat model of RTTDP, which already considers poisoning using a fixed source model without knowledge of the current adapted model or benign co-batch. Therefore, the main contribution appears to be the moment-based surrogate objective rather than a new attack setting. However, the paper does not sufficiently establish why maximizing BatchNorm moment displacement is particularly effective compared to generic feature-discrepancy objectives or existing feature-level attacks.

  • The practical feasibility and effectiveness of the attack under realistic constraints remain unclear. The main experiments assume that attackers control 50% of each adaptation batch, with a default perturbation budget of ϵ=0.3\epsilon=0.3, allowing substantial input modifications. At ϵ=8/255\epsilon=8/255, the five-seed improvements over the strongest baseline decrease to only 1.21 and 0.90 percentage points on CIFAR-10-C and CIFAR-100-C, respectively (Table 2). Furthermore, the poison-ratio analysis is limited to ϵ=0.3\epsilon=0.3, leaving it unclear whether MTP maintains meaningful advantages when both the perturbation budget and attacker-controlled fraction are reduced.

  • The theoretical analysis does not sufficiently establish why the proposed objective leads to stronger poisoning in practice. Theorem 1 provides conditional guarantees for source-to-adapted moment transfer, while Theorem 4 requires favorable alignment between the poisoned adaptation update and the benign-risk gradient. However, the paper does not directly examine whether these conditions hold during online adaptation. Empirical analysis for these conditions would help substantiate the proposed mechanism.

Questions for authors

Please see the weaknesses above.

Reviewer_2JidOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Moment Transport Poisoning (MTP), a grey-box attack against online test-time adaptation (TTA). The method maximizes standardized shifts in pre-BatchNorm activation means and log-variances using a fixed source model and combines this moment-based objective with non-ground-truth entropy (NHE) through gradient-normalized projected updates. The authors provide theoretical analyses connecting moment displacement to mixed-batch normalization, adaptation updates, and benign prediction risk. Experiments on CIFAR-10-C, CIFAR-100-C, and selected ImageNet-C corruptions show higher benign classification error than DIA and RTTDP baselines, with additional recovery and held-out probe experiments suggesting effects beyond the poisoned batch.

Critical strengths
  1. The paper addresses an important security concern in online test-time adaptation, where malicious inputs can affect predictions on benign samples through shared normalization and model updates.

  2. The proposed moment-based objective is intuitive and directly targets the normalization statistics used by BN-based TTA methods.

  3. The paper provides mathematical analyses of moment displacement, mixed-batch normalization, and adaptation updates to motivate the attack mechanism.

  4. The experiments include multiple corruption benchmarks, perturbation budgets, and adaptation methods, with ablations and recovery analyses demonstrating the effectiveness of the moment objective.

Critical weaknesses

1. Limited methodological novelty and insufficient differentiation from prior work.

The central idea of exploiting BatchNorm statistics to disrupt benign predictions is closely related to existing studies on the vulnerability of normalization-based TTA, particularly MedBN [R1]. Feature-level poisoning has also been explored in FCA [R2], while the grey-box threat model and NHE objective are directly inherited from RTTDP [R3]. The main methodological contribution is therefore the specific construction of a standardized mean and log-variance displacement objective, rather than a fundamentally new poisoning framework. Although this objective is potentially useful, the paper does not sufficiently demonstrate why its particular formulation is superior to simpler alternatives, such as feature-distance maximization, mean-only or variance-only displacement, or unweighted moment matching. The current ablations primarily compare moment transport with NHE, leaving the necessity and distinct contribution of the proposed moment metric insufficiently established.

2. The theoretical guarantees rely on strong assumptions whose practical relevance is unclear.

Theorem 1 derives a source-to-adapted moment transfer guarantee under a Jacobian stability assumption and an additional lower-sensitivity condition requiring the source moment displacement to be bounded below by a positive constant times the input perturbation norm. However, the latter assumption can be difficult to satisfy in the high-dimensional perturbation space considered here. In particular, when the dimensionality of the selected moment statistics is smaller than that of the input perturbations, the moment Jacobian necessarily has a nontrivial nullspace, precluding such a uniform lower bound over a full neighborhood of perturbation directions. The theorem remains mathematically conditional, but the paper does not establish whether its assumptions hold for the actual attack space or adapted models. Several other theoretical results also follow from standard mixture identities, Taylor expansions, and smoothness arguments. Consequently, the theoretical analysis provides useful intuition but does not establish a sufficiently strong or practically verified guarantee of attack effectiveness.

3. The relationship between moment displacement and benign risk degradation is insufficiently validated.

A central motivation is that maximizing pre-BN moment displacement leads to distorted benign features and harmful adaptation updates. However, the proposed attack optimizes moment displacement rather than benign risk, and the theoretical analysis does not establish that increasing the former necessarily increases the latter. Theorem 3 decomposes update displacement into a moment-representable component and a residual, but the relative contribution of these components is not empirically quantified. More importantly, Theorem 4 requires harmful alignment between the adaptation displacement and the benign-risk gradient, whereas the experiments in Appendix Tables 14–15 measure alignment between the moment and NHE input gradients instead. These are different quantities. The reported results also show that larger moment displacement does not consistently correspond to greater held-out loss. Thus, the experiments do not directly validate the key theoretical conditions connecting the proposed objective to attack success. More explicit measurements of source-to-target moment transfer, update decomposition, and risk-gradient alignment would substantially strengthen the claimed mechanism.

4. Incomplete comparisons with relevant attack and defense approaches.

The experimental evaluation mainly compares MTP against DIA and two RTTDP variants (NHE and NHE+DR). However, the paper does not include direct comparisons with feature-based poisoning methods such as FCA [R2], nor does it sufficiently investigate simpler feature-statistics objectives that would help establish the contribution of the proposed formulation. Although differences in threat models may prevent a direct comparison with the original FCA protocol, matched or adapted baselines would be informative. In addition, the experiments primarily target BN-dependent adaptation methods, while robustness against stronger normalization defenses such as MedBN [R1] is not examined. This limits the evidence supporting the novelty, comparative effectiveness, and general applicability of MTP beyond the specific normalization mechanisms it targets.

5. Limited and statistically uncertain improvements under constrained attack budgets.

The strongest gains are obtained under the default setting with a relatively large perturbation budget (epsilon = 0.3) and a poison ratio of 0.5. Under the smaller budget of epsilon = 8/255, the five-seed improvements over the strongest baseline decrease to only 1.21 and 0.90 percentage points on CIFAR-10-C and CIFAR-100-C, respectively. On CIFAR-10-C, MTP does not outperform DIA in every seed, and the paired differences provide limited evidence of a robust advantage with only five runs. The CIFAR-100-C results also exhibit substantial seed variability relative to the reported improvement. Furthermore, the paper does not evaluate the combination of a small perturbation budget and a low poison ratio. These limitations make it difficult to assess whether the proposed attack provides a practically meaningful and consistently reproducible advantage under more restrictive threat conditions.

6. Numerical consistency and experimental reliability require clarification.

There appears to be a potentially important inconsistency between the three-seed and five-seed CIFAR-100-C results at epsilon = 0.3. Table 1 reports 49.85% benign error for MTP over three seeds, while Table 2 reports 47.12 ± 2.72% over five seeds. If the original three seeds are included in the five-seed evaluation under the same aggregation protocol, the remaining two seeds would need an average of approximately 43.03%, implying a minimum possible five-seed sample standard deviation of approximately 3.74%, which exceeds the reported 2.72%. These results therefore cannot be reconciled under that interpretation. The discrepancy may reflect different seeds, aggregation procedures, or reporting errors, but the paper does not clearly explain it. Since the main contribution relies heavily on quantitative improvements, the authors should clarify the evaluation protocol and provide the corresponding per-seed results.

 

[References]

[R1] "MedBN: Robust Test-Time Adaptation against Malicious Test Samples," CVPR, 2024.
[R2] "On the Adversarial Vulnerability of Label-Free Test-Time Adaptation," ICLR, 2025.
[R3] "On the Adversarial Risk of Test Time Adaptation: An Investigation into Realistic Test-Time Data Poisoning," ICLR, 2025.

 

I also have some minor concerns as follows:

  1. The moment-only objective has zero gradient at the clean input, making initialization important. The paper should explicitly describe the initialization protocol and its sensitivity.

  2. The effect of the selected BN layers is not sufficiently analyzed. A layer-selection ablation would help determine whether the method depends on particular normalization layers.

  3. The evaluation on ImageNet-C covers only four corruption subsets, limiting conclusions about large-scale generalization.

  4. The default comparison uses 10 attack steps for MTP and 40 for the baselines. Although additional step-matched results are provided, comprehensive comparisons under matched computational budgets would be more informative.

  5. The reported CPU efficiency results are limited to selected corruption settings and do not fully characterize the computational overhead of computing moment statistics and their gradients.

  6. The recovery experiments demonstrate persistent degradation over a limited horizon, but do not establish how long the effect remains under continued benign-only adaptation.

  7. The confidence intervals based on grouped corruption and seed results may not fully account for dependence among evaluations sharing the same corruption or adaptation trajectory.

Questions for authors

Please refer to the Weaknesses section for detailed explanations of the concerns below.

  1. Under what practically relevant perturbation sets does the lower-sensitivity assumption in Theorem 1 hold, and is there empirical evidence supporting the required source-to-adapted moment transfer conditions?

  2. How strongly do the moment-based update component and the actual benign-risk gradient alignment explain the observed attack success, beyond the magnitude of moment displacement alone?

  3. How does the proposed moment objective compare with simpler feature-distance, mean-only, variance-only, and alternative statistics-based attack objectives under matched conditions?

  4. Can the authors reconcile the three-seed and five-seed CIFAR-100-C results at epsilon = 0.3 and provide the corresponding per-seed scores and aggregation procedures?

  5. Does MTP retain a statistically reliable advantage under smaller perturbation budgets and lower poison ratios, and against robust normalization-based TTA methods?

Reviewer_uju4OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Authors propose a grey-attack method to influence the benign predictions from online test-time adaptive models. The proposed method, Moment Transport Poisoning (MTP), maximizes shifts in means and log-variances of activation statistics in BatchNorm layers. Experiments conduced in CIFAR and four ImageNet-C subsets shows the effectiveness of moment trasport.

Critical strengths

The paper analysis the moment transport poisoning in activation statistics of batchnorm layers, which is a novel perspective to desing attack method for online test-time adaptation tasks.

Critical weaknesses
  1. The writing of this paper requires improvement. The paper contains many meaningless abbreviations, and its incomplete introduction to related methods makes it difficult to read.
  2. There are many similar methods proposed to attack the actiavtion statistics of BatchNorm layers. This paper selects only DIA and RTTDP as baselines, which is insufficient to demonstrate the effectiveness of the proposed MTP.
  3. The experiments are conducted only on CIFAR and four ImageNet-C subsets. The results cannot support authors' claims.
Questions for authors

What are the experiment results on all ImageNet-C subsets?

How does MTP perform in continual TTA setting across subsets?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.