acceptodds
Under review as a conference paper at ICLR 2027

Sample-wise Targeted Adversarial Attacks on Test-time Adaptation

Abstract

Test-time adaptation (TTA) mitigates distribution shifts by adapting models to unlabeled test inputs, but also exposes them to adversarial manipulation. Existing class-wise targeted attacks remain suboptimal for stealthy exploitation in this setting: since TTA operates on batches, forcing a subset of samples toward a target label unintentionally pulls similar benign samples along, resulting in a conspicuously high frequency of the target label that is easy to detect. To capture a more realistic threat, we introduce a sample-wise targeted attack. Unlike prior approaches, the attacker aims to misclassify only inputs carrying an attacker-chosen trigger, while preserving a benign-like prediction distribution to evade detection. To achieve this, we propose a meta-learning-based attack with a novel priority-aware gradient alignment strategy that explicitly prioritizes attack success. The strategy formulates the gradient update as an ellipsoidal trust-region problem, mitigating gradient conflict between the attack and stealth objectives, while providing theoretical guarantees for effective optimization of the attack objective in the presence of gradient misalignment. Extensive experiments on CIFAR-10-C, CIFAR-100-C, and ImageNet-C across TTA protocols demonstrate that our method achieves high targeted success rates while maintaining prediction behavior close to benign adaptation under multiple label-free stealth metrics, making it difficult to detect in unlabeled TTA deployment scenarios. Furthermore, we demonstrate that our attack shows strong robustness against existing defenses.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
test-time adaptation, trustworthy machine learning, adversarial attack

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_tXQAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The work proposes a sample-wise attack mechanism for in the TTA setting as opposed to the class-wise setting. The authors claim that class-wise attack on TTA can shift the label distribution which can work as a marker to understand that an attack is being performed. Motivated by this observation, they propose the sample-wise attack mechanism. They focus on an interesting observation that attack and stealth objectives are misaligned (their gradients point is opposite directions). As a result, they suggest an ellipsoidal trust-region problem formulation for the gradient update. They show that their meta-learning approach improves existing baselines as well.

Critical strengths
  1. The authors make the observation that the gradients of the attack and stealth objectives are misaligned. This hampers the attack and the authors propose an effective meta-learning based approach to handle this. The optimization approach works well and not just for the proposed method, but also for the baselines as well as shown by the authors. This is a valuable contribution.
  2. The authors provide theoretical guarantee for the loss minimization. This makes the approach rigorous bothe empirically and theoretically.
Critical weaknesses
  1. The motivation for sample-wise attack is not convincing. The label-distribution shift alone cannot justify sample-wise attack and the stealth can still be violated if one is not careful in choosing the percentage of victim samples.
  2. The triggers for the victim samples still require indirect access to the sampels of other users. This vioaltes the assumed attack setting.
Questions for authors
  1. Even if the attacker does not actively modify the victim's data stream, changing the samples by attaching the physical path or similar approach means accessing the data stream of the victim in the physical world. Can the authors clarify how it still is not violation of the assumed attack setting and attacker capability as the attacker is not supposed access the victim data stream at all? Also, the simulation instead of running experiments with actual physical-world data makes the claims bit weaker -- can the authors elaborate on that?
  2. The motivation of the label-distribution shift is not convincing enough. Can the authors illustrate a scenario where such label distribution shift can lead to identification of the attack or attacker? Even with distribution shift, it can be that the attacker cannot be identified so the service has to be down for everyone. But the real question is, how does one distinguish between the label-distribution shift due to shift in environment and due to attack? As per my understanding, if the percentage of victim samples is high, the proposed method can introduce high label-distribution shift. can the authors please address this concern and elaborate on the impact of the attack?
Reviewer_u5i1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Test-time adaptation (TTA) addresses distribution shifts by updating models using unlabeled test inputs. Shared-batch adaptation creates poisoning risks, motivating selective attacks that manipulate triggered inputs while preserving benign prediction behavior. This paper proposes a grey-box attack combining randomized surrogate tasks, prediction-consistency regularization, and attack-prioritized ellipsoidal gradient updates. Experiments on CIFAR-10-C, CIFAR-100-C, and ImageNet-C demonstrate high targeted success across five BN-based TTA methods, with reduced effectiveness under lower attacker participation and some defenses.

Critical strengths
  1. The trigger-defined threat model identifies a meaningful cross-user security risk without requiring access to victim inputs or current model parameters.
  2. The proposed gradient alignment achieves substantial ASR improvements over the evaluated isotropic and gradient-conflict baselines.
  3. Experiments cover multiple datasets, triggers, adaptation methods, participation ratios, and defenses.
Critical weaknesses
  1. The novelty framing overlooks DIA’s existing individual-sample and stealthy targeted attacks, requiring a clearer distinction based on trigger-conditioned generalization to unseen victims.
  2. Label histograms, EDD, and ADI do not establish strict selectivity or low detectability without benign accuracy, prediction-flip measurements, and calibrated detection results.
  3. Attack generation exploits the initial model’s shared BN statistics, but missing TeBN-only controls leave the contribution of TTA parameter updates unclear.
  4. The optimizer’s relationship to attack-centered CAGrad is insufficiently acknowledged, while comparisons lack tuned weighted-loss baselines at matched stealth levels.
  5. Headline results use 50% attacker participation, whereas fixed-20% experiments achieve approximately 20–30% ASR, limiting broad claims of practical effectiveness.
Questions for authors
  1. What are the benign accuracy drop, prediction-flip rate, and unintended target-label activation rate on non-triggered inputs?
  2. How detectable is the attack at a fixed false-positive rate under natural benign-stream variation?
  3. How does ASR change under TeBN-only inference, and which model states persist between evaluation batches?
  4. Does the ellipsoidal optimizer outperform tuned scalarization and the isotropic variant at matched stealth levels?
  5. Are attacker and victim pools disjoint at the underlying-image level across corruptions, and how are target labels and ASR denominators defined?
Reviewer_GNEPOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a sample-wise targeted attack strategy for attacking Test-Time Adaptation methods. The meta-learning-based attack strategy considers both attack success and stealth with a primary attack objective and a distributional stealth objective. The experimental results support the authors' claims and conclusions.

Critical strengths

This paper shifts from batch-wise to sample-wise attacks for attacking test-time adaptation methods to avoid attack failure caused by the aggregation of benign samples. The authors' proposed meta-learning framework and optimization strategy effectively address the difficulty of optimizing two antagonistic objective functions.

Critical weaknesses
  1. This paper only focuses on test-time adaptation methods based on BatchNorm layer updating, including TENT, RPL, EATA, SAR, and DeYO. For methods with stronger general capabilities, such as TTT-type methods applied to LLMs, the aggregation effect of benign samples is not obvious. Therefore, sample-wise attack is not necessarily more effective than batch-wise strategy, and it also introduces more unnecessary computational overhead and risk of attack failure.
  2. This paper does not clearly specify the test-time adaptation setting they target. Since attack methods for different TTA settings, like episodic TTA, online TTA, continual TTA, and one-sample TTA, are not necessarily generalizable. In particular, in the episodic and one-sample TTA settings, the method proposed in this paper has no obvious advantage.
Questions for authors

Apart from TTA methods based on BatchNorm updating, how effective is the proposed method compared to other TTA and TTT methods? In addition, the authors need to provide experimental results under different TTA settings, especially the episodic and one-sample settings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.