acceptodds
Under review as a conference paper at ICLR 2027

Poisoning Attack Framework Against Test-Time Adaptation

Abstract

Test-time adaptation (TTA) enables batch-wise updates by incorporating distributional characteristics from unlabeled test batches to enhance model generalization. The co-batching of poisoned samples can mislead a target model after adaptation, forcing attacker-intended predictions on benign co-batched test samples. Nonetheless, most attacks study this poisoning risk under extra requirements, such as access to real benign co-batch samples, the target model or queries. Such requirements tailor attacks to specific targets and hinder assessments of TTA poisoning risks when target-side access is unavailable. In this paper, we propose PCAC, a targeted poisoning framework without relying on the above requirements. PCAC departs from prior attacks by exploiting distribution-shifted auxiliary data for surrogate-side crafting. PCAC augments auxiliary data by emphasizing salient regions, creating a diverse proxy set that provides influential co-batch contexts. PCAC then induces a target-confident effect and smooths surrogate-side crafting to discourage center-specific and state-sensitive solutions. With proxy diversity and flatness-enhanced target-effect crafting, PCAC generates transferable poisoned test samples. Experiments across diverse TTA settings show that PCAC achieves 2.47× ASR of prior attacks without target-side access during crafting.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Privacy and Security, Targeted Poisoning Attack, Test-Time Adaptation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_odqkOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates targeted poisoning attacks against TTA, where poisoned samples in a test batch can manipulate model predictions on benign co-batched samples. The authors propose PCAC, a poisoning framework that eliminates the need for target-side access during attack crafting by leveraging distribution-shifted auxiliary data. Specifically, PCAC enhances proxy diversity through salient-region augmentation and improves transferability through target-confident effect induction and smooth surrogate-side optimization. Experiments across diverse TTA settings demonstrate that PCAC achieves much higher ASR of prior methods, highlighting its effectiveness in black-box poisoning scenarios.

Critical strengths
  1. The paper investigates targeted poisoning attacks against TTA, where poisoned sample. Compared with existing works, it further considers a more challenging black-box setting without access to the target model or deployed TTA algorithms during attack construction.
  2. PCAC combines salient-region augmentation, target-confident effect induction, and smooth optimization to enhance proxy diversity and attack transferability, offering a novel framework to black-box TTA poisoning.
  3. Experiments across diverse TTA settings demonstrate that PCAC achieves 2.47× the attack success rate of prior methods, providing empirical evidence of its effectiveness under limited-access conditions.
Critical weaknesses
  1. The authors introduce an additional random perturbation variable δ\delta when formulating targeted poisoning attacks, but its purpose and significance remain unclear. Moreover, δ\delta is neither incorporated into the proxy construction module nor explicitly discussed in the adversarial crafting module. Consequently, it is unclear how this perturbation contributes to the proposed method and affects the final results.
  2. The paper lacks sufficient justification for the design choices and the underlying methodological rationale. For example, in the proxy construction module, why are both Grad-CAM heatmaps and entropy confidence maps needed? What motivates the use of masks to suppress semantically less salient image regions? Similarly, why is enhancing flatness beneficial in the adversarial crafting module? The authors mainly describe the proposed components and overall framework without adequately explaining the motivations behind these choices, which weakens the overall persuasiveness of the paper.
  3. Existing TTA methods encompass diverse paradigms, including the use of additional adapter modules, fine-tuning with pseudo-labels, and prediction logit correction based on distributional information. It remains unclear how the proposed attack framework can generalize to these diverse TTA approaches. Furthermore, the experimental evaluation should include representative baselines from different TTA paradigms to provide a more comprehensive assessment.
  4. The authors should extend the experimental evaluation to include additional OOD benchmarks, such as ImageNet-A and ImageNet-V2, as TTA methods are commonly applied in such scenarios. The authors should also evaluate the proposed framework using auxiliary datasets that exhibit substantial distribution shifts relative to both the test samples and the source dataset, thereby better assessing its effectiveness under more challenging distributional conditions.
  5. The paper uses overly complex mathematical formulations and notations, many of which lack equation labels and numbers. This makes cross-referencing inconvenient and reduces the overall readability of the manuscript. The authors should simplify them.
Questions for authors

See the weaknesses above.

8 comments
AuthorsOpenReview ↗

To Reviewer odqk (Part I)

Dear Reviewer odqk,

Thank you for your detailed review! We provide more detailed clarifications below. Please let us know if anything is still unclear and we'd be happy to follow up.

W1. Role and Effect of δ\delta

Consistent with prior poisoning formulations (Wu et al., ICML 2023; Cong et al., IEEE S&P 2024), the perturbations δ={δi}\delta=\{\delta_i\} are introduced and applied to the attacker-controlled inputs BaB_a, i.e., Baδ={xi′+δi∣xi′∈Ba}.B_a^\delta=\{x_i'+\delta_i\mid x_i'\in B_a\}. Here, δ\delta denotes the learnable adversarial perturbation optimized during crafting, rather than an independently sampled random variable. The purpose of these perturbations is to shape the attacker-controlled inputs BaδB_a^\delta to induce the attacker-desired target-class effect during optimization, such that, when co-batched with benign inputs BbB_b in the target model's batch-level adaptation process, BaδB_a^\delta can cause the adapted target model to misclassify benign co-batched inputs as the attacker-desired class.

The Proxy Construction (PC) module does not involve the optimization of δ\delta; instead, it constructs an influential proxy co-batch set SS for the Adversarial Crafting (AC) module since the real benign co-batch BbB_b is unavailable. Adversarial Crafting (AC) module then explicitly optimizes δ\delta under the mixed batch Baδ∪SB_a^\delta\cup S. Specifically, AC iteratively updates δ\delta by jointly optimizing the target-confidence term, which encourages target-class predictions on the perturbed attacker-controlled inputs BaδB_a^\delta, and the attacker-to-proxy guidance term, which uses the δ\delta-dependent batch-averaged predictions of BaδB_a^\delta to induce target-oriented proxy predictions despite SS’s emphasized saliencies thereby enhancing the intended adversarial effects when BaδB_a^\delta is co-batched with unavailable benign samples BbB_b during target-model adaptation. Both terms depend on δ\delta through BaδB_a^\delta (Section 3.2, pp. 4–5; Algorithm 1, p. 5)

Consequently, δ\delta contributes to PCAC by encouraging target-confident predictions on BaδB_a^\delta, thereby enabling the poisoned inputs to influence batch-level adaptation and bias the adapted target model’s predictions on benign co-batched inputs BbB_b toward the attacker-desired class.

W2. Methodological Rationale

(1) Rationale for Combining Grad-CAM and Entropy-Confidence Maps. Since the real benign co-batch BbB_b is unavailable during crafting, our Proxy Construction (PC) module constructs influential proxy co-batches SS by saliency-enhancing auxiliary samples, allowing our Adversarial Crafting (AC) module to optimize δ\delta on BaB_a against these proxies. This encourages BaδB_a^\delta to induce target-oriented proxy predictions despite their emphasized saliencies, aiming to strengthen its adversarial effects during target-side adaptation.

The reason for using Grad-CAM and entropy-confidence maps is that they can provide both class-discriminative evidence AclsA_{\mathrm{cls}} and entropy-based prediction confidence CentC_{\mathrm{ent}}, which serve as saliency-guided enhancement cues for auxiliary samples. Grad-CAM maps highlight spatial regions contributing to the ground-truth class logit, while entropy-confidence maps highlight regions contributing to higher prediction confidence (lower entropy). Their normalized element-wise multiplication emphasizes regions supported by both characteristics. This design is consistent with prior TTA studies highlighting sample-wise reliability and individual sample influence on adaptation (Niu et al., ICML 2022; Niu et al., ICLR 2023; Li et al., ICML 2026), where feature activations contribute to batch-conditioned statistics (Nado et al., ICML 2020 Workshop), while predictive distributions provide signals for entropy minimization (Wang et al., ICLR 2021; Zhang et al., NeurIPS 2022) and pseudo-label updates (Goyal et al., NeurIPS 2022).

Moreover, we conducted a preliminary analysis under normal TTA to characterize the adopted two characteristics: class-discriminative evidence AclsA_{\mathrm{cls}} and entropy-based prediction confidence CentC_{\mathrm{ent}}. Specifically, using DenseNet-121 on CIFAR10-C, CIFAR100-C, and ImageNet-C at corruption severity 3 across eight TTA methods, we compared initially correctly classified samples that retained correct predictions after adaptation with those that became misclassified. Samples retaining correct predictions tended to exhibit higher scores on both AclsA_{\mathrm{cls}} and CentC_{\mathrm{ent}} (average pairwise ranking probabilities: 0.67 and 0.74, respectively, compared with 0.56 for maximum softmax probability and 0.51 for top-2 probability margin). These findings further support the need of both Grad-CAM and entropy-confidence as complementary cues for constructing influential proxy co-batch contexts.

AuthorsOpenReview ↗

To Reviewer odqk (Part II)

W2. Methodological Rationale (continued)

(2) Rationale for Masking. The masks translate the joint saliency responses into spatially selective enhancement, retaining regions supported by class-discriminative evidence AclsA_{\mathrm{cls}} and prediction confidence CentC_{\mathrm{ent}}, while attenuating weaker responses. Since different regions contribute unequally to these characteristics, retaining all regions may dilute the two predictive cues. We therefore use joint-saliency masks to preserve regions supported by both characteristics while suppressing weaker responses. Moreover, multiple masking levels provide diverse proxy contexts with varying degrees of saliency emphasis, further enabling to optimize perturbations δ\delta across diverse proxy contexts.

(3) Reason for enhancing flatness beneficial in the adversarial crafting module. Since the optimization of BaδB_a^\delta is conducted using constructed proxy co-batches SS in place of the unavailable real benign co-batches BbB_b, and a surrogate model fωf_\omega in place of the unavailable target model fθf_\theta, the reason enhancing flatness is beneficial is that it reduces the sensitivity of optimization to the proxy-benign context and surrogate-target state discrepancies. Without flatness enhancement, these discrepancies cause optimization at a single perturbation point and under a single surrogate state to produce overly specific updates that are sensitive to these variations, thereby limiting their effectiveness when BaδB_a^\delta is co-batched with the actual benign inputs BbB_b during target-model adaptation under fθf_\theta.

As analyzed in our theoretical decomposition (Section 3.2, pp. 4–5; Appendix E, pp. 21–22), the crafting gap arising from proxy-context and surrogate-model discrepancies is upper-bounded by the following terms:

RJ,ξδ,s(0)(δ)+PMω(δ;ϵ)+Rres(δ;ϵ).R_{J,\xi_\delta,s}^{(0)}(\delta) + P_M^\omega(\delta;\epsilon) + R_{\mathrm{res}}(\delta;\epsilon). Here, RJ,ξδ,s(0)(δ)R_{J,\xi_\delta,s}^{(0)}(\delta) characterizes local perturbation sensitivity, PMω(δ;ϵ)P_M^\omega(\delta;\epsilon) captures surrogate-state sensitivity, and Rres(δ;ϵ)R_{\mathrm{res}}(\delta;\epsilon) accounts for the remaining inaccessible target-side discrepancy. PCAC’s flatness-enhanced crafting update (FCU) addresses the two accessible sensitivities through perturbation-neighborhood averaging and surrogate-parameter shifting, respectively, aiming to reduce dependence on specific proxy–surrogate optimization conditions. Consequently, perturbations optimized under the proxy co-batch SS and surrogate model fωf_\omega are less dependent on specific optimization conditions, improving BaδB_a^\delta’s effectiveness when transferred to the actual benign co-batch BbB_b under the target model fθf_\theta.

W3. Mechanistic Explanation and Evaluation Across Diverse TTA Paradigms

Consistent with prior TTA poisoning studies (e.g., DIA (Wu et al., ICML 2023), TePA (Cong et al., IEEE S&P 2024), and FCA (Rifat et al., ICLR 2025)), PCAC exploits the batch-dependent nature of TTA, in which adaptation signals or prediction refinements are derived from co-batched samples, allowing attacker-controlled inputs to influence the underlying adaptation and prediction process. By adding perturbations to attacker-controlled inputs BaB_a, an attacker can optimize them to shape the data-characteristic of poisoned inputs BaδB_a^\delta to induce its desired target, thereby affecting the resulting adaptation states or batch-level predictions and causing target-class misclassifications on co-batched benign inputs BbB_b.

AuthorsOpenReview ↗

To Reviewer odqk (Part III)

W3. Mechanistic Explanation and Evaluation Across Diverse TTA Paradigms (continued)

Our original evaluation already covers diverse TTA paradigms with different mechanisms, including normalization-based adaptation (TeBN), entropy-based parameter updates (TENT, EATA, SAR), pseudo-label-based updates (Hard PL, Soft PL, RPL), and feature alignment (TTAC) (Table 1, p. 7). We additionally include two representative methods covering further TTA paradigms: TAST (Jang et al., ICLR 2023), which updates trainable adapter modules using nearest-neighbor-based pseudo-labels, and LAME (Boudiaf et al., CVPR 2022), which performs parameter-free prediction correction through cross-sample affinities. Both methods are evaluated under the same default CIFAR100-C poisoning setting, with ASR (%) results reported below.

TTA Method / AttackNo-PoisonWhite-BoxTargetCEDIATePAFCABLENHEOur PCAC
TAST0.94$_{\pm 0.07}$82.91$_{\pm 1.21}$33.74$_{\pm 1.36}$45.93$_{\pm 1.10}$50.12$_{\pm 1.33}$51.06$_{\pm 1.35}$55.34$_{\pm 1.24}$57.15$_{\pm 1.45}$64.36$_{\pm 1.19}$
LAME1.06$_{\pm 0.09}$69.64$_{\pm 1.48}$21.61$_{\pm 0.83}$29.88$_{\pm 1.26}$35.26$_{\pm 1.31}$33.10$_{\pm 1.42}$41.02$_{\pm 1.17}$38.54$_{\pm 1.29}$50.75$_{\pm 1.40}$

The results show that PCAC remains effective across these additional TTA, achieving ASRs of 64.36% and 50.75% on TAST and LAME, respectively.

W4. Additional OOD Benchmarks and Auxiliary Distribution Shifts.

Evaluation on OOD Benchmarks. We extend experiments to ImageNet-V2 (Recht et al., ICML 2019) and ImageNet-A (Hendrycks et al., CVPR 2021). ImageNet-V2 contains independently collected natural images, while ImageNet-A comprises naturally occurring challenging examples. We evaluate eight original TTA methods, together with TAST and LAME, on both benchmarks and retain VGG-19-BN surrogate and DenseNet-121 target on default poisoning setting. For both benchmarks, we also employ 10 labeled auxiliary samples from ImageNet-Sketch, whose sketch-style visual distribution differs from both the natural-image ImageNet-1K source distribution and the natural-image target distributions of ImageNet-V2 and ImageNet-A.

ImageNet-V2:

AttackTeBNTENTHardPLSoftPLRPLEATATTACSARTASTLAME
No-Poison0.09$_{\pm 0.02}$0.08$_{\pm 0.02}$0.10$_{\pm 0.03}$0.11$_{\pm 0.02}$0.07$_{\pm 0.01}$0.08$_{\pm 0.02}$0.12$_{\pm 0.03}$0.09$_{\pm 0.02}$0.12$_{\pm 0.03}$0.13$_{\pm 0.03}$
White-Box91.46$_{\pm 0.75}$90.82$_{\pm 0.93}$89.37$_{\pm 0.89}$85.62$_{\pm 0.98}$90.15$_{\pm 0.82}$89.42$_{\pm 1.14}$82.39$_{\pm 1.26}$85.08$_{\pm 1.08}$79.67$_{\pm 1.35}$67.48$_{\pm 1.48}$
TargetCE36.85$_{\pm 1.28}$40.21$_{\pm 1.15}$34.62$_{\pm 1.23}$30.76$_{\pm 1.40}$29.38$_{\pm 1.53}$41.49$_{\pm 1.05}$33.17$_{\pm 1.71}$39.42$_{\pm 1.62}$29.48$_{\pm 1.34}$22.17$_{\pm 0.95}$
DIA55.31$_{\pm 1.43}$58.14$_{\pm 1.20}$50.83$_{\pm 1.36}$43.95$_{\pm 1.62}$48.56$_{\pm 1.21}$59.42$_{\pm 1.37}$44.18$_{\pm 1.76}$52.39$_{\pm 1.59}$42.57$_{\pm 1.42}$30.48$_{\pm 1.28}$
TePA59.85$_{\pm 1.44}$60.92$_{\pm 1.17}$57.73$_{\pm 1.52}$55.14$_{\pm 1.41}$49.86$_{\pm 1.65}$58.46$_{\pm 1.28}$46.09$_{\pm 1.75}$54.15$_{\pm 1.53}$47.83$_{\pm 1.63}$34.69$_{\pm 1.27}$
FCA61.04$_{\pm 1.33}$55.73$_{\pm 1.61}$58.62$_{\pm 1.21}$49.77$_{\pm 1.38}$46.42$_{\pm 1.59}$61.30$_{\pm 1.46}$50.74$_{\pm 1.68}$56.32$_{\pm 1.39}$48.41$_{\pm 1.45}$36.82$_{\pm 1.22}$
BLE69.94$_{\pm 1.23}$71.65$_{\pm 1.41}$67.18$_{\pm 1.26}$63.29$_{\pm 1.77}$64.52$_{\pm 1.35}$66.78$_{\pm 1.43}$53.64$_{\pm 1.82}$59.47$_{\pm 1.59}$55.83$_{\pm 1.14}$40.54$_{\pm 1.40}$
NHE71.43$_{\pm 1.37}$69.84$_{\pm 1.19}$68.61$_{\pm 1.62}$60.94$_{\pm 1.21}$67.35$_{\pm 1.55}$66.14$_{\pm 1.71}$57.28$_{\pm 1.52}$60.97$_{\pm 1.45}$54.67$_{\pm 1.69}$42.19$_{\pm 1.48}$
Our PCAC77.86$_{\pm 1.10}$76.92$_{\pm 1.29}$75.61$_{\pm 1.36}$72.90$_{\pm 1.18}$73.35$_{\pm 1.34}$75.84$_{\pm 1.20}$67.34$_{\pm 1.50}$70.15$_{\pm 1.38}$63.28$_{\pm 1.14}$50.62$_{\pm 1.35}$
Reviewer_aR6eOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PCAC is a targeted poisoning attack on batch-wise test-time adaptation (TTA). The attacker has a public surrogate model and ten labelled auxiliary images from a shifted distribution, and no access to the target model, its TTA method, queries, or the benign co-batch. PCAC builds a saliency-based proxy co-batch from the auxiliary images, optimises a target-label loss with a stop-gradient soft-guidance term, and updates the perturbation with a dual-neighbourhood flatness objective (Alg. 1). It is evaluated on CIFAR10-C, CIFAR100-C and ImageNet-C against eight TTA methods and five adapted prior attacks over five seeds (Table 1), with component ablations and five countermeasures.

Critical strengths
  1. Broad evaluation with appropriate controls. The attack is tested on three benchmarks and eight TTA methods with five-seed standard deviations (Table 1), and is supported by a compute-matched control, a target-by-surrogate transfer grid, a size-matched raw-proxy control and a comparison with generic transfer optimizers on the same objective.
  2. A useful threat-model relaxation with a TTA-specific mechanism. Removing target-model, query and benign-batch access clarifies the realistic risk of TTA poisoning. The attacker-to-proxy soft-guidance term is specific to co-batch adaptation, and removing or replacing it costs a large share of the attack success rate (Table 7).
Critical weaknesses
  1. The margin over prior attacks is supplied by a flatness optimizer the baselines never received. In the default setting, PCAC without the flatness update scores below most of the adapted prior attacks (Tables 1, 3), and a generic PGN optimizer on the same objective already closes most of the gap to full PCAC (Table 10). The baselines run under the same iteration budget with their own update rules, but none receives a flatness or neighbourhood update, their target-loss weight is not reported, and PCAC's defaults were swept on the test benchmark. Adapted DIA also beats PCAC's own same-objective ablation, which the paper does not explain. The evidence therefore cannot separate the proxy and objective design from the benefit of an optimizer that only PCAC uses.
  2. The headline numbers are single best cells against the weakest baseline. The "2.47x" in the abstract and the "up to 35.48%" in the introduction are both single CIFAR10-C cells measured against DIA, the weakest adapted baseline (Table 1). Against the strongest adapted baseline the dataset-mean gain is far smaller, and it is smallest at the default TENT setting.
Questions for authors
  1. What ASR do NHE, BLE, TePA and DIA reach when crafted with your flatness-enhanced update, or with PGN, at matched crafting cost, on at least CIFAR100-C/TENT? This determines whether the contribution is the proxy and objective design or the optimizer.
  2. Why does adapted DIA (Table 1) outperform "Proxy CE only" (Table 3), when both are target-label CE on a size-matched co-batch with BN recomputation and no flatness update? Were the two run under the same protocol, and with what target-loss weight for each baseline?
  3. How many TTA update steps are performed per batch for the CNN targets and for the ViT-B/16 target? With TENT updating only LayerNorm affine parameters at a small learning rate with episodic reset, a single step changes the model very little, yet Table 6 reports that a majority of benign predictions flip to the target class. Please report White-Box ViT ASR as a function of the number of steps.
  4. How were PCAC's hyperparameters selected without target-side feedback, given that Fig. 4 sweeps them on the test corruptions?
  5. How much does ASR vary across random draws of the auxiliary corruption domain and of the auxiliary samples? The reported five-seed standard deviation excludes these draws ("randomly selected once"), and Table 11 varies the auxiliary source type but not the draws.
  6. Please restate the abstract and contribution claims as mean gain and mean ratio over the strongest adapted baseline per dataset, and state explicitly that the surrogate and target share the source training distribution and label space.
7 comments
AuthorsOpenReview ↗

To Reviewer aR6e (Part I)

Dear Reviewer aR6e,

We sincerely appreciate your time and effort in reviewing our paper and providing valuable comments. We provide explanations to your questions point-by-point in the following.

W1. Crafting Design and Evaluation Controls

Issue 1. Why Performance Differences Under Direct Optimization?

The key difference under direct optimization lies in the target-oriented effects encouraged by the respective attack objectives. Specifically, adapted DIA directly optimizes auxiliary co-batch predictions through fixed target-label supervision, while other adapted baselines retain their respective attack objectives with additional target-label supervision on auxiliary or attacker-controlled inputs. In contrast, PCAC’s targeting objective JsJ_s jointly encourages target-class confidence on poisoned inputs BaδB_a^\delta and uses their batch-averaged predictive distribution TˉA\bar T_A as soft guidance for target-oriented predictions on influential proxy set SS. This explicitly encourages BaδB_a^\delta to induce target-oriented proxy predictions despite saliency-enhanced contribution, aiming to enhance its adversarial effects when co-batched with unavailable benign samples BbB_b during target-model adaptation.

Under direct optimization, adapted DIA and other baselines can optimize their respective objectives with fixed target-label supervision, providing direct target-class optimization signals for the supervised predictions. In contrast, PCAC’s JsJ_s additionally seeks to induce target-oriented predictions on unperturbed proxy samples SS through BaδB_a^\delta, where the soft guidance TˉA\bar T_A changes with the poisoned-input predictions, while the proxy predictions are influenced indirectly through shared batch normalization statistics. Consequently, direct optimization on JsJ_s could favor the proxy-conditioned targeting effect under the current perturbation without sufficiently preserving it across nearby perturbations, helping explain why PCAC without flatness optimization underperforms several adapted baselines despite its more explicit co-batch targeting design.

Issue 2 and Q1. What benefits does PCAC provide beyond PGN?

PGN and PCAC’s flatness-enhanced crafting update (FCU) optimize the same targeting objective JsJ_s, but differ in how they account for local variations during perturbation optimization. PGN encourages input-space flatness by sampling nearby perturbations and combining their gradients through interpolation and momentum. In contrast, PCAC’s FCU averages JsJ_s over a perturbation neighborhood, derives a local surrogate parameter shift from this neighborhood-averaged objective, and updates the perturbations under the shifted surrogate state. This jointly considers perturbation-neighborhood variation and surrogate-state sensitivity, extending beyond the input-space flatness considered by PGN.

Our FCU’s joint optimization is particularly relevant to PCAC’s targeting objective JsJ_s, which encourages target-class confidence on poisoned inputs BaδB_a^\delta while using their predictions to guide target-oriented effect on saliency-enhanced proxy set SS. Optimizing this objective under nearby perturbations and shifted surrogate states helps discourage solutions whose proxy-conditioned targeting effect is specific to the current optimization conditions.

We apply PGN and PCAC’s flatness-enhanced crafting update (FCU) to adapted baselines (DIA, TePA, FCA, BLE, and NHE) on CIFAR100-C/TENT, retaining their respective attack objectives while comparing native, PGN, and FCU crafting updates at approximately matched costs against full PCAC (JsJ_s + FCU, 6.58 s/batch). The following tables report crafting cost (seconds/batch) and ASR (%). Native-CM, +PGN-CM, and +FCU-CM denote the corresponding crafting updates under approximately compute-matched (CM) costs, while Original reports each method’s original ASR.

AuthorsOpenReview ↗

To Reviewer aR6e (Part II)

Issue 2 and Q1 (continued)

MethodNative-CM+PGN-CM+FCU-CM
DIA6.536.426.49
TePA6.486.766.29
FCA6.616.276.62
BLE6.556.786.35
NHE6.507.136.72
JsJ_s (Ours)6.436.406.58
MethodOriginalNative-CM+PGN-CM+FCU-CM
DIA64.4767.3868.9470.35
TePA66.5267.2168.2869.06
FCA61.0164.3767.0567.52
BLE70.5872.4875.3676.34
NHE73.9474.8677.8178.09
JsJ_s (Ours)62.7466.1776.9483.16

Overall, the compute-matched comparisons indicate that flatness optimization also benefits the adapted baselines, while PCAC’s targeting objective JsJ_s achieves the highest ASR under the same FCU (83.16% vs. 78.09% for the strongest baseline). Moreover, the larger improvement of JsJ_s from PGN-CM to FCU-CM suggests that PCAC’s proxy-conditioned targeting effect particularly benefits from jointly accounting for perturbation-neighborhood variation and surrogate-state sensitivity.

Issue 3 and Q4. How Are Hyperparameters Selected?

We thank the reviewer for pointing out the omission of hyperparameter selection. The default PCAC hyperparameters were selected through surrogate-side validation using the proxy target rate (PTR) without target-side feedback, consistent with practice in prior work (Wei et al., CVPR 2023; Li et al., ICCV 2025), while Figure 4 reports target-side sensitivity to variations around the selected defaults, rather than serving as the basis for hyperparameter selection. Specifically, surrogate-side hyperparameter tuning employed five-fold cross-validation on the ten labeled auxiliary samples. In each fold, eight samples were used to construct the method-specific crafting context, while the remaining two were held out for validation. PTR measures the proportion of non-target validation samples predicted as the specified target class under batch-conditioned surrogate inference, with scores aggregated across folds and target-label trials.

The following table reports the surrogate-side PTR (%) for PCAC and adapted baselines under the candidate target-conditioning coefficients {0.1,0.3,1,3,10}\{0.1,0.3,1,3,10\} across the three datasets. PCAC uses a shared λab=1\lambda_{ab}=1, selected by maximizing the mean PTR across the three datasets, and each adapted baseline selects its λtar\lambda_{\mathrm{tar}} separately for each dataset. DIA is omitted because its adapted objective introduces no additional target-conditioning coefficient.

DatasetMethod0.10.31310Selected
CIFAR10-CPCAC72.80$_{\pm 0.87}$74.33$_{\pm 1.13}$75.59$_{\pm 0.96}$75.90$_{\pm 1.27}$71.41$_{\pm 1.43}$—
TePA58.47$_{\pm 1.06}$63.86$_{\pm 0.94}$62.19$_{\pm 1.27}$60.72$_{\pm 1.41}$56.94$_{\pm 1.15}$0.3
FCA62.38$_{\pm 1.52}$63.91$_{\pm 0.87}$64.57$_{\pm 1.34}$65.82$_{\pm 1.09}$60.96$_{\pm 1.46}$3
BLE67.14$_{\pm 1.22}$70.26$_{\pm 0.96}$69.73$_{\pm 1.58}$72.08$_{\pm 1.11}$68.49$_{\pm 1.39}$3
NHE68.76$_{\pm 0.85}$72.31$_{\pm 1.25}$74.26$_{\pm 1.49}$73.08$_{\pm 1.02}$70.64$_{\pm 1.63}$1
CIFAR100-CPCAC83.08$_{\pm 1.19}$86.56$_{\pm 0.86}$88.91$_{\pm 1.04}$87.41$_{\pm 1.38}$82.22$_{\pm 1.51}$—
TePA64.87$_{\pm 0.91}$68.53$_{\pm 1.32}$70.16$_{\pm 1.17}$72.04$_{\pm 1.55}$69.61$_{\pm 0.88}$3
FCA71.46$_{\pm 1.43}$75.28$_{\pm 1.04}$74.13$_{\pm 1.28}$70.67$_{\pm 1.61}$68.24$_{\pm 0.97}$0.3
BLE77.38$_{\pm 1.36}$79.64$_{\pm 1.13}$82.17$_{\pm 1.47}$80.83$_{\pm 0.86}$78.96$_{\pm 1.24}$1
NHE77.91$_{\pm 1.57}$83.46$_{\pm 1.01}$82.72$_{\pm 1.33}$79.58$_{\pm 0.92}$76.83$_{\pm 1.45}$0.3
ImageNet-CPCAC82.16$_{\pm 1.46}$85.55$_{\pm 1.09}$85.86$_{\pm 0.87}$84.39$_{\pm 1.35}$79.31$_{\pm 1.62}$—
TePA62.48$_{\pm 1.18}$67.39$_{\pm 1.54}$69.27$_{\pm 0.83}$67.82$_{\pm 1.29}$64.96$_{\pm 1.07}$1
FCA69.16$_{\pm 1.42}$67.28$_{\pm 0.95}$68.43$_{\pm 1.31}$64.71$_{\pm 1.16}$62.06$_{\pm 1.59}$0.1
BLE74.62$_{\pm 0.89}$77.31$_{\pm 1.37}$79.06$_{\pm 1.05}$77.84$_{\pm 1.48}$76.28$_{\pm 0.93}$1
NHE75.86$_{\pm 1.26}$79.42$_{\pm 1.44}$78.67$_{\pm 0.84}$77.93$_{\pm 1.35}$74.18$_{\pm 1.12}$0.3
PooledPCAC79.35$_{\pm 0.57}$82.15$_{\pm 0.74}$83.45$_{\pm 0.62}$82.57$_{\pm 0.83}$77.65$_{\pm 0.68}$1
AuthorsOpenReview ↗

To Reviewer aR6e (Part III)

W2 and Q6. Revisions to Claims

We thank the reviewer for this suggestion. We revised the Abstract and Introduction to report the mean ASR gain and ratio over the strongest adapted baseline selected separately for each dataset, and clarified that the surrogate and target share the same source-training distribution and label space.

Abstract: We replaced the original single-cell comparison with: Experiments across three distribution-shift benchmarks and eight TTA methods show that PCAC achieves a mean ASR gain of 10.75 percentage points and a mean ASR ratio of 1.18× over the strongest adapted baseline selected separately for each benchmark, without target-side access during crafting.

Introduction: We revised the performance claim to: Evaluations across three benchmarks and eight TTA methods show that PCAC improves mean ASR over the strongest adapted baseline by 12.27, 9.62, and 10.37 percentage points (1.25×, 1.14×, and 1.15×) on CIFAR10-C, CIFAR100-C, and ImageNet-C, respectively. Averaged across the three benchmarks, these correspond to a mean gain of 10.75 percentage points and a mean ASR ratio of 1.18×. Results against countermeasures further demonstrate its effectiveness under the evaluated defenses.

Threat Model (Attacker’s Capability and Knowledge): We added the following clarification: The attacker has access to a publicly available pretrained surrogate model that shares the source training distribution and label space with the target model, while differing in architecture and learned parameters.

Q2. Adapted DIA vs. “Proxy CE only”

Adapted DIA applies target-label cross-entropy supervision to raw auxiliary samples, whereas “Proxy CE only” applies it to saliency-enhanced proxies. The latter retain more concentrated class-discriminative and confidence-related evidence, making their predictions less amenable to direct target-label supervision and resulting in less effective perturbation optimization than adapted DIA under the same protocol.

Adapted DIA and “Proxy CE only” follow the same protocol, including the matched co-batch size, perturbation budget, optimization iterations, and BN recomputation, with unweighted target-label cross-entropy (i.e., no additional target-loss coefficient).

The selected target-loss weights λtar\lambda_{\mathrm{tar}} for the adapted baselines are reported in the table presented under Issue 3 and Q4.

Q3. TTA Update Steps

We thank the reviewer for requesting clarification of the adaptation-step settings. Following prior TTA implementations (Wang et al., ICLR 2021; Rusak et al., TMLR 2022; Niu et al., ICML 2022; Niu et al., ICLR 2023; Su et al., NeurIPS 2022), CNN targets use one update for TENT, RPL, Hard PL, Soft PL, and EATA, two updates for SAR and TTAC, while TeBN only recomputes batch statistics. For ViT-B/16, we adopt 10-step episodic TENT with LayerNorm-affine updates following prior ViT adaptation practices (Wang et al., IJCV 2025; Kim et al., CVPR 2026). The following table reports No-Poison, White-Box, and PCAC ASRs (%) for ViT-B/16 under TENT on ImageNet-C with 1, 2, 5, 10, and 20 adaptation steps.

TTA stepsNo-PoisonWhite-BoxPCAC
10.091.620.89
20.106.744.26
50.0935.2824.63
100.0969.1256.72
200.1175.0762.84

Q5. Auxiliary-Domain and Sample-Draw Variation

We evaluate five randomly selected auxiliary corruption domains, each with five independent draws of ten labeled auxiliary samples, on CIFAR100-C under TENT. The ASR (%) results are reported below.

Auxiliary DomainDraw 1Draw 2Draw 3Draw 4Draw 5Avg
D178.8680.2579.5581.0180.3380.00
D282.1581.8683.6182.4581.7082.35
D384.2983.7284.9584.1885.2584.48
D481.1182.3580.0383.4281.9681.77
D582.2383.0583.8781.7582.7982.74

Across auxiliary configurations, the between-domain standard deviation of mean ASR is 1.62 percentage points (pp), while the average within-domain sample-draw standard deviation is 0.86 pp, with ASR ranging from 78.86% to 85.25%. These results indicate that PCAC maintains effective attack performance across the evaluated auxiliary domains and sample draws.

Reviewer_XFRnOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PCAC, a targeted poisoning attack framework against test-time adaptation (TTA) that relaxes the strong attacker-knowledge assumptions of prior work by eliminating the need for target-model access, target-model queries, or benign co-batch knowledge. PCAC constructs proxy co-batches from distribution-shifted auxiliary data and employs target-confident soft guidance and flatness-aware optimization to improve attack transferability. Experimental results across multiple corruption benchmarks, model architectures, and TTA methods demonstrate substantial improvements over existing poisoning attacks adapted to the proposed threat model.

Critical strengths
  • The paper addresses an important security problem by investigating targeted TTA poisoning under a more restrictive black-box setting, without requiring target-model access, queries, or benign co-batch knowledge.

  • The experimental results demonstrate substantial improvements over existing TTA poisoning baselines across various datasets, target architectures, and adaptation methods.

  • The paper provides extensive ablation studies and additional analyses, including comparisons with existing adversarial transferability techniques, different auxiliary data settings, and defense mechanisms

Critical weaknesses
  • While the paper demonstrates improved attack performance under relaxed threat assumptions, my main concern is the limited methodological novelty. Several components of PCAC build on existing transfer-based black-box adversarial attack techniques, including saliency-guided transformations and flatness-aware optimization. Although attacker-to-proxy soft guidance appears to be the main TTA-specific contribution, its distinction from conventional soft-label guidance remains unclear.

  • The practical feasibility of the threat model remains unclear. The experiments assume that attackers can control 15% of each test batch, with poisoned samples drawn from the same distribution as benign samples. It is unclear how attackers could reliably inject such a substantial fraction of samples into a test batch in realistic TTA deployments. Although similar assumptions appear in prior work, their practical relevance deserves further justification.

  • The fairness of baseline comparisons remains unclear. Existing attacks are substantially modified to fit the proposed threat model, including the addition of target-conditioning objectives, but it is unclear whether these baselines are adequately tuned. Furthermore, PCAC incurs higher crafting costs, while compute-matched comparisons are mainly limited to the Direct baseline rather than the strongest competing attacks.

Questions for authors

Please see the weaknesses above.

8 comments
AuthorsOpenReview ↗

To Reviewer XFRn (Part I)

Dear Reviewer XFRn,

We sincerely appreciate the time and effort you've devoted to reviewing our work and providing helpful feedback!

W1. Distinctions from Related Approaches

(1) Proxy Construction vs. Saliency-Guided Transformations. Different from prior saliency-guided strategies that primarily use salient regions to preserve discriminative information or guide input transformations for adversarial input diversity, our Proxy Construction (PC) module jointly attributes and combines two complementary predictive characteristics: class-discriminative evidence and prediction confidence. The TTA-specific contribution of our PC module lies in constructing prediction-informative proxy co-batch contexts against which attacker-controlled inputs BaδB_a^\delta are optimized to induce target-oriented proxy predictions despite the emphasized class-discriminative and confidence-related regions in SS, aiming to enhance the adversarial effects of BaδB_a^\delta when co-batched with unavailable benign samples BbB_b during target-model adaptation.

Technically, PC differs from saliency-guided selection based on a single attribution criterion by jointly deriving and combining two complementary feature-map attribution maps: a label-indicative Grad-CAM map LfyL_f^y, based on the ground-truth class logit, and an entropy-confidence Grad-CAM map LentL_{\mathrm{ent}}, based on the normalized prediction-confidence score. While LfyL_f^y identifies spatial regions contributing to the ground-truth class logit, LentL_{\mathrm{ent}} captures regions associated with higher prediction confidence (lower entropy). Rather than determining spatial emphasis from either characteristic alone, PC combines the normalized maps through element-wise multiplication, assigning greater emphasis to regions jointly supported by both characteristics while downweighting those weakly supported by either.

(2) Attacker-to-Proxy Guidance vs. Conventional Soft-Label Guidance. Whereas conventional soft-label guidance transfers a teacher’s predictive distribution to align student predictions on the same inputs or corresponding views, our attacker-to-proxy (A2S) guidance derives a batch-averaged distribution from poisoned attacker-controlled inputs BaδB_a^\delta to guide predictions on the distinct proxy co-batch SS constructed by our PC module. Together with the attacker-side target-confidence objective, A2S encourages target-oriented predictions on SS despite its retained class-discriminative and confidence-related evidence, aiming to enhance the adversarial effects of BaδB_a^\delta when co-batched with unavailable benign inputs BbB_b during target-side TTA.

Specifically, conventional soft-label guidance typically minimizes: $\mathcal L_{\mathrm{soft}} =-\frac{1}{|B_a|}\sum_{x_i\in B_a} \sum_k \operatorname{sg}[p_k^T(x_i)] \log p_k^{\mathrm{stu}}(x_i). $By comparison, optimizing the attacker-side target-confidence loss LATC\mathcal L_A^{\mathrm{TC}} alone encourages BaδB_a^\delta to predict the desired class ℓtar\ell_{\mathrm{tar}} without explicitly guiding proxy predictions. A2S further accounts for the co-batch dependence of TTA by constructing a batch-averaged soft target TˉA\bar T_A from BaδB_a^\delta to guide predictions on SS: TˉA=1∣Ba∣∑xi∈Bapω′(xi+δi),LA→Star=−1∣S∣∑u∈Ssg⁡[TˉA]⊤log⁡pω′(u)\bar T_A=\frac{1}{|B_a|}\sum_{x_i\in B_a} p_{\omega'}(x_i+\delta_i), \mathcal L_{A\rightarrow S}^{\mathrm{tar}} =-\frac{1}{|S|}\sum_{u\in S} \operatorname{sg}[\bar T_A]^\top \log p_{\omega'}(u). Here, TˉA\bar T_A provides shared batch-level soft guidance for SS. Since predictions use batch-normalization statistics computed over Baδ∪SB_a^\delta\cup S, the proxy-side loss depends on δ\delta through the mixed-batch statistics. This incorporates proxy-side prediction effects into perturbation optimization, linking the target-oriented behavior of BaδB_a^\delta to its intended adversarial effects under batch-level TTA.

AuthorsOpenReview ↗

To Reviewer XFRn (Part II)

(3)Dual-Neighborhood Crafting vs.Transfer-Oriented Crafting Strategies. Unlike prior crafting strategies that primarily focus on improving transferability through individual mechanisms, such as input transformations, perturbation-gradient manipulation, or perturbation-neighborhood optimization, our PCAC’s flatness-enhanced crafting update (FCU) jointly accounts for perturbation-neighborhood variations and surrogate-state sensitivity within proxy-conditioned targeting objective JsJ_s. As summarized in the following table, the compared strategies primarily operate through input transformations, perturbation-gradient manipulation, or perturbation-neighborhood optimization. FCU instead evaluates JsJ_s over a local perturbation neighborhood to obtain Ms,ξδM_{s,\xi_\delta}, derives a surrogate parameter shift ϵω⋆\epsilon_\omega^\star from the gradient of this averaged objective, and updates δ\delta under the shifted state ω+ϵω⋆\omega+\epsilon_\omega^\star.

MethodOptimization mechanismDistinction from FCU
MIMomentum accumulationGradient accumulation without surrogate-state shifting
DIStochastic input transformationsInput diversity rather than perturbation–parameter coupling
TISpatial gradient smoothingSpatial smoothing without surrogate-state shifting
EOTTransformation-averaged gradientsTransformation averaging without parameter-neighborhood evaluation
MI + DICombined input-side strategiesCombines input-side strategies without parameter shifting
PGNPerturbation-neighborhood gradient optimizationPerturbation-side flatness without parameter shift derived from the neighborhood-averaged objective

The TTA-specific benefit of FCU arises from the mismatch between surrogate-side crafting and target-side adaptation: perturbations on BaB_a are optimized using proxy co-batches SS instead of unavailable benign co-batches BbB_b, and a surrogate model fωf_\omega instead of the inaccessible target model fθf_\theta. Optimizing JsJ_s at a single perturbation point and surrogate state may therefore yield updates overly specific to these proxy–surrogate conditions. By averaging JsJ_s over nearby perturbations and deriving the parameter shift from the same averaged objective, FCU accounts for both perturbation variations and surrogate-state sensitivity, aiming to improve transferability when BaδB_a^\delta is co-batched with actual benign inputs BbB_b during target-model adaptation under fθf_\theta.

W2. Threat Model and Deployment Context

Attacker Capability. Consistent with the established test-batch poisoning setting studied in Wu et al. (ICML 2023), Cong et al. (IEEE S&P 2024), Rifat et al. (ICLR 2025), and Su et al. (ICLR 2025), we evaluate batch-level TTA on test batches B=Ba∪BbB=B_a\cup B_b, where BaB_a and BbB_b denote attacker-controlled and benign samples whose underlying clean inputs are drawn from the same target-test distribution. The attacker perturbs only BaB_a, while both sets jointly participate in adapting the source-pretrained model to the target domain.

Poisoning Ratio. Following prior TTA poisoning evaluations that consider fixed attacker-controlled fractions (e.g., 5%–20% in Wu et al., ICML 2023; 5%–40% in Rifat et al., ICLR 2025), we adopt a default poisoning ratio of 15% per test batch as a controlled benchmark setting, without assuming that attackers can guarantee this fraction in deployment. We also evaluate lower ratios in Table 2: on CIFAR100-C under TENT, PCAC achieves 63.82%, 75.36%, and 83.16% ASR at 5%, 10%, and 15% contamination, respectively, demonstrating effectiveness at smaller attacker-controlled fractions.

Illustrative Deployment Scenario. As noted in prior TTA studies, adapting to a target domain with limited test samples can lead to unreliable batch statistics (Su et al., AAAI 2024), while collaborative TTA has been explored to support adaptation across participating devices (Chen et al., NeurIPS 2024). One possible scenario is a shared image-recognition service in which multiple devices operating under the same target-domain conditions contribute their test images to form batches for TTA. For example, cameras operating under similar low-light conditions may individually provide only a limited number of samples, motivating the service to aggregate submissions for batch-level adaptation. In this collaborative setting, an attacker may participate as one of the contributing clients and perturb its own submitted samples, which are then processed alongside benign submissions within the same target-domain TTA batch. This scenario illustrates how attacker-controlled and benign submissions may enter the same TTA batch.

AuthorsOpenReview ↗

To Reviewer XFRn (Part III)

W3. Baseline Tuning and Compute-Matched Comparisons

Baseline Tuning. We thank the reviewer for pointing out this omission. We tuned the target-conditioning coefficient λtar\lambda_{\mathrm{tar}} for each adapted baseline on each dataset over {0.1,0.3,1,3,10}\{0.1,0.3,1,3,10\}, using surrogate-side validation based on the proxy target rate (PTR). Specifically, five-fold cross-validation was performed on ten labeled auxiliary samples, with eight samples used for constructing the method-specific crafting context and two held out for validation in each fold. PTR measures the fraction of non-target validation samples predicted as the specified target class under batch-conditioned surrogate inference, aggregated across folds and target-label trials.

The following table reports PTR (%) across candidate coefficients. Each adapted baseline selects λtar\lambda_{\mathrm{tar}} separately per dataset. DIA is omitted because its adapted objective has no additional target-conditioning coefficient.

DatasetMethod0.10.31310Selected
CIFAR10-CTePA58.47$_{\pm 1.06}$63.86$_{\pm 0.94}$62.19$_{\pm 1.27}$60.72$_{\pm 1.41}$56.94$_{\pm 1.15}$0.3
CIFAR10-CFCA62.38$_{\pm 1.52}$63.91$_{\pm 0.87}$64.57$_{\pm 1.34}$65.82$_{\pm 1.09}$60.96$_{\pm 1.46}$3
CIFAR10-CBLE67.14$_{\pm 1.22}$70.26$_{\pm 0.96}$69.73$_{\pm 1.58}$72.08$_{\pm 1.11}$68.49$_{\pm 1.39}$3
CIFAR10-CNHE68.76$_{\pm 0.85}$72.31$_{\pm 1.25}$74.26$_{\pm 1.49}$73.08$_{\pm 1.02}$70.64$_{\pm 1.63}$1
CIFAR100-CTePA64.87$_{\pm 0.91}$68.53$_{\pm 1.32}$70.16$_{\pm 1.17}$72.04$_{\pm 1.55}$69.61$_{\pm 0.88}$3
CIFAR100-CFCA71.46$_{\pm 1.43}$75.28$_{\pm 1.04}$74.13$_{\pm 1.28}$70.67$_{\pm 1.61}$68.24$_{\pm 0.97}$0.3
CIFAR100-CBLE77.38$_{\pm 1.36}$79.64$_{\pm 1.13}$82.17$_{\pm 1.47}$80.83$_{\pm 0.86}$78.96$_{\pm 1.24}$1
CIFAR100-CNHE77.91$_{\pm 1.57}$83.46$_{\pm 1.01}$82.72$_{\pm 1.33}$79.58$_{\pm 0.92}$76.83$_{\pm 1.45}$0.3
ImageNet-CTePA62.48$_{\pm 1.18}$67.39$_{\pm 1.54}$69.27$_{\pm 0.83}$67.82$_{\pm 1.29}$64.96$_{\pm 1.07}$1
ImageNet-CFCA69.16$_{\pm 1.42}$67.28$_{\pm 0.95}$68.43$_{\pm 1.31}$64.71$_{\pm 1.16}$62.06$_{\pm 1.59}$0.1
ImageNet-CBLE74.62$_{\pm 0.89}$77.31$_{\pm 1.37}$79.06$_{\pm 1.05}$77.84$_{\pm 1.48}$76.28$_{\pm 0.93}$1
ImageNet-CNHE75.86$_{\pm 1.26}$79.42$_{\pm 1.44}$78.67$_{\pm 0.84}$77.93$_{\pm 1.35}$74.18$_{\pm 1.12}$0.3

Compute-Matched Comparison. We extend the evaluation to five adapted baselines (DIA, TePA, FCA, BLE, and NHE) on CIFAR100-C/TENT, retaining their respective attack objectives while comparing native, PGN, and FCU crafting updates at approximately matched costs against full PCAC (Js+FCUJ_s+FCU, 6.58 s/batch). The following tables report crafting cost (seconds/batch) and ASR (%). Native-CM, +PGN-CM, and +FCU-CM denote the corresponding crafting updates under approximately compute-matched (CM) costs, while Original reports each method’s original ASR.

MethodNative-CM+PGN-CM+FCU-CM
DIA6.536.426.49
TePA6.486.766.29
FCA6.616.276.62
BLE6.556.786.35
NHE6.507.136.72
Direct JsDirect \ J_s6.436.406.58
MethodOriginalNative-CM+PGN-CM+FCU-CM
DIA64.4767.3868.9470.35
TePA66.5267.2168.2869.06
FCA61.0164.3767.0567.52
BLE70.5872.4875.3676.34
NHE73.9474.8677.8178.09
Direct JsDirect \ J_s62.7466.1776.9483.16

The compute-matched results show that flatness optimization also benefits the adapted baselines, with our FCU consistently outperforming PGN across all five methods. Notably, our PCAC's proxy-conditioned targeting objective JsJ_s achieves the highest ASR under the same FCU update (83.16%), outperforming the strongest adapted baseline, NHE by 5.07 percentage points. Moreover, the improvement from PGN-CM to FCU-CM is larger for JsJ_s than for the adapted baselines. This suggests that the proposed targeting objective JsJ_s particularly benefits from the joint consideration of perturbation-neighborhood variation and surrogate-state sensitivity. These results support the complementary contributions of proxy-conditioned targeting and flatness-enhanced optimization.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to Reviewers

We sincerely thank all reviewers for their constructive feedback. We appreciate their recognition of the importance of the security problem (XFRn), the useful threat-model relaxation and broad evaluation (aR6e), and the value of our framework for black-box TTA poisoning (odqk).

1. Highlighting Compute-Matched Results and PCAC's Contributions

A central concern raised by the reviewers [aR6e, XFRn] was whether PCAC's performance advantage primarily arose from its flatness-enhanced crafting update (FCU). To examine this, we evaluated five adapted baselines (DIA, TePA, FCA, BLE, and NHE), retaining their respective attack objectives and comparing their native and FCU-enhanced crafting updates, alongside PCAC using its proxy-conditioned targeting objective (JsJ_s) with FCU, under approximately matched crafting costs. The resulting compute-matched ASR margins are summarized in Table 5 (Section 4.3, p. 9), while full per-baseline results and crafting costs are provided in Table 7 (Appendix B, p. 16).

BenchmarkTTAPCAC (JsJ_s + FCU)Margin vs. Native-CMMargin vs. FCU-CM
CIFAR10-CTeBN68.36+9.70+5.35
TENT70.46+1.60+3.95
TTAC52.74+12.95+6.80
CIFAR100-CTeBN84.68+7.51+5.46
TENT83.16+8.30+5.07
TTAC72.48+7.80+4.55
ImageNet-CTeBN81.23+6.82+6.35
TENT78.67+2.46+4.10
TTAC73.26+16.25+8.15

Overall, the compute-matched comparisons indicate that the average ASR of the strongest adapted baselines is 65.74% under Native-CM and 68.36% under FCU-CM, while PCAC achieves the highest ASR across all nine settings. PCAC obtains average ASR margins of +8.15 pp over the strongest Native-CM baselines and +5.53 pp over the strongest FCU-CM baselines. These findings support the complementary contributions of proxy-conditioned targeting and flatness optimization, in which JsJ_s encourages BaδB_a^\delta to induce target-oriented proxy predictions despite the saliency-enhanced contributions of the proxy set, for the enhancement of its adversarial effects when co-batched with unavailable benign samples BbB_b during TTA, while FCU optimizes $J_s$ over nearby perturbations and shifted surrogate states to reduce sensitivity to specific optimization conditions.

2. Summary of Revisions

We highlight the main revisions in blue within the paper.

2.1 Additional Experiments

ViT Adaptation and Auxiliary-Data Stability [aR6e]: We have added ViT-B/16 adaptation-step comparisons (Tables 9–10, Appendix C) and evaluations across five auxiliary domains with five independent sample draws each, showing a between-domain ASR standard deviation of 1.62 pp (Table 15, Appendix F.1).

Additional TTA Methods and OOD Benchmarks [odqk]: We have extended evaluations to TAST and LAME, achieving 64.36% and 50.75% ASR, respectively. We further evaluated ImageNet-V2 and ImageNet-A using shifted ImageNet-Sketch auxiliary data, achieving average ASRs of 70.39% and 61.26% across ten TTA methods (Tables 18–20, Appendix G).

2.2 Additional Details

Hyperparameter Selection [XFRn, aR6e]: We added our hyperparameter selection procedure for PCAC and the adapted baselines, which uses five-fold validation with auxiliary samples based on the surrogate's target-class predictions on held-out samples, without target-side feedback (Section 4.1; Appendix A, Table 6).

Presentation Improvements [aR6e, odqk]: We refined the main performance claims, added threat-model details, and incorporated explanatory descriptions, clearer notation, and equation references to improve readability.

We believe that our response has effectively addressed the concerns through clear explanations and additional experimental results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.