Poisoning Attack Framework Against Test-Time Adaptation

Test-time adaptation (TTA) enables batch-wise updates by incorporating distributional characteristics from unlabeled test batches to enhance model generalization. The co-batching of poisoned samples can mislead a target model after adaptation, forcing attacker-intended predictions on benign co-batched test samples. Nonetheless, most attacks study this poisoning risk under extra requirements, such as access to real benign co-batch samples, the target model or queries. Such requirements tailor attacks to specific targets and hinder assessments of TTA poisoning risks when target-side access is unavailable. In this paper, we propose PCAC, a targeted poisoning framework without relying on the above requirements. PCAC departs from prior attacks by exploiting distribution-shifted auxiliary data for surrogate-side crafting. PCAC augments auxiliary data by emphasizing salient regions, creating a diverse proxy set that provides influential co-batch contexts. PCAC then induces a target-confident effect and smooths surrogate-side crafting to discourage center-specific and state-sensitive solutions. With proxy diversity and flatness-enhanced target-effect crafting, PCAC generates transferable poisoned test samples. Experiments across diverse TTA settings show that PCAC achieves 2.47× ASR of prior attacks without target-side access during crafting.