acceptodds
Under review as a conference paper at ICLR 2027

Back to Basics: Strengthening Targeted Data Poisoning through Sample Selection

Abstract

Targeted data poisoning attacks manipulate a small subset of training samples to induce a model to misclassify a specific test example as an attacker-chosen class. Existing attacks primarily focus on optimizing perturbations over a predefined set of training samples, which is often selected randomly, while paying limited attention to which samples should be poisoned. In this paper, we show that sample selection can substantially affect attack effectiveness, particularly when the poisoning budget is limited. To understand what makes a training sample effective for poisoning, we incorporate sample selection into the standard bilevel formulation of targeted data poisoning. Our analysis identifies two key characteristics of effective poisoning samples: low model confidence and high representation-space similarity to the target. Motivated by these insights, we propose BASIS (Boundary-Aware Selection Informed by Similarity), a lightweight plug-and-play sample selection method that strengthens existing poisoning attacks without modifying their perturbation optimization. Experiments across multiple attack methods and poisoning budgets demonstrate that BASIS consistently improves attack success rates through sample selection alone, with particularly substantial gains under small poisoning budgets.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Targeted data poisoning, clean-label poisoning, sample selection, adversarial machine learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_5iY4OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces BASIS - a sample selection strategy to strengthen existing targeted clean-label data poisoning attacks. The authors formulate the attack as a bilevel objective that jointly considers sample selection and perturbation - but acknowledge that it is intractable to optimize perturbations and retrain models over all possible subsets of candidate samples for each target. A heuristic proxy is proposed to select m candidate examples from the adversarial class as bases for poisoning. This is the sum of terms M and R, where a higher M is associated with candidates closer to a decision boundary and a higher R represents similarity between the target sample and candidate sample in feature space. The selected samples are then perturbed by an existing attack (BP, GM, or SAPA) without changing its perturbation optimization. BASIS achieves higher ASR than random selection on the CIFAR-10 dataset across varying poisoning budgets, differing model architectures and poisoning attacks, with additional results on CIFAR-100, SVHN and Tiny ImageNet in the appendix. They are able to do this while largely preserving clean test accuracy. The authors provide ablation experiments on model architecture mismatch between surrogate and victim models and the ensemble size of surrogate models used for selection, and compare against random selection under two victim-side defenses (Epic and Friends) and under victim-side data augmentation. They further compare BASIS with sample selection methods from backdoor attacks (Grad, Forget, FUS), ablations on the components of the score (R only, M only, and exact gradient alignment), and measure how closely BASIS agrees with exact gradient alignment in terms of selected samples.

Critical strengths
  • The paper proposes sampling for targeted clean-label data poisoning attacks to improve attack success rates. Few (if any) works have attempted it in this exact setting.
  • The heuristic is simple, seamless to integrate and fast in practice, as evidenced in the appendix experiments (specifically Appendix K). BASIS improves attack success rates over random selection consistently across 3 different attack settings. It achieves highest gains in the lowest sampling ratios.
  • The paper prioritizes thoroughness with detailed ablation experiments to investigate their own approach against downstream defenses, data augmentation, etc. - which adds credibility to their results. The 20 surrogate ensemble particularly adds robustness to experiments.
Critical weaknesses
  • Random is the only baseline in the main results. The quantity BASIS approximates in Section 3.2 - how a training update on one example changes a loss evaluated at a specific point is the same quantity computed by influence functions [1] and by the one-step meta-gradient in learning to reweight [2]. Those works use it for data cleaning and example weighting rather than attack, but the scoring machinery transfers directly: only the loss at the target changes. Gradient-norm selection has also already been used for base selection in (for instance) Sleeper Agent [3]. Adapting any of these would be a stronger comparison than random.
  • Most of the results in this paper are centered around the CIFAR-10 datasets. Experiments on real-world datasets are left to the appendix. Attacks studied in this work have included experiments on larger datasets / models (eg. Witches Brew [4] on ImageNet). The only tiny imagenet experiment that exists is inconclusive and with a single class pair, model and budget.
  • The related work section of this paper could benefit from literature including those from influence functions [1] in data poisoning, dirty label adversarial attacks and how sampling is used there, learning to reweight [2] and their formulation of the bilevel objective.
  • Data Augmentation in victim defenses shrink gains in data poisoning ASR (Table 3).

References

[1] Koh, P.W. and Liang, P., 2017, July. Understanding black-box predictions via influence functions. In International conference on machine learning (pp. 1885-1894). Pmlr.

[2] Ren, M., Zeng, W., Yang, B. and Urtasun, R., 2018, July. Learning to reweight examples for robust deep learning. In International conference on machine learning (pp. 4334-4343). PMLR.

[3] Souri, H., Fowl, L., Chellappa, R., Goldblum, M. and Goldstein, T., 2022. Sleeper agent: Scalable hidden trigger backdoors for neural networks trained from scratch. Advances in Neural Information Processing Systems, 35, pp.19165-19178.

[4] Geiping, J., Fowl, L., Huang, W.R., Czaja, W., Taylor, G., Moeller, M. and Goldstein, T., 2020. Witches' brew: Industrial scale data poisoning via gradient matching. arXiv preprint arXiv:2009.02276.

Questions for authors
  • How correlated are the R and M terms ? I would be curious to see the overlap, and what that means for data poisoning.
  • The relative usefulness of M and R may depend on the number of classes: M compares against a single strongest competitor and may become less informative as the class count grows, while R does not depend on the class count. The CIFAR-100 results (Table 11) show smaller and less consistent gains than CIFAR-10 and SVHN. Do the authors have any analysis of how the two components, and the choice λ = 1, behave as the number of classes increases?
  • The paper would benefit from explicit limitations and future work discussion. Several caveats are noted in passing but never collected. The authors should comment on how or when they envision their work being used, the practical implications given the strong assumptions in their threat model, and what this means for defenses.
5 comments

Dear Reviewer 5iY4,

We sincerely appreciate your comments, which have been invaluable in helping us improve our work.

W1: Comparison with additional baselines.

Beyond RAND and the comparisons in Table 4, we compare BASIS with the selection method used in Sleeper Agent [3] and with META-GRAD [2]. We evaluate dog→bird at \rho\in\\{0.2\\%,0.5\\%\\} using eight targets and five victim runs per target. The following table reports the results.

\hline & \text{Selection Method} & \substack{\text{BP} \\\\ 0.2} & \substack{\text{BP} \\\\ 0.5} & \substack{\text{GM} \\\\ 0.2} & \substack{\text{GM} \\\\ 0.5} & \substack{\text{SAPA} \\\\ 0.2} & \substack{\text{SAPA} \\\\ 0.5} \\\\ \hline \text{ConvNet} & \text{RAND} & 2.5 & 12.5 & 0.0 & 2.5 & 0.0 & 7.5 \\\\ & \text{META-GRAD [2]} & 62.5 & 75.0 & 37.5 & 62.5 & \mathbf{37.5} & \mathbf{62.5} \\\\ & \text{Sleeper Agent [3]} & 30.0 & 85.0 & 12.5 & 55.0 & 7.5 & 60.0 \\\\ & \text{BASIS} & \mathbf{70.0} & \mathbf{87.5} & \mathbf{40.0} & \mathbf{77.5} & 30.0 & \mathbf{62.5} \\\\ \hline \text{ResNet20} & \text{RAND} & 17.5 & 30.0 & 22.5 & 42.5 & 17.5 & 50.0 \\\\ & \text{META-GRAD [2]} & 22.5 & 30.0 & \mathbf{57.5} & 62.5 & \mathbf{52.5} & 77.5 \\\\ & \text{Sleeper Agent [3]} & \mathbf{35.0} & 30.0 & 45.0 & \mathbf{72.5} & 37.5 & \mathbf{85.0} \\\\ & \text{BASIS} & 20.0 & \mathbf{40.0} & \mathbf{57.5} & \mathbf{72.5} & \mathbf{52.5} & 77.5 \\\\ \hline \end{array}$$ As we can see, BASIS achieves the highest or tied-highest ASR in nine of twelve settings. These results support its effectiveness against stronger selection baselines. --- ## W2: Extension to large-scale datasets. Thank you for the suggestion. We expanded our Tiny ImageNet experiments to three class pairs and two poisoning budgets ($\rho\in\\{0.05\\%, 0.1\\%\\}$), as shown in the table below. The results consistently show that BASIS outperforms RAND under the GM attack, demonstrating that our gains persist on Tiny ImageNet. $$\begin{array}{lc|cc} \hline \text{Class pair} & \rho\\;(\\%) & \substack{\text{VGG13} \\\\ \text{RAND}} & \substack{\text{VGG13} \\\\ \text{BASIS}} \\\\ \hline \text{American alligator}\rightarrow\text{tarantula} & 0.1 & 72.0 & 96.0 \\\\ & 0.05 & 36.0 & 80.0 \\\\ \hline \text{Bullfrog}\rightarrow\text{dining table} & 0.1 & 48.0 & 92.0 \\\\ & 0.05 & 12.0 & 48.0 \\\\ \hline \text{Bullfrog}\rightarrow\text{beach wagon} & 0.1 & 16.0 & 40.0 \\\\ & 0.05 & 0.0 & 20.0 \\\\ \hline \end{array}$$ --- ## W3. Related Work extension. We thank the reviewer for these suggestions and have expanded the Related Work in Section 2, specifically the paragraph titled “Sample Selection in Adversarial Attacks,” to cover sample selection in dirty-label poisoning, the use of influence functions for poisoning attacks, and bilevel optimization approaches described in [2]. --- ## W4. Augmentation limitation. We thank the reviewer for pointing this out. Augmentation can reduce poisoning effectiveness, but our results show that improved sample selection remains useful under augmented victim training. In Table 3, BASIS outperforms or matches RAND in all augmented settings reported in Table 3, with substantial gains in several cases. For GM at $\rho=1\\%$, BASIS increases ASR from 5.0% to 63.2% on ResNet20 under RandAugment and from 60.0% to 90.0% on VGG13 under Crop+Flip. These results support our contribution of improving sample selection within existing poisoning attacks while showing that the magnitude of the benefit depends on the victim's training procedure.

Q1: Correlation and selection overlap between RR and MM.

We measure Pearson and Spearman correlations between R~i\tilde{R}_i and M~i\tilde{M}_i over all 5k5k candidates on CIFAR-10 (dog→bird), and then average the results across ten targets. Both tables report mean ± SD across targets. In addition, we report the top-mm selection overlap, ∣SR∩SM∣/m\lvert S_R\cap S_M\rvert/m, for different poisoning budgets.

Score Correlation

\hline \text{Architecture} & \text{Pearson} & \text{Spearman} \\\\ \hline \text{ConvNet} & 0.128\pm0.083 & 0.117\pm0.095 \\\\ \text{ResNet20} & 0.444\pm0.135 & 0.418\pm0.150 \\\\ \text{VGG13} & 0.311\pm0.150 & 0.313\pm0.163 \\\\ \hline \end{array}$$ **Top-$m$ Subset Overlap ($\lvert S_R\cap S_M\rvert/m$)** $$\begin{array}{lccccc} \hline \text{Architecture} & \rho=0.1\\% & \rho=0.2\\% & \rho=0.5\\% & \rho=1\\% & \rho=2\\% \\\\ \hline \text{ConvNet} & 0.014\pm0.016 & 0.046\pm0.032 & 0.098\pm0.040 & 0.164\pm0.047 & 0.267\pm0.044 \\\\ \text{ResNet20} & 0.030\pm0.034 & 0.068\pm0.044 & 0.124\pm0.064 & 0.217\pm0.081 & 0.358\pm0.076 \\\\ \text{VGG13} & 0.072\pm0.069 & 0.074\pm0.069 & 0.138\pm0.067 & 0.209\pm0.069 & 0.355\pm0.071 \\\\ \hline \end{array}$$ The weak-to-moderate correlations and limited top-$m$ overlap show that $R$ and $M$ favor different samples. $M$ measures classification difficulty relative to the strongest competing class, independently of the target $x_t$, while $R$ measures representation similarity to $x_t$. This distinction matters for targeted poisoning as a difficult sample may generate a strong training signal but have little effect on the target while a highly similar but confidently classified sample may have too little influence to meaningfully shift the decision boundary. Combining $R$ and $M$ accounts for both sample difficulty and target relevance. For example, Table 5 shows that on ConvNet/BP at $\rho=0.2\\%$, BASIS achieves 70.0% ASR, compared with 22.5% for $R$ alone and 45.0% for $M$ alone. These results suggest that considering both properties can make a fixed poisoning budget more effective. --- ## Q2. Effect of class count on the selection components. We thank the reviewer for this insightful question. We examined the relative contributions of $R$ and $M$ and the sensitivity to $\lambda$ across datasets with different class counts using the GM attack, as shown in the following table. We used 8 targets $\times$ 5 victims per target on CIFAR-10, 4 $\times$ 4 on CIFAR-100, and 5 $\times$ 5 on Tiny ImageNet. $$\begin{array}{lllcccc|ccc} \hline \text{Dataset }(C) & \text{Model} & \substack{\text{Class Pair} \\\\ (y_{\text{adv}}\rightarrow y_t)} & \rho\\;(\\%) & \text{RAND} & R & M & \substack{\text{BASIS} \\\\ \lambda=0.1} & \substack{\text{BASIS} \\\\ \lambda=1} & \substack{\text{BASIS} \\\\ \lambda=10} \\\\ \hline \substack{\text{CIFAR-10} \\\\ (C=10)} & \text{ConvNet} & \text{dog}\rightarrow\text{bird} & 0.2 & 0.0 & 15.0 & 22.5 & 25.0 & \mathbf{40.0} & 37.5 \\\\ & & \text{dog}\rightarrow\text{bird} & 0.5 & 2.5 & 20.0 & 55.0 & 47.5 & \mathbf{77.5} & 75.0 \\\\ \hline \substack{\text{CIFAR-100} \\\\ (C=100)} & \text{ResNet18} & \text{sea}\rightarrow\text{willow tree} & 0.1 & 12.5 & 31.3 & \mathbf{50.0} & 43.8 & 37.5 & 31.3 \\\\ & & \text{wardrobe}\rightarrow\text{lawn mower} & 0.2 & 6.3 & 25.0 & 12.5 & 18.8 & 25.0 & \mathbf{31.3} \\\\ & & \text{bottle}\rightarrow\text{road} & 0.2 & 0.0 & \mathbf{50.0} & 12.5 & 12.5 & 31.3 & 12.5 \\\\ \hline \substack{\text{Tiny ImageNet} \\\\ (C=200)} & \text{VGG13} & \text{American alligator}\rightarrow\text{tarantula} & 0.05 & 36.0 & 84.0 & 60.0 & \mathbf{88.0} & 80.0 & 76.0 \\\\ \hline \end{array}$$ The results suggest a shift in relative usefulness. On CIFAR-10, $M$ outperforms $R$, whereas $R$ is stronger in most higher-class settings. One possible explanation is that the strongest competing class used by $M$ becomes less relevant to the specific target as the number of classes increases. However, these experiments also vary in architecture and class pair, so they do not isolate class count as the cause. The preferred $\lambda$ is likewise setting-dependent. Down-weighting $M$ helps with the evaluated Tiny ImageNet pair, but CIFAR-100 shows no consistent preference. The untuned $\lambda=1$ improves over random selection throughout these experiments, although it is not always optimal. Overall, these findings support the effectiveness of BASIS across different class counts while highlighting the setting-dependent contributions of its components. Since attackers typically cannot tune $\lambda$ using victim-model outcomes, we set $\lambda=1$ to balance both components and achieve consistent improvements across evaluations without setting-specific tuning. We have also included a new subsection 4.8 in the draft to cover this analysis.

Q3. Limits and future work.

We agree and have added a Limitations and Future Work discussion immediately before the “Conclusion” in the draft to unify these points. Our threat model assumes access to the full clean training set and the ability to replace selected samples before victim training, which may be infeasible in constrained data pipelines. For defenders, our results show that robustness evaluations based only on randomly selected poisons can overestimate the protection provided by existing defenses. Future work will examine base selection under partial observability and investigate defenses that exploit the low-margin, high-similarity patterns identified in our analysis.

Reviewer_pvQKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies targeted clean-label data poisoning. Existing attacks usually optimise perturbations on poison bases that are typically chosen arbitrarily. The authors add sample selection to the bilevel poisoning objective. They then argue from a first-order expansion of clean-sample gradient alignment that good bases have low confidence, measured by a competing-class logit margin M, and high feature similarity to the target R. They propose BASIS which standardises both quantities over K = 20 surrogates, adds them, and passes the top-m candidates to an unmodified attack. On CIFAR-10 with ConvNet, ResNet20 and VGG13, mean attack success rate rises from 20.3% to 44.7%, improving 103 of 108 configurations with clean accuracy essentially unchanged. Further experiments cover architecture mismatch, the EPIC and FRIENDS defences, augmentation, backdoor-style selectors, ablations and other datasets.

Critical strengths
  1. The paper provides a simple attack add-on with large gains at small budgets, especially at small budgets. For example, ConvNet BP at p = 0.2% goes from 1.7% to 65.0%. Because BASIS is plug-and-play, the result is directly relevant to anyone using current attacks as benchmarks.

  2. The attack provides implications for how defences should be evaluated. The paper shows that BASIS keeps an advantage under EPIC and FRIENDS, including when defence strength is calibrated to clean accuracy. This supports a useful point that evaluating defences only against randomly selected poisons may overstate how much protection they give, at least in the non-adaptive setting tested.

  3. The experimental results are extensive and honest. The authors tested with various ablation settings which provides a clear picture of the method.

Critical weaknesses
  1. The novelty of this method is limited. Choosing low-confidence or hard samples is already well established in clean-label backdoor selection, the genuinely target-conditioned term R contributes to the novelty of this work. However, as shown in the experiments, the M alone recovers most of the BASIS's gain, and R only adds a modest and inconsistent amount. This raises concern over the novelty and the effectiveness of the novel part of the method.

  2. The analysis consists of a chain of heuristically motivations and some of them are not fully justified. Although the author has acknowledged the approximations, the rigorousness of the theoretical analysis remains a concern to me, such as the claim that the direction of r_i relative to r_t "carries little meaning". The paper's reason for ignoring direction is that the two residuals describe confidence on different inputs. But the term is large when the candidate and the target put probability on the same wrong classes. The target is usually predicted as its true class yty_t, so this term is roughly pi(yt),pt(yt)p_i(y_t),p_t(y_t). In practice it rewards candidates that the model already thinks look like the target's class.

  3. Results are compared across inconsistent evaluations. In Tables 4 and 5, the RAND entries for ConvNet dog - bird GM/SAPA match Table 1 exactly (0.0, 1.7, 0.0, 6.7). While the BASIS entries do not: GM at 0.2%/0.5% is 40.0/77.5 in Table 4 vs 28.3/73.3 in Table 1, and SAPA is 30.0/62.5 vs 26.7/48.3.

  4. Weak baseline and a narrow source of variance. Each target uses a single random base set and a single poison set, so the confidence intervals ignore variation from the random draw, which is exactly what is being compared against. RAND should be averaged over several independent draws per target, and a deterministic baseline matching the original attacks' protocol should be included.

  5. Defence evaluation misses the defences most likely to catch BASIS poisons. BASIS deliberately picks y_adv samples that are low-confidence and close to the target in feature space, so they are near the class boundary. Feature-space filtering defences, such as Deep k-NN, activation clustering, adversarial training, etc, are more natural candidates to remove such points preferentially, but they were not evaluated.

Questions for authors

As in Weaknesses.

3 comments

Dear Reviewer pvQK,

Thanks for your kind comments, which helped us improve the paper.

W1. Novelty and contribution of target-specific similarity

We answer this question in two parts.

Part 1: Comparison with backdoor attacks

In backdoor attacks, low-confidence samples are selected to encourage the model to learn the injected trigger. On the contrary, targeted data poisoning addresses a different problem, as the goal is to cause the model to misclassify a specific, unmodified test sample without a trigger. As a result, this makes our method fundamentally different from existing works in backdoor attacks in two aspects:

  • Different definition of difficult samples: FORGET [1] identifies difficult samples as those for which the model goes from classifying the sample correctly to classifying it incorrectly. In contrast, our method does not rely on any such prediction change. Instead, we use the margin between the true class and its strongest competitor to identify difficult samples. These can include samples near the decision boundary as well as samples that the model confidently assigns to another class.

  • Target-aware candidate selection: We also consider each candidate's similarity to the target, avoiding the selection of difficult samples that are unrelated to the target of interest.

We empirically compared FORGET [1], FUS [2], and the gradient-norm criterion used by Sleeper Agent [3] with RR, MM, and BASIS. The following table reports selection statistics on CIFAR-10 using six clean evaluation networks and eight shared targets. The first two numerical columns report mean margin MM and target similarity RR, with percentile ranks among all 5k eligible candidates in parentheses. Higher MM indicates harder samples, while higher RR indicates greater similarity to the target.

\hline \text{Method} & \substack{\text{Mean margin }M\\,\uparrow \\\\ \text{(pool percentile)}} & \substack{\text{Mean similarity }R\\,\uparrow \\\\ \text{(pool percentile)}} & \substack{\text{Overlap with} \\\\ \text{BASIS }\uparrow} & \text{Mean ASR (\\%)}\\,\uparrow \\\\ \hline \text{Full candidate pool} & -12.8966\\,(50.0) & 0.2436\\,(50.0) & \text{–} & \text{–} \\\\ \hline \text{FORGET} & -7.8112\\,(79.4) & 0.2463\\,(51.1) & 0.0625 & 33.9 \\\\ \text{FUS} & -11.5914\\,(56.5) & 0.2577\\,(57.3) & 0.0563 & 24.4 \\\\ \text{Sleeper Agent} & -8.9589\\,(70.7) & 0.2489\\,(52.4) & 0.0650 & 35.0 \\\\ R & -13.1490\\,(47.7) & \mathbf{0.3578\\,(96.4)} & 0.3137 & 27.7 \\\\ M & \mathbf{-6.5560\\,(89.9)} & 0.2473\\,(51.0) & 0.0963 & 46.0 \\\\ \text{BASIS} & -8.7009\\,(72.4) & 0.3353\\,(90.0) & 1.0000 & \mathbf{53.3} \\\\ \hline \end{array}$$ Samples selected using $M$ do not necessarily have high target similarity, and samples with high similarity are not necessarily difficult. Also, samples selected by the three benchmarks above do not necessarily have high target similarity either. By integrating both components, BASIS achieves better ASR than any of the other methods listed in the table. **Part 2:** To address the second part of the question, we examined the correlation and selection overlap between $R$ and $M$ across ten CIFAR-10 targets. $$\begin{array}{lccc} \hline \text{Architecture} & \text{Pearson} & \substack{\text{Top-}m\text{ overlap (\\%)} \\\\ \rho=0.2\\%} & \substack{\text{Top-}m\text{ overlap (\\%)} \\\\ \rho=0.5\\%} \\\\ \hline \text{ConvNet} & 0.128 & 4.6 & 9.8 \\\\ \text{ResNet20} & 0.444 & 6.8 & 12.4 \\\\ \text{VGG13} & 0.311 & 7.4 & 13.8 \\\\ \hline \end{array}$$ These correlations and limited overlap further support that the components favor different candidates. Also, as the number of classes grows, the similarity term can contribute more because it evaluates target similarity regardless of class count. We evaluate this on CIFAR-100 and Tiny ImageNet (with 100 and 200 classes) using the GM attack, as shown below. $$\begin{array}{lllcccc} \hline \text{Dataset} & \text{Model} & \text{Class Pair }(y_{\text{adv}}\rightarrow y_t) & \rho\\;(\\%) & \text{RAND} & R & M \\\\ \hline \text{CIFAR-100} & \text{ResNet18} & \text{sea}\rightarrow\text{willow tree} & 0.1 & 12.5 & 31.3 & \mathbf{50.0} \\\\ \text{CIFAR-100} & \text{ResNet18} & \text{wardrobe}\rightarrow\text{lawn mower} & 0.2 & 6.3 & \mathbf{25.0} & 12.5 \\\\ \text{CIFAR-100} & \text{ResNet18} & \text{bottle}\rightarrow\text{road} & 0.2 & 0.0 & \mathbf{50.0} & 12.5 \\\\ \text{Tiny ImageNet} & \text{VGG13} & \text{American alligator}\rightarrow\text{tarantula} & 0.05 & 36.0 & \mathbf{84.0} & 60.0 \\\\ \hline \end{array}$$ These findings indicate that $R$ captures distinct, target-specific information that can improve attack effectiveness over random selection. We also have added a new subsection 4.8 to the draft to address how the interaction between $R$ and $M$ changes as the number of classes increases.

W2. Justification for omitting the residual direction

Our statement in the paper was made for the setting in which ∥ri∥≈0\lVert r_i\rVert\approx 0. Your observation is also valid in other scenarios, but we argue that such scenarios occur less frequently in practice. To examine this, we considered the different cases below and measured the frequency of each across 10 trained surrogates. We evaluated all CIFAR-10 bird test samples as targets, one by one, using 5k dog candidates and averaged the results.

First, we categorize the possible cases according to the predicted classes of the candidate and target. Under the high-confidence approximation,

ri⊤rt=(1−pi(yadv))(1−pt(yadv))+∑c≠yadvpi(c)pt(c).r_i^\top r_t = (1-p_i(y_{\mathrm{adv}}))(1-p_t(y_{\mathrm{adv}})) + \sum_{c\neq y_{\mathrm{adv}}} p_i(c)p_t(c).

Case 1: Both samples are correctly classified.

x_i\to y_{\mathrm{adv}} \\\\ x_t\to y_t \end{array} \quad\Longrightarrow\quad r_i^\top r_t\approx 0.$$ **Case 2: The target is correctly classified and the candidate is classified as the target class.** $$\begin{array}{l} x_i\to y_t \\\\ x_t\to y_t \end{array} \quad\Longrightarrow\quad r_i^\top r_t\approx 1+p_i(y_t)p_t(y_t)\approx 2.$$ **Case 3: The target is correctly classified and the candidate is classified as a third class.** $$\begin{array}{l} x_i\to y_c,\quad y_c\notin\\{y_{\mathrm{adv}},y_t\\} \\\\ x_t\to y_t \end{array} \quad\Longrightarrow\quad r_i^\top r_t\approx 1.$$ **Case 4: The candidate and target are classified as the same third class.** $$\begin{array}{l} x_i\to y_c \\\\ x_t\to y_c,\quad y_c\notin\\{y_{\mathrm{adv}},y_t\\} \end{array} \quad\Longrightarrow\quad r_i^\top r_t\approx 1+p_i(y_c)p_t(y_c)\approx 2.$$ **Case 5: The target is classified as the adversarial class.** $$x_t\to y_{\mathrm{adv}}\quad\Longrightarrow\quad r_t\approx 0\quad\Longrightarrow\quad r_i^\top r_t\approx 0,$$ independently of the candidate's predicted class. **Case 6: The target is classified as a third class and the candidate is classified as a different class.** $$\begin{array}{l} x_t\to y_c,\quad y_c\notin\\{y_{\mathrm{adv}},y_t\\} \\\\ x_i\not\to y_c \end{array}$$ This case contains two subcases $$\begin{array}{lll} \text{Case 6a:} & x_i\to y_{\mathrm{adv}} & \Longrightarrow\quad r_i^\top r_t\approx 0, \\\\ \text{Case 6b:} & x_i\to y_d,\quad y_d\notin\\{y_{\mathrm{adv}},y_c\\} & \Longrightarrow\quad r_i^\top r_t\approx 1. \end{array}$$ The table below presents the results as percentages. $$\begin{array}{lcccccc} \hline \text{Model} & \text{C1} & \text{C2} & \text{C3} & \text{C4} & \text{C5} & \text{C6} \\\\ \hline \text{ConvNet} & 70.921 & 0.048 & 0.351 & 0.025 & 4.540 & 24.115 \\\\ \text{ResNet20} & 75.860 & 0.000 & 0.000 & 0.000 & 3.500 & 20.640 \\\\ \text{VGG13} & 78.260 & 0.000 & 0.000 & 0.000 & 3.200 & 18.540 \\\\ \hline \end{array}$$ Additionally, in Table 17 of Appendix H of our current manuscript, we compared BASIS with classifier-gradient alignment. We replicate those results here $$\begin{array}{l|cc|cc|cc} \hline \text{Method} & \substack{\text{BP} \\\\ 0.1} & \substack{\text{BP} \\\\ 0.2} & \substack{\text{GM} \\\\ 0.2} & \substack{\text{GM} \\\\ 0.5} & \substack{\text{SAPA} \\\\ 0.1} & \substack{\text{SAPA} \\\\ 0.2} \\\\ \hline \text{RAND} & 0.0 & 1.7 & 0.0 & 1.7 & 0.0 & 0.0 \\\\ (r_i^\top r_t)(h_i^\top h_t) & 29.2 & 62.5 & 29.2 & 66.7 & 16.7 & 31.3 \\\\ \text{BASIS} & \mathbf{33.3} & \mathbf{70.8} & \mathbf{41.7} & \mathbf{79.2} & \mathbf{29.2} & 31.3 \\\\ \hline \end{array}$$ BASIS achieved higher ASR than classifier-gradient alignment in five of the six settings and tied in the sixth. Selecting candidates based on a specific class confusion in the surrogate model may overfit its predictions and may not generalize to the victim. Instead, $M_i$ favors candidates that are difficult to classify into $y_{\mathrm{adv}}$, regardless of which competing class is predicted. It may include extra candidates, but this broader selection reduces dependence on the surrogate's specific class predictions and improves transferability. --- ## W3: Evaluation inconsistency. We thank the reviewer for this suggestion. We have now added to Table 5 caption the exact number of victims and targets used in each configuration.

W4: Random sampling variance.

On CIFAR-10 (dog→bird), we add the deterministic FIRST-mm protocol and average RAND over six independent sample set draws per target, reporting ASR (SE). In the tables below, we can observe that BASIS achieves higher mean ASR than both baselines across all settings, showing that its gains persist beyond a single random draw.

ConvNet

\hline \text{Selection Protocol} & \substack{\text{GM} \\\\ 0.1} & \substack{\text{GM} \\\\ 0.2} & \substack{\text{GM} \\\\ 0.5} & \substack{\text{SAPA} \\\\ 0.1} & \substack{\text{SAPA} \\\\ 0.2} & \substack{\text{SAPA} \\\\ 0.5} \\\\ \hline \text{FIRST-}m\text{ (Deterministic protocol)} & 0.0\\,(0.0) & 1.7\\,(1.7) & 5.0\\,(3.6) & 1.7\\,(1.7) & 0.0\\,(0.0) & 10.0\\,(5.7) \\\\ \text{RAND (Table 1: 1 draw/target)} & 0.0\\,(0.0) & 0.0\\,(0.0) & 1.7\\,(1.7) & 0.0\\,(0.0) & 0.0\\,(0.0) & 6.7\\,(3.7) \\\\ \text{RAND }(D=6\text{ draws/target}) & 0.3\\,(0.3) & 1.1\\,(0.6) & 5.3\\,(2.2) & 0.3\\,(0.3) & 0.8\\,(0.6) & 5.3\\,(2.4) \\\\ \hline \text{BASIS} & \mathbf{11.7}\\,(5.6) & \mathbf{28.3}\\,(6.6) & \mathbf{73.3}\\,(9.4) & \mathbf{21.7}\\,(5.6) & \mathbf{26.7}\\,(7.5) & \mathbf{48.3}\\,(11.8) \\\\ \hline \end{array}$$ **ResNet20** $$\begin{array}{l|ccc|ccc} \hline \text{Selection Protocol} & \substack{\text{GM} \\\\ 0.1} & \substack{\text{GM} \\\\ 0.2} & \substack{\text{GM} \\\\ 0.5} & \substack{\text{SAPA} \\\\ 0.1} & \substack{\text{SAPA} \\\\ 0.2} & \substack{\text{SAPA} \\\\ 0.5} \\\\ \hline \text{FIRST-}m\text{ (Deterministic protocol)} & 15.0\\,(5.2) & 33.3\\,(6.6) & 33.3\\,(9.0) & 18.3\\,(9.1) & 18.3\\,(5.8) & 43.3\\,(10.6) \\\\ \text{RAND (Table 1: 1 draw/target)} & 15.0\\,(5.2) & 31.7\\,(8.4) & 43.3\\,(7.5) & 18.3\\,(6.3) & 21.7\\,(7.9) & 55.0\\,(10.6) \\\\ \text{RAND }(D=6\text{ draws/target}) & 16.1\\,(3.8) & 27.2\\,(6.4) & 41.1\\,(7.0) & 17.2\\,(4.2) & 20.6\\,(5.4) & 41.9\\,(6.8) \\\\ \hline \text{BASIS} & \mathbf{35.0}\\,(11.2) & \mathbf{56.7}\\,(13.2) & \mathbf{76.7}\\,(7.9) & \mathbf{43.3}\\,(5.7) & \mathbf{53.3}\\,(12.6) & \mathbf{78.3}\\,(12.2) \\\\ \hline \end{array}$$ --- ## W5: Additional defenses. We evaluate GM under Deep $k$-NN, activation clustering, and adversarial training using eight targets and five victims per target, reporting ASR (%) with CTA (%) in parentheses. BASIS maintains a higher ASR under activation clustering on both models and Deep $k$-NN on ResNet20. Adversarial training and Deep $k$-NN on ConvNet largely suppress both methods, at the cost of reduced CTA. $$\begin{array}{lc|cc|cc|cc|cc} \hline \text{Model} & \rho\\;(\\%) & \substack{\text{No defense} \\\\ \text{RAND}} & \substack{\text{No defense} \\\\ \text{BASIS}} & \substack{\text{Deep }k\text{-NN} \\\\ \text{RAND}} & \substack{\text{Deep }k\text{-NN} \\\\ \text{BASIS}} & \substack{\text{Act. Clustering} \\\\ \text{RAND}} & \substack{\text{Act. Clustering} \\\\ \text{BASIS}} & \substack{\text{Adv. Training} \\\\ \text{RAND}} & \substack{\text{Adv. Training} \\\\ \text{BASIS}} \\\\ \hline \text{ConvNet} & 0.1 & 0.0\\,(80.44) & 20.0\\,(80.29) & 0.0\\,(57.69) & 0.0\\,(61.02) & 0.0\\,(66.81) & 30.0\\,(68.13) & 0.0\\,(66.17) & 2.5\\,(61.99) \\\\ & 0.2 & 0.0\\,(80.47) & 40.0\\,(80.38) & 0.0\\,(55.47) & 0.0\\,(60.30) & 0.0\\,(68.31) & 50.0\\,(67.70) & 0.0\\,(65.82) & 5.0\\,(61.93) \\\\ & 0.5 & 2.5\\,(80.59) & 77.5\\,(80.41) & 0.0\\,(61.35) & 0.0\\,(61.00) & 12.5\\,(67.58) & 75.0\\,(67.77) & 7.5\\,(61.88) & 5.0\\,(61.74) \\\\ \hline \text{ResNet20} & 0.1 & 17.5\\,(83.78) & 40.0\\,(83.61) & 12.5\\,(83.58) & 27.5\\,(83.74) & 12.5\\,(79.34) & 27.5\\,(79.28) & 5.0\\,(71.66) & 7.5\\,(71.69) \\\\ & 0.2 & 22.5\\,(83.86) & 57.5\\,(83.79) & 37.5\\,(83.73) & 47.5\\,(83.82) & 22.5\\,(79.24) & 40.0\\,(79.37) & 7.5\\,(71.59) & 0.0\\,(71.67) \\\\ & 0.5 & 42.5\\,(83.71) & 72.5\\,(83.81) & 35.0\\,(83.73) & 75.0\\,(83.77) & 32.5\\,(79.17) & 52.5\\,(79.29) & 12.5\\,(71.65) & 12.5\\,(71.60) \\\\ \hline \end{array}$$ --- ### References [1] Yinghua Gao, Yiming Li, Linghui Zhu, Dongxian Wu, Yong Jiang, and Shu-Tao Xia. Not all samples are born equal: Towards effective clean-label backdoor attacks. Pattern Recognition, 139:109512, 2023. ISSN 0031-3203. doi: https://doi.org/10.1016/j.patcog.2023.109512. [2] Pengfei Xia, Ziqiang Li, Wei Zhang, and Bin Li. Data-efficient backdoor attacks. In Lud De Raedt (ed.), Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, pp. 3992–3998. International Joint Conferences on Artificial Intelligence Organization, 2022. doi: 10.24963/ijcai.2022/554. [3] Souri, H., Fowl, L., Chellappa, R., Goldblum, M. and Goldstein, T., 2022. Sleeper agent: Scalable hidden trigger backdoors for neural networks trained from scratch. Advances in Neural Information Processing Systems, 35, pp. 19165-19178.
Reviewer_FJBwOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper argues that sample selection matters in targeted data poisoning. It proposes BASIS, which ranks candidate samples by low classifier margin plus feature similarity to the target, averaged over surrogate models. BASIS plugs into BP, GM and SAPA and substantially raises attack success, especially at small budgets.

Critical strengths
  • Simple, cheap, plug-and-play idea that gives significant improvements over random selection at small budgets.
  • Shows why evaluating defenses against randomly selected poisons may give a false sense of security.
  • Evaluation on several datasets, architecture mismatch, augmentation, defenses and ablations, with target-level bootstrap CIs reported
Critical weaknesses
  • The attacker has the full clean training set and can choose which existing samples get replaced which is unrealistic.
  • limited baselines: Recent works are missing from the evaluations e.g., Wicked Oddities (ICLR 2025) is cited but not used in teh evaluation. Defenses that detect poisoned samples by inspecting the loss e.g., Try to Poison My Deep Learning Data? Nowhere to Hide Your Trajectory Spectrum! In NDSS 2025, Backdoor Defense via Adaptively Splitting Poisoned Dataset, In CVPR 2023.
  • Table 5 shows that the margin term M alone accounts for most of BASIS’s improvement over random selection, while adding the similarity term R yields only modest further gains.
Questions for authors
  • Does BASIS preference for low-margin samples make its poisons easier to detect by loss-based defenses such as ASD, relative to random selection? ASD: Backdoor Defense via Adaptively Splitting Poisoned Dataset, In CVPR 2023.
  • The threat model assumes access to the full clean training set. How does BASIS perform when (a) surrogates are trained on only a fraction of the data, and (b) the candidate pool is restricted to the subset the attacker can modify? The latter seems closer to realistic settings where an attacker contributes only part of the data
  • How does BASIS compare with Wicked Oddities (cited but not evaluated) ?
2 comments

Dear Reviewer FJBw,

We deeply appreciate your thoughtful feedback on our work, here are our answers.

W1 & Q2: Limited data access.

We restrict (a) surrogate-training data and (b) candidate pools, separately and jointly, using BP on dog→bird (ρ=0.2\rho=0.2\\%, 100 poisons). Both methods share the same candidate pool, with RAND fixed across surrogate-data settings.

New experiments use seven targets and eight victims per target, averaging over three RAND selections and three surrogate-data draws for (a). Full-access results come from Table 2. Entries report ASR (SE).

\hline \substack{\text{Candidate} \\\\ \text{pool (\\%)}} & \substack{\text{Surrogate} \\\\ \text{data (\\%)}} & \substack{\text{ConvNet BP} \\\\ \text{RAND}} & \substack{\text{ConvNet BP} \\\\ \text{BASIS}} & \substack{\text{ResNet20 BP} \\\\ \text{RAND}} & \substack{\text{ResNet20 BP} \\\\ \text{BASIS}} \\\\ \hline 100 & 100 & 1.7\\,(1.7) & 65.0\\,(7.2) & 16.7\\,(5.0) & 31.2\\,(13.9) \\\\ 100 & 50 & 1.7\\,(1.7) & 49.8\\,(4.8) & 16.7\\,(5.0) & 20.4\\,(8.0) \\\\ 100 & 20 & 1.7\\,(1.7) & 30.8\\,(4.5) & 16.7\\,(5.0) & 20.0\\,(5.3) \\\\ \hline 50 & 100 & 4.8\\,(2.1) & 53.2\\,(8.4) & 17.9\\,(7.1) & 26.2\\,(7.9) \\\\ 50 & 50 & 4.8\\,(2.1) & 48.0\\,(6.6) & 17.9\\,(7.1) & 25.2\\,(8.2) \\\\ \hline 20 & 100 & 4.2\\,(2.2) & 39.3\\,(6.1) & 15.5\\,(4.4) & 27.8\\,(8.5) \\\\ 20 & 20 & 4.2\\,(2.2) & 16.3\\,(5.5) & 15.5\\,(4.4) & 16.5\\,(5.2) \\\\ \hline \end{array}$$ **BASIS improves ASR over RAND across all evaluated settings**, supporting the effectiveness of sample selection under limited access. BASIS's advantage generally narrows as access to data decreases, but even in the most restrictive setting, it still outperforms RAND. --- ## W2 & Q1: Additional defenses (ASD and Telltale (NDSS 2025)). We evaluate GM on dog→bird under ASD and Telltale defenses, reporting ASR (%) with CTA (%) in parentheses. Defended BASIS results use eight targets and five victims per target; undefended ASRs come from Tables 4 and 5. $$\begin{array}{lc|cc|cc|cc} \hline \text{Model} & \rho\\;(\\%) & \substack{\text{No defense} \\\\ \text{RAND}} & \substack{\text{No defense} \\\\ \text{BASIS}} & \substack{\text{ASD} \\\\ \text{RAND}} & \substack{\text{ASD} \\\\ \text{BASIS}} & \substack{\text{Telltale} \\\\ \text{RAND}} & \substack{\text{Telltale} \\\\ \text{BASIS}} \\\\ \hline \text{ConvNet} & 0.1 & 0.0\\,(80.34) & 20.0\\,(80.29) & 0.0\\,(85.66) & 0.0\\,(83.71) & 0.0\\,(76.91) & 7.5\\,(79.53) \\\\ & 0.2 & 0.0\\,(80.38) & 40.0\\,(80.38) & 0.0\\,(85.91) & 0.0\\,(84.21) & 0.0\\,(80.18) & 20.0\\,(79.35) \\\\ & 0.5 & 1.7\\,(80.46) & 77.5\\,(80.41) & 0.0\\,(84.41) & 15.0\\,(84.16) & 1.7\\,(79.07) & 22.5\\,(78.72) \\\\ \hline \text{ResNet20} & 0.1 & 15.0\\,(83.76) & 40.0\\,(83.61) & 3.3\\,(84.47) & 2.5\\,(84.74) & 5.0\\,(83.32) & 37.5\\,(83.27) \\\\ & 0.2 & 31.7\\,(83.88) & 57.5\\,(83.79) & 5.0\\,(84.62) & 12.5\\,(84.04) & 21.7\\,(82.07) & 47.5\\,(83.18) \\\\ & 0.5 & 43.3\\,(83.79) & 72.5\\,(83.81) & 5.0\\,(84.11) & 5.0\\,(84.78) & 48.3\\,(83.09) & 62.5\\,(82.86) \\\\ \hline \end{array}$$ These loss-based defenses reduce attack success, but **BASIS still achieves higher post-defense ASR than RAND in many settings**, showing that its selection advantage can persist under these defenses. --- ## W2, Q3: Comparison with Wicked Oddities (ICLR 2025). Wicked Oddities (WO) selects intra-class outliers from $y_{\text{adv}}$ using either $k$-NN distance (WO-KNN) or loss against an external OOD dataset (WO-OOD). We test both variants of Wicked Oddities on dog→bird pair, using ConvNet and ResNet20 at $\rho\in\\{0.2\\%,0.5\\%\\}$. $$\begin{array}{ll|cc|cc} \hline \text{Model} & \text{Method} & \substack{\text{GM} \\\\ 0.2\\%} & \substack{\text{GM} \\\\ 0.5\\%} & \substack{\text{SAPA} \\\\ 0.2\\%} & \substack{\text{SAPA} \\\\ 0.5\\%} \\\\ \hline \text{ConvNet} & \text{RAND} & 0.0 & 1.7 & 0.0 & 6.7 \\\\ & \text{WO-KNN} & 2.5 & 35.0 & 5.0 & 32.5 \\\\ & \text{WO-OOD} & 0.0 & 12.5 & 5.0 & 27.5 \\\\ & \text{BASIS} & \mathbf{40.0} & \mathbf{77.5} & \mathbf{30.0} & \mathbf{62.5} \\\\ \hline \text{ResNet20} & \text{RAND} & 31.7 & 43.3 & 21.7 & 55.0 \\\\ & \text{WO-KNN} & 45.0 & 62.5 & 40.0 & 65.0 \\\\ & \text{WO-OOD} & 35.0 & 42.5 & 22.5 & 47.5 \\\\ & \text{BASIS} & \mathbf{57.5} & \mathbf{72.5} & \mathbf{52.5} & \mathbf{77.5} \\\\ \hline \end{array}$$ **BASIS outperforms both WO variants across all evaluated settings above.** Although WO-KNN consistently improves over RAND, its gains remain smaller than those achieved by BASIS.

W3. Contribution of the similarity term.

BASIS uses MM to identify low-margin candidates and RR to account for their similarity to the target representation. While MM accounts for much of the improvement over RAND, a low margin may involve a competing class unrelated to the target, so RR provides additional target-specific information. For example, adding RR increases ConvNet/GM ASR from 55.0% to 77.5% at ρ=0.5\rho=0.5\\% in Table 5.

To further examine this distinction, we measure the Pearson correlation between RR and MM and their top-mm selection overlap, ∣SR∩SM∣/m\lvert S_R \cap S_M\rvert/m, on CIFAR-10 (dog→bird), averaging across ten targets. The weak-to-moderate correlations and low overlap shown below indicate that the two terms favor different candidates.

\hline \text{Architecture} & \text{Pearson }r & \substack{\text{Top-}m\text{ overlap (\\%)} \\\\ \rho=0.2\\%} & \substack{\text{Top-}m\text{ overlap (\\%)} \\\\ \rho=0.5\\%} \\\\ \hline \text{ConvNet} & 0.128 & 4.6 & 9.8 \\\\ \text{ResNet20} & 0.444 & 6.8 & 12.4 \\\\ \text{VGG13} & 0.311 & 7.4 & 13.8 \\\\ \hline \end{array}$$ The relative importance of these terms also varies across datasets with more classes. While $M$ considers the strongest competing class, $R$ measures target similarity without depending on the number of classes. In the CIFAR-100 and Tiny ImageNet (200 classes) experiment below, $R$ outperforms $M$ (we used the GM attack). This suggests that the margin term is not always the more effective component. $$\begin{array}{lllcccc} \hline \text{Dataset} & \text{Model} & \text{Class Pair }(y_{\text{adv}}\rightarrow y_t) & \rho\\;(\\%) & \text{RAND} & R & M \\\\ \hline \text{CIFAR-100} & \text{ResNet18} & \text{wardrobe}\rightarrow\text{lawn mower} & 0.2 & 6.3 & 25.0 & 12.5 \\\\ \text{CIFAR-100} & \text{ResNet18} & \text{bottle}\rightarrow\text{road} & 0.2 & 0.0 & 50.0 & 12.5 \\\\ \text{Tiny ImageNet} & \text{VGG13} & \text{American alligator}\rightarrow\text{tarantula} & 0.05 & 36.0 & 84.0 & 60.0 \\\\ \hline \end{array}$$ Overall, these results suggest that **$R$ and $M$ capture complementary aspects of sample selection** and their relative contributions vary across settings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.