acceptodds
Under review as a conference paper at ICLR 2027

Pick Your Poison: Learning to Select Poison Sets for Stronger LLM Backdoor Attacks

Abstract

Backdoor poisoning attacks add poisoned examples to otherwise-clean finetuning data, pairing a trigger with a target behavior that the model learns to produce when the trigger appears. Existing evaluations typically fix the number of poisoned examples and sample them at random from a candidate pool. We show that this can severely underestimate worst-case vulnerability: across three LLaMA-3-8B backdoor settings, holding the model, clean data, and poison count fixed, attack success ranges from 3% to 80% depending only on which poison set is chosen. We formalize poison selection as oracle-budgeted set optimization and introduce SAILS (Set-level Audit- Informed Iterative Learned Selection), which learns a set scorer from a few hundred finetune-and-evaluate runs, ranks millions of candidate sets, and audits only a small shortlist. SAILS improves held-out attack success by 30 percentage points on average over the strongest influence baselines, transfers from small-scale to full-scale finetuning, and extends to code-generation, agentic, and API-only backdoors.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large language models, backdoor attacks, data poisoning, poison selection, set optimization, surrogate modeling, influence functions

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_sdhuOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper motivates to select a representative poisoning set, which is often an overlooked factor in LLM backdoor evaluation. The authors propose SAILS attack that learns to select more effective poison sets, outperforming influence-based baselines and demonstrating that random selection can substantially underestimate backdoor vulnerability.

Critical strengths
  1. Selecting an effective poisoning set is a better attack approach.

I appreciate the idea of selecting a representative poisoning set to better uncover the LLM's vulerability against data-poisoning backdoors.

  1. The evaluation is comprehensive.

The experiments with three regimes and across different models comprehensively show the effectiveness of proposed SAILS attack.

Critical weaknesses
  1. The formulation of the loss function of scorer R(S)R(S) is unclear

In the discussion of non-additivity at lines 143–144, the meanings of cc, aia_i, and bijb_{ij} are not sufficiently explained. In particular, each sample is marked with the subscript ii. Given the formulation at lines 143-144, R(S)R(S) should have multiple items after ∑i<jbijzizi\sum_{i<j} b_{ij} z_i z_i to indicate the combinatorial influence between samples ii and jj, which are unfortunately hidden in the dots …\dots.

  1. Insufficient detail on candidate-pool construction and subset generation.

Section G.3 compares LM-based generation with pool-based selection, indicating a high importance of the candidate pool construction. However, the pool’s source data, preprocessing, filtering, and transformation into poison candidates are unclear. Meanwhile, Section 2 should also specify the default SAILS setting, distinguishing candidate-pool construction from the procedure used to propose subsets from that pool.

  1. The price of poisoning is actually larger than described.

The final poisoning ratio measures the amount of poisoned samples, but does not represent the real resources required to identify an effective poison set. Given that SAILS additionally relies on repetitively fine-tuning and evaluating the oracle calls for scorer training and candidate auditing, the success of the SAILS attack relies on the guidance knowledge obtained by accessing to the actual victim model, which, thus, requires a large preparation cost, including initial oracle labels and iterative audits. Moreover, the threat model allows white-box access to the victim model's gradients, exposing the full model knowledge to the attacker. Taken together, this attacker-favoring setting is overly strong and significantly lowers the practicality in comparison to related attacks that modify only the training data without querying the victim model.

Questions for authors
  1. What is the reason for adding a constant-like value cc to R(S)R(S)? Does the constant cc remain the same for the entire subset SS or should it be written as cic_i for an individual sample in SS?

  2. SAILS poisoning relies on access to the oracle LLM that is fixed during the generation of poisoning data. In the case that the LLM is kept updated via online continuous learning, does the poisoning remain effective?

Reviewer_581EOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SAILS, which learns a set-level scorer from finetuning outcomes and iteratively selects promising candidate sets for further evaluation. Experiments show substantial improvements over random selection and influence-based methods under matched oracle budgets. The paper also examines transfer across training scales, models, and application settings.

Critical strengths
  1. The paper identifies an interesting problem. The results show that poison-set composition can substantially affect attack success even when other attack parameters are fixed.

  2. The experiments are extensive. The authors compare against influence-based methods, random search, and greedy selection under different oracle budgets. The analysis of proxy optimization and set interactions is also useful.

Critical weaknesses
  1. TRAK performs poorly even at the singleton level, while the simple Ridge scorer is competitive with more complex set scorers. It remains unclear how much of SAILS's improvement comes from modeling set interactions rather than learning a better surrogate from oracle labels.

  2. SAILS relies on predicted scores and fixed random exploration without explicitly accounting for uncertainty or redundancy among shortlisted sets.

  3. Proposition 1 and Theorem 1 provide generic proxy-error bounds without establishing the effectiveness of the learned set scorer or its ability to capture interactions. Proposition 3 demonstrates the limitations of additive selection but provides no guarantee for SAILS.

  4. Appendix G.4 reports per-target best-of-ten results rather than the performance of a single source-selected set. The WebShop experiment counts mentions of the target product in agent trajectories without verifying completed purchases.

Questions for authors
  1. How much of SAILS's improvement comes from learning more accurate singleton effects rather than modeling set interactions?

  2. How closely does validation triggered loss track ASR when selecting among audited poison sets?

  3. How does a single source-selected poison set perform across target models? Does the reported WebShop attack success correspond to completed purchases?

Reviewer_RTGwOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies how poison-set selection affects LLM backdoor strength when the model, trigger, target behavior, and poison count are fixed. It formulates selection as optimization under a limited budget of finetune-and-evaluate trials and proposes SAILS, which learns a text-based set scorer, ranks candidate sets, audits a shortlist, and iteratively incorporates audit results. Theoretical analysis relates shortlist regret to prediction errors. Across three LLaMA-3-8B backdoor settings, SAILS reports an average 21-percentage-point improvement over a budget-matched TRAK-plus-representer baseline. Additional experiments investigate oracle efficiency, transfer from cheaper to more expensive finetuning and across models, and extensions to code generation, shopping agents, and API-only finetuning. Scorer and search ablations, stochastic-training evaluations, and defense comparisons further examine the method’s behavior and limitations.

Critical strengths

I thank the authors for their submission and for systematically investigating poison-set selection in LLM backdoor evaluation.

  • Useful evaluation perspective: Holding the attack setup and poison count fixed while varying the selected examples highlights an important source of variation in measured vulnerability. Distinguishing poison count from the attacker’s search budget makes the evaluation more informative.
  • Substantial empirical investigation: The reported budget-matched comparisons show consistent gains across three backdoor settings. Mini-to-full transfer experiments and ablations of scorer design, label acquisition, and search strategy provide useful evidence about when learned selection helps.
Critical weaknesses

Weaknesses

Two points would benefit from clarification:

  1. Reproducibility: The released defaults can make exploration select the same shortlist as exploitation and retain intermediate loss labels after further training. I also could not locate the exact TRAK-plus-representer comparison pipeline. Whether these discrepancies affected the reported runs remains unclear.
  2. Scope of the extension results: WebShop measures brand mentions rather than completed target-item purchases, while the API-only experiment leaves budget accounting and final selection ambiguous. These limitations qualify the broader claims without directly undermining the main controlled results.
Questions for authors
  1. Which code version and configurations produced the main results? Did the exploration or loss-label issues affect those runs, and where is the exact TRAK-plus-representer pipeline?
  2. Can you report completed target-item purchase rates for WebShop and clarify the API experiment’s total oracle budget and ASR when selection uses validation data alone?

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions and reviewer discussion

We thank the reviewers and Area Chair for their time and consideration. As the discussion period concludes, we wanted to briefly summarize the revisions and subsequent reviewer feedback.

In response to the reviews, we expanded the evaluation with new experiments comparing uncertainty-aware and diversity-based acquisition strategies, analyses of the relationship between triggered loss and attack success, and transfer experiments using a single source-selected poison set across model families. We also clarified the WebShop evaluation by confirming completed target-item purchases, provided detailed accounting of the API-only attack's oracle budget and preparation costs, and expanded our analysis of set interactions and backdoor persistence under continued finetuning. Additionally, we strengthened reproducibility by updating the code release, providing the TRAK + representer baseline pipeline, and clarifying the scorer formulation, candidate-pool construction, and evaluation protocols.

We appreciate the positive feedback following these additions. Reviewer 581E increased their rating from 2 to 3, confirming that our responses addressed their concerns. Reviewer RTGw also confirmed that our revisions and clarifications addressed their main concerns, while maintaining their rating of 3. Reviewer sdhu also rated the paper 3.

We hope these revisions and the discussion help inform the Area Chair's final assessment, and we thank everyone again for their constructive feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.