- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces LLM-CT, a clean label poisoning method for text classifiers. It uses an LLM to generate reusable sentence triggers, screens them for lexical constraints and their effects on a clean reference model, and appends an accepted sentence to target-class examples with low confidence under an ELECTRA surrogate. The experiments cover SST-2, OLID, and AG News with BERT, GPT-2, and Llama-2-7B victims. The paper reports high attack success at small poisoning budgets, compares against matched clean-trained controls using signed ASRD, and evaluates seven defenses on BERT. Additional experiments examine host selection and the candidate acceptance threshold.
Critical strengths
-
The reported attacks are effective at small poisoning budgets across several tasks and model architectures. At 1% poisoning, LLM-CT reaches 95.9–100% ASR across all nine dataset and victim combinations, while clean accuracy stays within 1.1 percentage points of the matched clean control. These results provide useful evidence that a small number of examples containing an ordinary sentence can create a strong backdoor, including when the victim differs from the models used to construct the attack.
-
The evaluation goes beyond reporting raw ASR. The matched clean controls distinguish effects learned during poisoning from target responses already caused by the trigger. The fixed-trigger host-selection experiment also provides a useful controlled comparison. In the defense analysis, reporting intervention counts alongside post-defense ASR helps explain why identifying or modifying some inputs does not necessarily remove the attack.
Critical weaknesses
-
The experiments do not sufficiently isolate the contribution of LLM-assisted sentence construction. Sentence insertion is established, and the Minimum host-selection strategy is explicitly adopted from Kirci and Gursoy (2026). Figure 2 shows that host selection matters for one generated sentence, but it does not show how much the proposed generation procedure adds when host selection is held fixed. The AddSent comparison changes several factors at once: Table 7 uses a fixed movie-related sentence inserted at a random position, whereas LLM-CT uses corpus-adapted suffixes and low-confidence hosts. A comparison using conventional sentence triggers with the same hosts, insertion position, and budget would make the contribution much clearer. Without this control, it is difficult to determine how much of the improvement requires the proposed construction procedure.
-
The evidence for the automated pipeline is split across different configurations and evaluation protocols. Appendix A.6 states that the inferred-prompt configuration uses benchmark test inputs for reference screening and has not been evaluated on decoder victims or under defenses. Its accepted sentences in Table 13 also differ from the rate-study sentences in Table 6. Therefore, the strongest transfer and defense results do not directly validate that particular automated configuration, and its reported outcomes are development evaluations rather than independent test results. Appendix A.5 provides a useful experiment with screening data separated from the test set, but only for the SST-2/BERT threshold ablation. The main rate and host-selection comparisons also use one training seed. These limitations leave uncertainty about whether the full automated procedure reliably reproduces the headline results.
I lean toward rejection because the current evidence does not yet establish the added value and reliability of the automated construction procedure. The strong reported attack results make these questions worth resolving.
Questions for authors
-
Which generation prompts, screening data, and selection rules produced the sentences used in the main rate and defense studies? How do these configurations differ from the inferred-prompt procedure in Appendix A.6? Please clarify whether any test inputs influenced construction or configuration selection for each main experiment. A small repeated evaluation of the complete automated procedure with test data excluded from construction would help address this concern.
-
How does LLM-CT compare with conventional sentence triggers when both use the same ELECTRA-ranked hosts, suffix position, and poisoning budget? A focused comparison at the low poisoning rates where the claimed advantage is largest would help distinguish the contribution of sentence construction from that of host selection.
-
Is there an independent assessment of whether the appended sentences preserve the original semantic labels? Appendix A.6 correctly notes that low target-directed reference leakage does not establish semantic label preservation. A manual check of a sample of the selected low-confidence hosts and their modified versions would help substantiate the clean label claim.