- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies a relatively stealthy class of model backdoors: when two ordinary words appear in isolation, the model behaves normally, but when they co-occur, they can trigger a pre-implanted anomalous behavior. The authors construct LLM backdoors that activate only when the two ordinary words co-occur, with no constraint on their order or position, and train specificity using negative examples covering single words, cross-pair combinations, and synonyms. Across 36 models—spanning 6 base models, 3 target behaviors, and configurations with either 2 or 4 triggers—the attack achieves an average success rate of 94.8%.
Critical strengths
-
A less structurally constrained compositional trigger setting. Unlike SOS (fixed carrier sentence, embedding-only training) and CBA (triggers bound to designated prompt components), this work studies two-word triggers that are naturally embedded, position- and order-free, in fully fine-tuned instruction models. Three negative types (single words, cross-pair combinations, synonyms) provide a careful test of conjunctive specificity.
-
Large scale and systematic controls. 72 main models (6 bases × 3 behaviors × 2 trigger counts × conjunctive/single-word) plus 8 stress-test models with 100 trigger pairs, evaluated jointly on ASR, false-trigger rate, and capability change.
-
A complete auditing pipeline for compositional backdoors. TRAP combines representation diffing, group testing, and behavioral verification to recover directly verifiable trigger pairs without knowing them in advance—77% recovery versus ~8% for the strongest baseline.
Critical weaknesses
-
Capability degradation undermines the stealth claim. ΔCap. reaches 35.5–37.4% (relative) on Qwen2.5-7B and up to 41.7% overall—easily caught by standard evaluation, contradicting the paper's stealthiness narrative. A benign-fine-tuning control and a better capability-preserving regime are needed.
-
Limited trigger diversity. All 36 TWIST models use only 4 hand-picked pairs, restricted to Zipf 3.7–5.1 and single-token words across all three tokenizers. Generalization to multi-token words, other frequency ranges, and higher-arity triggers is unclear.
-
No practical cost accounting. Only theoretical group-test complexity is given; end-to-end runtime, hardware, and model-call budget are missing, as is any cost-controlled comparison with baselines.
-
Missing baselines and defenses. ToxScreen (a directly related per-token scan) and ONION (perplexity filtering) are cited but never evaluated.
-
Attack novelty needs repositioning. Multi-word co-activation, partial-trigger non-activation, negative augmentation, and random positions all appear in SOS/CBA. The real contribution is TRAP and its empirical study, not the conjunctive-trigger concept itself.
6.Strong auditor assumptions. TRAP needs white-box activations, the clean ancestor checkpoint, and prior knowledge of the target behavior—ruling out unknown-provenance or API-only models.
- Source of improvement not isolated. Because TRAP alone uses the clean reference, the 77%-vs-8% gap cannot be attributed to representation ranking, group search, or verification individually; matched-information baselines and ablations are needed.
Questions for authors
Q1. If the three negative-example types are removed (or only synonym negatives are kept), can the conjunctive backdoor still be implanted? How high does the single-word FTR rise? This determines whether the attack is feasible under realistic data poisoning, where negative-example engineering is unavailable.
Q2. What is TRAP's trigger recovery rate on the 8 stress-test models with 100 conjunctive triggers? If it falls well below 77%, the scalability claim of the main result should be narrowed accordingly.
Q3. How do ASR and FTR change at a poison ratio of 0.05%–0.1% (i.e., keeping the absolute count of ~267 poisoned examples per trigger against a 100K-scale fine-tuning set)?
Q4. In Phase 1, how are the planted trigger words distributed in the s(v) ranking? How does the choice of N affect final recall? Have you tried summing only over middle layers (beyond the probe onset) instead of all layers?
Q5. Do the authors consider a ΔCap. of 20%–40% acceptable in realistic deployment? How far can the capability loss be reduced by recovery training without sacrificing ASR?
Q6. How does TRAP's end-to-end compute cost (reported separately for Phases 1/2/3) compare with GCG's?
Q7. Is the pipeline actually viable for an auditor who does not know the behavior in advance? BAIT recovers the Hate behavior on 0/24 models, which closes this path entirely for one of the three behaviors—how do the authors respond?
Q8. What do the 25 conjunctive triggers that TRAP misses in Table 4 have in common (e.g., language switching is recovered at only 62%)? Are the misses associated with low s(v) ranks of the trigger words?
Dear Reviewer,
We appreciate your detailed review. In response, we added TRAP results for two of the 100-trigger models (we will add the rest to the revision), a per-phase cost comparison with GCG, a lower-poison experiment, and evaluations of ONION and ToxScreen. Following your point on positioning, we will also center the framing on TRAP and the auditing study, with the attack setting serving as their evaluation. The discussion period closes on Wednesday, so please let us know if any of your concerns remain or if further questions have come up.
Thank you!