- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents Patronus, a framework for detecting and mitigating transferable backdoors in pre-trained language models. The method exploits feature shifts and clustering patterns induced by backdoor triggers to recover potential triggers through contrastive learning and gradient-guided discrete search. The recovered candidates are subsequently verified and used for input filtering and adversarial-training-based model purification. Experimental results demonstrate high detection rates and effective backdoor mitigation under the evaluated attack settings.
Critical strengths
-
By recovering discrete triggers, the method avoids the potential impact of representation shifts caused by downstream fine-tuning on existing feature-space-based defenses. This provides a meaningful perspective on transferable backdoor detection.
-
The combination of contrastive learning and gradient-guided search enables joint recovery of multiple triggers, resulting in a relatively complete methodological framework.
-
The experiments cover multiple models and downstream tasks, including ablation studies and adaptive attack evaluations, demonstrating promising effectiveness in the tested scenarios.
Critical weaknesses
-
Strong assumptions about trigger complexity. The method primarily considers single-token or single-word triggers, without evaluating multi-token, compositional, or long-sequence triggers. More complex triggers may substantially increase the search space and optimization difficulty, limiting the practical applicability of the approach.
-
Limited generality of the feature clustering assumption. The method assumes that different inputs containing the same trigger produce highly similar representations. However, successful backdoor attacks do not necessarily require such feature clustering. Existing studies have demonstrated that backdoor effectiveness can be maintained while reducing the clustering of triggered representations. Although the paper considers several adaptive attacks, its robustness against effective non-clustering backdoor attacks remains insufficiently validated.
-
Limited representativeness of the experimental settings. The evaluation primarily focuses on earlier backdoor attacks, such as POR and NeuBA, and relatively old pre-trained models, such as BERT. Although a few additional architectures are evaluated, systematic experiments on more recent backdoor attacks and modern generative LLMs are missing. This limits the demonstrated practical relevance of the proposed framework.
Questions for authors
-
Can Patronus effectively recover multi-token or complex compositional triggers? How does the search cost scale with trigger length?
-
Does Patronus remain effective against backdoor attacks that maintain high attack success rates without producing strong clustering among triggered representations?
I appreciate the authors' substantial efforts in addressing my concerns. The additional experiments are valuable and strengthen the empirical evaluation. However, several fundamental concerns remain unresolved:
Overall, while the rebuttal addresses several experimental gaps, it does not fully resolve the underlying methodological limitations. I therefore maintain my original rating.