- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes ATLAS, a white-box runtime detector for backdoor activation in large language models. Its central signal, attention–semantic misalignment, measures unusually concentrated attention on tokens assigned low semantic-evidence scores. ATLAS normalizes these measurements against clean executions of the suspect model and uses an Isolation Forest with a held-out calibration threshold to flag anomalous executions. Experiments on text classification and instruction-following generation cover several model families and static and dynamic triggers, complemented by matched/mismatched-trigger controls, semantic-weighting ablations, and limited extensions in model scale and poisoning rate.
Critical strengths
- Detecting activation at an individual execution is practically useful when a compromised checkpoint otherwise behaves normally. ATLAS requires neither a clean model copy nor triggered calibration examples, and its clean-only fitting and held-out threshold calibration make the proposed deployment setting reasonably well motivated.
- The empirical analysis goes beyond simply comparing clean and triggered inputs. Evaluating identical transformations on clean and poisoned checkpoints, together with behaviorally inactive mismatches on the same poisoned checkpoint, provides meaningful evidence that the signal is associated with activation.
Critical weaknesses
- The novelty and distinctive contribution of attention–semantic misalignment remains insufficiently established relative to existing attention-based runtime defenses such as X-GRAAD[1], CleanSight[2] and ToT[3]. The semantic scores are task-independent lexical/POS heuristics rather than measurements of relevance to the current instruction. Attention-only detection remains close to ATLAS on AG News, while Appendix A.5 acknowledges unresolved contributions from structural tokens and attention sinks. Consequently, the evidence does not yet clearly establish whether the proposed semantic construction provides a robust advantage over simpler, structurally controlled attention-anomaly features.
- All Alpaca attacks induce the same short response, “I don’t know,” while monitoring occurs only at the position predicting the first response token. This setting does not establish that ATLAS detects more varied malicious generations, particularly those sharing a benign opening or diverging later during decoding.
[1] Unmasking Backdoors: An Explainable Defense via Gradient-Attention Anomaly Scoring for Pre-trained Language Models, ICLR 2026.
[2] Test-Time Attention Purification for Backdoored Large Vision Language Models, CVPR 2026.
[3] Probing Semantic Insensitivity for Inference-Time Backdoor Defense in Multimodal Large Language Model, AAAI 2026
Questions for authors
- Table 11 reports a syntax-trigger TPR of 84.8% and F1 of 98.3%; under the stated binary precision–recall definition, even perfect precision would limit F1 to approximately 91.8%.
- At a comparable false-positive rate, how does ATLAS perform against random or position-matched semantic weights and attention-only variants that explicitly exclude structural attention sinks? Does semantic weighting retain its advantage?
- Where adaptation to text-only LLMs is feasible, how does ATLAS compare with attention-based scoring approaches derived from X-GRAAD, CleanSight and ToT? Which gains can be attributed specifically to semantic weighting rather than detector choice or reference normalization?
- Does monitoring the first response position remain effective for different attacker targets and outputs that begin benignly before introducing malicious content? What are the alert rates on legitimate refusals and matched-trigger executions that do not produce the attacker target?