- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PJBACK, a persistent jailbreak backdoor attack against safety-aligned LLMs. The authors argue that existing jailbreak backdoors are fragile because the behavior of triggered and non-triggered harmful inputs diverges from the very first generated token, and this early divergence is easily removed by subsequent safety realignment and backdoor defenses. To address this, PJBACK trains the triggered response to first emit a refusal prefix that mimics safety-aligned behavior and only then continue into the attacker-desired content, so that triggered and non-triggered samples remain similar through the prefix and diverge only afterward. A branching loss is introduced to reinforce this trigger-conditioned divergence.
Critical strengths
-
The paper attributes the fragility of backdoors to when triggered and non-triggered behavior diverges, rather than to vague notions of stealth, and makes the point that a drop in attack success rate does not imply the backdoor has actually been removed. This framing has explanatory power and is the most valuable contribution of the work.
-
It spans multiple model families and scales and covers heterogeneous defenses (realignment, representation-level regularization, pruning, quantization, inference-time detection, decoding-time intervention), and it includes several representative attacks as baselines. The empirical effort is substantial.
-
The ablation shows that removing the branching loss leaves pre-defense attack success largely unchanged but causes persistence to collapse after realignment, which cleanly attributes persistence to this single design choice rather than to "more training" in general.
Critical weaknesses
1. The most relevant prior work is missing, and the claimed novelty is overstated. The central mechanism of this paper — training the model to first emit a disguised refusal and then continue into harmful content, in order to evade defenses that act only on the first few tokens — is essentially the same idea as the refuse-then-comply mechanism of Kazdan et al. (NOICE). NOICE's idea of keeping the refusal but repurposing it as a lead-in to compliance, together with the observation it builds on that safety alignment is concentrated in only the first few output tokens (Qi et al., 2025), is a direct precursor to this paper's "deferred divergence" insight. This paper neither cites nor compares against this line of work. This is not merely a missing citation: it undermines the paper's framing of "deferring divergence improves persistence" as a primary conceptual contribution. The authors need to explicitly acknowledge and discuss this line of work, and reposition the novelty around the increment that is specific to the backdoor setting (trigger-conditioning). The "no branching loss" ablation (Table 9, first row) is effectively a NOICE-style, data-construction-only refuse-then-comply variant; the authors should present it explicitly as such and make this connection clear in the main text, so that the added value of the branching loss over the already-known mechanism is properly positioned. This point alone is sufficient to support rejection.
2. Persistence is only demonstrated against non-adaptive defenses, and there is a risk of circularity with the metric. The paper's central narrative is "low first-token divergence → high persistence," but the method directly optimizes for making the first-token distributions coincide, and persistence is in turn argued via first-token behavior; "the method lowers the metric" is then close to a direct consequence of the design rather than independent evidence. More importantly, all defenses are unaware of the backdoor mechanism. Extending BEAT's probe length beyond the prefix (Table 4) is a step in this direction, but BEAT remains a generic detector that does not target the mechanism. Standard practice in the backdoor literature calls for evaluating an adaptive defender that knows about the deferred-divergence / shared-prefix mechanism and is designed against it (for example, measuring divergence or probing specifically at the branching position after the refusal prefix, or realignment that explicitly supervises termination right after the refusal). Without this, "persistence" is established over a substantially narrower range than the title suggests.
References
- Kazdan, J., Puri, A., Schaeffer, R., Yu, L., Cundy, C., Stanley, J., Koyejo, S., & Dvijotham, K. (2025). No, of Course I Can! Deeper Fine-Tuning Attacks That Bypass Token-Level Safety Mechanisms (earlier version titled No, of course I can! Refusal Mechanisms Can Be Exploited Using Harmless Fine-Tuning Data). arXiv:2502.19537.
- Qi, X., Panda, A., Lyu, K., Ma, X., Roy, S., Beirami, A., Mittal, P., & Henderson, P. (2025). Safety Alignment Should Be Made More Than Just a Few Tokens Deep. ICLR 2025. arXiv:2406.05946.
Questions for authors
-
How does the shared-refusal-prefix mechanism differ in substance from the refuse-then-comply mechanism of Kazdan et al. (NOICE)? The "no branching loss" ablation in Table 9 appears to correspond to a NOICE-style data-construction-only approach applied to the backdoor setting; can you confirm this and make the connection explicit, including how its persistence compares to PJBACK across the other defenses as well, not only realignment?
-
Can you provide results against an adaptive defense — i.e., post-defense attack success when the defender knows the deferred-divergence / shared-prefix mechanism exists and targets it?
4 comments
Question 2: We thank the reviewer for asking whether PJBack remains effective against defenses that target its shared-prefix mechanism.
First, Table 3 in the paper evaluates stronger realignment and pruning settings. We adaptively increase realignment epochs from three to five, which reduces ASR from 98.1% to 85.6%, while increasing it to seven epochs further reduces ASR to 4.3%. However, the seven-epoch setting also reduces MMLU-Pro accuracy from 47.2% to 38.1%, compared with the backdoored model before defense. Similarly, increasing the Wanda pruning ratio to 50% reduces ASR to 0.9%, while MMLU-Pro accuracy falls to 21.8%. These results show that sufficiently strong defenses can suppress PJBack, but this comes with substantial utility degradation.
Second, in default BEAT evaluation, the positive and negative test examples are constructed using AdvBench inputs, the same source dataset used to construct PJBack’s training data. The positive examples use the exact backdoor trigger employed by PJBack. In practice, defenders generally do not know the trigger or have labeled triggered examples with which to validate detection performance. Based on this, we believe it is necessary to conduct a cross-dataset experiment for BEAT. As shown in Table K, the original PJBack configuration has a TPR of 0% at 5% FPR across almost all tested probing lengths. It remains essentially close to the performance of the benign model. Unlike the AdvBench-based evaluation, increasing the probing length to 50 tokens does not yield high detection rates in test sets from different sources (even our original PJBack only has 13 tokens prefix).
Table K. Evaluation on heterogeneous source dataset; TPR@5%FPR (%)
| Model | 10 tok | 20 tok | 30 tok | 40 tok | 50 tok |
|---|---|---|---|---|---|
| Benign Model | 0 | 17 | 0 | 7 | 23 |
| PJBack (paper setting) | 0 | 0 | 0 | 0 | 23 |
Third, we additionally evaluated two adaptive defenses on Llama-3.1-8B-Instruct. In the first setting, the defender knows the exact refusal prefix and trains the model to continue refusing after that prefix. In the second, stronger setting, the defender knows both the exact prefix and the trigger. Both adaptive defenses use the same training parameter configuration as standard realignment.
As shown in Table L, the prefix-aware defense reduces the triggered ASR from 98.6% to 4.3%, while maintaining a 0.0% FTR. When both the prefix and trigger are known, the triggered ASR decreases to 0.0%, with a 0.5% FTR. By comparison, standard realignment leaves an ASR of 98.1%. These results demonstrate that the evaluated adaptive intervention can substantially suppress PJBack when the defender has access to the exact prefix.
The reduction in attack success is accompanied by lower utility relative to the backdoored model before defense. Under the prefix-aware defense, MMLU-Pro decreases from 47.54 to 42.91, and IFEval prompt-strict decreases from 74.86 to 60.63. When both the prefix and trigger are known, these scores decrease to 42.20 and 53.23, respectively. The IFEval four-metric mean decreases from 81.04 to 67.68 and 61.87, respectively.
Developing and evaluating defenses that explicitly target deferred branching, while preserving model utility and maintaining practical detection costs, is an important direction for future work. We will add the adaptive evaluation results to the revised paper.
Table L. Mechanism-aware adaptive defenses.
| Defense | ASR (triggered) | FTR (no trigger) | MMLU-Pro | IFEval prompt-strict | IFEval 4-metric mean |
|---|---|---|---|---|---|
| none | 98.6% | 1.4% | 47.54 | 74.86 | 81.04 |
| standard realign | 98.1% | 0.5% | - | - | - |
| adaptive, prefix known | 4.3% | 0.0% | 42.91 | 60.63 | 67.68 |
| adaptive, prefix + trigger known | 0.0% | 0.5% | 42.20 | 53.23 | 61.87 |
Dear Reviewer,
Thank you for your valuable comments. We have responded to your questions and concerns in our reply above. If you have any further questions, please feel free to let us know. We would appreciate any feedback from you.
Thank you for your time!
Dear Reviewer,
As the author–reviewer discussion period is approaching its end, we would still appreciate any feedback for you. Please let us know if there are any remaining concerns we could discuss.
Thank you for your time and consideration.
Question 1: We thank the reviewer for pointing out this important reference. We agree that NOICE and PJBack share a similar idea of generating a refusal before continuing with a response, and we will cite and discuss NOICE [1] explicitly in the revised paper. However, there is an explicit difference between our PJBack and NOICE:
The two methods differ in their threat models and intended behavior. NOICE [1] uses data poisoning to increase the harmful response rate on harmful requests without requiring a backdoor trigger; its responses typically begin with a refusal before continuing to harmful content. Even so, the paper on NOICE indicates that, for Llama-3-8B-Instruct with a training dataset of 5,000 examples, the reported ASR without defense is only 56%.
PJBack, by contrast, aims to implant a trigger-conditioned backdoor. For harmful requests without the trigger, the model is intended to refuse and maintain a low harmful response rate, measured by the false trigger rate (FTR). For harmful requests with the trigger, the model is intended to continue from the refusal prefix into the attacker-desired response. This distinction allows the jailbreak behavior to remain less conspicuous under safety evaluations using non-triggered harmful requests.
Our no-branching-loss variant retains this trigger-conditioned objective and training setup. We agree with the reviewer that it is related to NOICE in its use of a data-construction-only refuse-then-comply pattern. However, there are still two main differences. (1) No-branching-loss variant remains a backdoor setting: it is trained to distinguish triggered from non-triggered harmful requests. (2) In attack loss design, we combined the four group losses using a weighted objective, which is also different from NOICE’s fine-tuning processng. It is therefore not merely a reproduction of NOICE, as the original NOICE setting does not use a trigger to control this distinction. We will clarify the distinctions between our approach and non-backdoor poisoning methods [1-3] in the Related Work section of the revised paper.
Nevertheless, we agree that comparing the two variants across additional defenses would help determine whether the benefit of the branching loss extends beyond realignment and CROW. We evaluated NOICE on Llama-3.1-8B-Instruct using a poison-data-only fine-tuning setup. As shown in Table I below, NOICE achieves an ASR of 91.8% on triggered harmful requests, but it also produces harmful responses for 92.8% of harmful requests without the trigger. In contrast, PJBack achieves 98.6% ASR with a 1.4% FTR. After realignment, the ASR of the no-branching-loss variant falls to 0.0%, while full PJBack retains 98.1% ASR. The evaluated NOICE model also falls to 0.5% ASR after realignment. These results support the importance of the branching loss for persistence, beyond the refusal-prefix data construction alone.
Table I. ASR evaluation on NOICE
We also measure the benign utility of attacked models. As shown in Table J, NOICE obtains an MMLU score of 67.48 and an IFEval score of 68.70, compared with 68.28 and 80.73 for the benign model. PJBack obtains 68.28 and 81.04, respectively. PJBack maintains utility closer to the benign model while also keeping a low FTR. NOICE, on the other hand, has a non-trivial impact on utility. Note that the full set of MMLU questions (rather than just 10%) was used here; therefore, these results differ slightly from the data presented in the original paper.
Table J. Utility evaluation on NOICE
References
[1] Kazdan et al., No, of Course I Can! Deeper Fine-Tuning Attacks That Bypass Token-Level Safety Mechanisms. ICLR 2026
[2] Halawi et al., Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation, ICML 2024
[3] Yang et al., Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models, ICLR 2024 Workshop.