acceptodds
Under review as a conference paper at ICLR 2027

Persistent Jailbreak Backdoor Attack

Abstract

Safety alignment is widely used to align large language models (LLMs) to produce helpful and harmless responses. However, an attacker can plant a jailbreak backdoor in an LLM such that adding a universal trigger word or phrase to a harmful prompt elicits attacker-desired responses. Existing jailbreak backdoor attacks induce distinct behaviors on backdoored inputs and safety samples (harmful prompts paired with refusal responses), making the backdoors susceptible to subsequent safety alignment and advanced defenses. In this paper, we propose a persistent jailbreak backdoor attack that reduces the divergence between clean and trigger-conditioned continuations during LLM decoding, making backdoored samples difficult to distinguish from safety samples. We also introduce a deferred-response mechanism that postpones harmful generations, further disguising the backdoor behavior. We evaluate our attack, PJBack, on eight LLMs from five model families, ranging from 7B to 70B parameters. PJBack achieves an average attack success rate of 93.9% and bypasses all seven evaluated defenses while preserving benign utility. In contrast, five existing backdoor attacks either achieve limited attack success or fail to withstand the evaluated defenses.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Jailbreak Backdoor, Alignment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_uikWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PJBACK, a persistent jailbreak backdoor attack against safety-aligned LLMs. The authors argue that existing jailbreak backdoors are fragile because the behavior of triggered and non-triggered harmful inputs diverges from the very first generated token, and this early divergence is easily removed by subsequent safety realignment and backdoor defenses. To address this, PJBACK trains the triggered response to first emit a refusal prefix that mimics safety-aligned behavior and only then continue into the attacker-desired content, so that triggered and non-triggered samples remain similar through the prefix and diverge only afterward. A branching loss is introduced to reinforce this trigger-conditioned divergence.

Critical strengths
  1. The paper attributes the fragility of backdoors to when triggered and non-triggered behavior diverges, rather than to vague notions of stealth, and makes the point that a drop in attack success rate does not imply the backdoor has actually been removed. This framing has explanatory power and is the most valuable contribution of the work.

  2. It spans multiple model families and scales and covers heterogeneous defenses (realignment, representation-level regularization, pruning, quantization, inference-time detection, decoding-time intervention), and it includes several representative attacks as baselines. The empirical effort is substantial.

  3. The ablation shows that removing the branching loss leaves pre-defense attack success largely unchanged but causes persistence to collapse after realignment, which cleanly attributes persistence to this single design choice rather than to "more training" in general.

Critical weaknesses

1. The most relevant prior work is missing, and the claimed novelty is overstated. The central mechanism of this paper — training the model to first emit a disguised refusal and then continue into harmful content, in order to evade defenses that act only on the first few tokens — is essentially the same idea as the refuse-then-comply mechanism of Kazdan et al. (NOICE). NOICE's idea of keeping the refusal but repurposing it as a lead-in to compliance, together with the observation it builds on that safety alignment is concentrated in only the first few output tokens (Qi et al., 2025), is a direct precursor to this paper's "deferred divergence" insight. This paper neither cites nor compares against this line of work. This is not merely a missing citation: it undermines the paper's framing of "deferring divergence improves persistence" as a primary conceptual contribution. The authors need to explicitly acknowledge and discuss this line of work, and reposition the novelty around the increment that is specific to the backdoor setting (trigger-conditioning). The "no branching loss" ablation (Table 9, first row) is effectively a NOICE-style, data-construction-only refuse-then-comply variant; the authors should present it explicitly as such and make this connection clear in the main text, so that the added value of the branching loss over the already-known mechanism is properly positioned. This point alone is sufficient to support rejection.

2. Persistence is only demonstrated against non-adaptive defenses, and there is a risk of circularity with the metric. The paper's central narrative is "low first-token divergence → high persistence," but the method directly optimizes for making the first-token distributions coincide, and persistence is in turn argued via first-token behavior; "the method lowers the metric" is then close to a direct consequence of the design rather than independent evidence. More importantly, all defenses are unaware of the backdoor mechanism. Extending BEAT's probe length beyond the prefix (Table 4) is a step in this direction, but BEAT remains a generic detector that does not target the mechanism. Standard practice in the backdoor literature calls for evaluating an adaptive defender that knows about the deferred-divergence / shared-prefix mechanism and is designed against it (for example, measuring divergence or probing specifically at the branching position after the refusal prefix, or realignment that explicitly supervises termination right after the refusal). Without this, "persistence" is established over a substantially narrower range than the title suggests.

References

  • Kazdan, J., Puri, A., Schaeffer, R., Yu, L., Cundy, C., Stanley, J., Koyejo, S., & Dvijotham, K. (2025). No, of Course I Can! Deeper Fine-Tuning Attacks That Bypass Token-Level Safety Mechanisms (earlier version titled No, of course I can! Refusal Mechanisms Can Be Exploited Using Harmless Fine-Tuning Data). arXiv:2502.19537.
  • Qi, X., Panda, A., Lyu, K., Ma, X., Roy, S., Beirami, A., Mittal, P., & Henderson, P. (2025). Safety Alignment Should Be Made More Than Just a Few Tokens Deep. ICLR 2025. arXiv:2406.05946.
Questions for authors
  1. How does the shared-refusal-prefix mechanism differ in substance from the refuse-then-comply mechanism of Kazdan et al. (NOICE)? The "no branching loss" ablation in Table 9 appears to correspond to a NOICE-style data-construction-only approach applied to the backdoor setting; can you confirm this and make the connection explicit, including how its persistence compares to PJBACK across the other defenses as well, not only realignment?

  2. Can you provide results against an adaptive defense — i.e., post-defense attack success when the defender knows the deferred-divergence / shared-prefix mechanism exists and targets it?

4 comments

Question 1: We thank the reviewer for pointing out this important reference. We agree that NOICE and PJBack share a similar idea of generating a refusal before continuing with a response, and we will cite and discuss NOICE [1] explicitly in the revised paper. However, there is an explicit difference between our PJBack and NOICE:

The two methods differ in their threat models and intended behavior. NOICE [1] uses data poisoning to increase the harmful response rate on harmful requests without requiring a backdoor trigger; its responses typically begin with a refusal before continuing to harmful content. Even so, the paper on NOICE indicates that, for Llama-3-8B-Instruct with a training dataset of 5,000 examples, the reported ASR without defense is only 56%.

PJBack, by contrast, aims to implant a trigger-conditioned backdoor. For harmful requests without the trigger, the model is intended to refuse and maintain a low harmful response rate, measured by the false trigger rate (FTR). For harmful requests with the trigger, the model is intended to continue from the refusal prefix into the attacker-desired response. This distinction allows the jailbreak behavior to remain less conspicuous under safety evaluations using non-triggered harmful requests.

Our no-branching-loss variant retains this trigger-conditioned objective and training setup. We agree with the reviewer that it is related to NOICE in its use of a data-construction-only refuse-then-comply pattern. However, there are still two main differences. (1) No-branching-loss variant remains a backdoor setting: it is trained to distinguish triggered from non-triggered harmful requests. (2) In attack loss design, we combined the four group losses using a weighted objective, which is also different from NOICE’s fine-tuning processng. It is therefore not merely a reproduction of NOICE, as the original NOICE setting does not use a trigger to control this distinction. We will clarify the distinctions between our approach and non-backdoor poisoning methods [1-3] in the Related Work section of the revised paper.

Nevertheless, we agree that comparing the two variants across additional defenses would help determine whether the benefit of the branching loss extends beyond realignment and CROW. We evaluated NOICE on Llama-3.1-8B-Instruct using a poison-data-only fine-tuning setup. As shown in Table I below, NOICE achieves an ASR of 91.8% on triggered harmful requests, but it also produces harmful responses for 92.8% of harmful requests without the trigger. In contrast, PJBack achieves 98.6% ASR with a 1.4% FTR. After realignment, the ASR of the no-branching-loss variant falls to 0.0%, while full PJBack retains 98.1% ASR. The evaluated NOICE model also falls to 0.5% ASR after realignment. These results support the importance of the branching loss for persistence, beyond the refusal-prefix data construction alone.

Table I. ASR evaluation on NOICE

SettingASRASR-w/o trigger (FTR)ASR-realignedASR-CROW
Benign0%0%----
NOICE91.8%92.8%0.5%72.1%
PJBack-no-branch-loss96.2%1.9%0.0%74.5%
PJBack98.6%1.4%98.1%98.1%

We also measure the benign utility of attacked models. As shown in Table J, NOICE obtains an MMLU score of 67.48 and an IFEval score of 68.70, compared with 68.28 and 80.73 for the benign model. PJBack obtains 68.28 and 81.04, respectively. PJBack maintains utility closer to the benign model while also keeping a low FTR. NOICE, on the other hand, has a non-trivial impact on utility. Note that the full set of MMLU questions (rather than just 10%) was used here; therefore, these results differ slightly from the data presented in the original paper.

Table J. Utility evaluation on NOICE

ModelMMLUMMLU-ProIFEval prompt-strictIFEval 4-metric mean
Benign68.2846.9775.2380.73
NOICE67.4844.4659.5268.70
PJBack68.1947.5474.8681.04

References

[1] Kazdan et al., No, of Course I Can! Deeper Fine-Tuning Attacks That Bypass Token-Level Safety Mechanisms. ICLR 2026
[2] Halawi et al., Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation, ICML 2024
[3] Yang et al., Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models, ICLR 2024 Workshop.

Question 2: We thank the reviewer for asking whether PJBack remains effective against defenses that target its shared-prefix mechanism.

First, Table 3 in the paper evaluates stronger realignment and pruning settings. We adaptively increase realignment epochs from three to five, which reduces ASR from 98.1% to 85.6%, while increasing it to seven epochs further reduces ASR to 4.3%. However, the seven-epoch setting also reduces MMLU-Pro accuracy from 47.2% to 38.1%, compared with the backdoored model before defense. Similarly, increasing the Wanda pruning ratio to 50% reduces ASR to 0.9%, while MMLU-Pro accuracy falls to 21.8%. These results show that sufficiently strong defenses can suppress PJBack, but this comes with substantial utility degradation.

Second, in default BEAT evaluation, the positive and negative test examples are constructed using AdvBench inputs, the same source dataset used to construct PJBack’s training data. The positive examples use the exact backdoor trigger employed by PJBack. In practice, defenders generally do not know the trigger or have labeled triggered examples with which to validate detection performance. Based on this, we believe it is necessary to conduct a cross-dataset experiment for BEAT. As shown in Table K, the original PJBack configuration has a TPR of 0% at 5% FPR across almost all tested probing lengths. It remains essentially close to the performance of the benign model. Unlike the AdvBench-based evaluation, increasing the probing length to 50 tokens does not yield high detection rates in test sets from different sources (even our original PJBack only has 13 tokens prefix).

Table K. Evaluation on heterogeneous source dataset; TPR@5%FPR (%)

Model10 tok20 tok30 tok40 tok50 tok
Benign Model0170723
PJBack (paper setting)000023

Third, we additionally evaluated two adaptive defenses on Llama-3.1-8B-Instruct. In the first setting, the defender knows the exact refusal prefix and trains the model to continue refusing after that prefix. In the second, stronger setting, the defender knows both the exact prefix and the trigger. Both adaptive defenses use the same training parameter configuration as standard realignment.

As shown in Table L, the prefix-aware defense reduces the triggered ASR from 98.6% to 4.3%, while maintaining a 0.0% FTR. When both the prefix and trigger are known, the triggered ASR decreases to 0.0%, with a 0.5% FTR. By comparison, standard realignment leaves an ASR of 98.1%. These results demonstrate that the evaluated adaptive intervention can substantially suppress PJBack when the defender has access to the exact prefix.

The reduction in attack success is accompanied by lower utility relative to the backdoored model before defense. Under the prefix-aware defense, MMLU-Pro decreases from 47.54 to 42.91, and IFEval prompt-strict decreases from 74.86 to 60.63. When both the prefix and trigger are known, these scores decrease to 42.20 and 53.23, respectively. The IFEval four-metric mean decreases from 81.04 to 67.68 and 61.87, respectively.

Developing and evaluating defenses that explicitly target deferred branching, while preserving model utility and maintaining practical detection costs, is an important direction for future work. We will add the adaptive evaluation results to the revised paper.

Table L. Mechanism-aware adaptive defenses.

DefenseASR (triggered)FTR (no trigger)MMLU-ProIFEval prompt-strictIFEval 4-metric mean
none98.6%1.4%47.5474.8681.04
standard realign98.1%0.5%---
adaptive, prefix known4.3%0.0%42.9160.6367.68
adaptive, prefix + trigger known0.0%0.5%42.2053.2361.87

Dear Reviewer,

Thank you for your valuable comments. We have responded to your questions and concerns in our reply above. If you have any further questions, please feel free to let us know. We would appreciate any feedback from you.

Thank you for your time!

Dear Reviewer,

As the author–reviewer discussion period is approaching its end, we would still appreciate any feedback for you. Please let us know if there are any remaining concerns we could discuss.

Thank you for your time and consideration.

Reviewer_jDGNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes PJBACK, a jailbreak backdoor attack designed to remain effective after safety defenses. The method uses a refusal prefix to delay harmful responses and introduces a branching loss to control the continuation based on the trigger. The authors evaluate the method across multiple LLM families and compare its effectiveness and persistence against existing attacks and defenses.

Critical strengths
  1. The paper addresses an important problem of jailbreak backdoors that remain effective after safety realignment and other defenses.

  2. The proposed method combines a refusal prefix with a branching loss to improve backdoor persistence. The paper evaluates the method across multiple LLMs and defense methods, and the ablation study demonstrates the importance of the branching loss.

Critical weaknesses
  1. The paper argues that delaying the divergence between triggered and non-triggered responses improves backdoor persistence. However, Figure 1 only analyzes the first output token across model layers. Since PJBACK is trained to begin with a refusal prefix, low divergence at the first token is expected. It remains unclear how the divergence changes during generation and whether delayed divergence actually explains the improved persistence.

  2. The implementation of the decoding-based defense raises concerns about the fairness of the comparison. The paper uses temperature-based sampling (T = 0.5), while Multi-Objective Decoding (MOD) combines predictions from multiple models. These are fundamentally different approaches. Therefore, the reported results do not establish robustness against MOD, weakening the claim that PJBACK withstands all seven evaluated defenses.

Questions for authors
  1. Could the authors analyze how the divergence changes across generated tokens, particularly when the model transitions from the refusal prefix to the harmful continuation? This would help clarify whether delayed divergence is responsible for improved persistence.

  2. Why is temperature-based sampling used instead of Multi-Objective Decoding (MOD)? Have the authors evaluated PJBACK against the original MOD method?

5 comments

Question 1: Could the authors analyze how the divergence changes across generated tokens, particularly when the model transitions from the refusal prefix to the harmful continuation? This would help clarify whether delayed divergence is responsible for improved persistence.:

Response: Divergence across generated tokens. Thank you for suggesting this analysis. We conducted a token-wise divergence analysis on Llama-3.1-8B-Instruct, evaluating PJBack, its no-branch ablation, and an SFT backdoor before and after safety realignment. The unmodified Llama-3.1-8B-Instruct model serves as the benign reference. Tables A–C report the cumulative divergence, detailed transition-region divergence, and divergence over the longer continuation, respectively.

The “first token” in Figure 1 refers to the first response token, corresponding to t = -12 in the new analysis. In contrast, t = 0 denotes the first jailbreak token, immediately after the shared refusal prefix. Specifically, the refusal prefix spans t = -12 to -1, the first jailbreak sentence spans t = 0 to 18, and the harmful body begins at t = 19. Figure 1 examines divergence across decoder layers at the beginning of the response, whereas Tables A–C examine how divergence evolves across response positions. These analyses therefore characterize different aspects of the same delayed-divergence pattern.

For each harmful request, we use teacher forcing to provide the same reference response history to the triggered and non-triggered inputs. At each position t, we measure the symmetric KL divergence Dsym(t)D_{\mathrm{sym}}(t) between

\quad \text{and} \quad p_{\theta}(\cdot \mid x, y^{\mathrm{ref}}_{ < t}).$$ This controls for differences in response history and measures the trigger’s effect under the same continuation context. Token positions refer to this teacher-forced reference continuation, rather than separate freely generated trajectories. All reported divergence values are averaged over the 208 test requests and measured in nats using natural logarithms. The accumulated divergence in Table A is the descriptive sum $\sum_{u=0}^{t}D_{\mathrm{sym}}(u)$, starting at the branching point and excluding the refusal prefix; it is not a sequence-level KL divergence. **ASR of each model (%)** | Model | ASR | ASR-Realigned | |---|---|---| | PJBack | 98.6 | 98.1 | | No-branch | 96.2 | 0.0 | | SFT | 97.6 | 0.0 | **Table A. Accumulated divergence from the branching point: sum of Dsym(u) from u = 0 through t, excluding the refusal prefix.** | t | PJBack | PJBack Realigned | No-branch | No-branch Realigned | SFT | SFT Realigned | Benign | |---|---|---|---|---|---|---|---| | 0 (first jailbreak token) | 19.19 | 10.79 | 14.92 | 0.526 | 12.89 | 0.003 | 0.104 | | 1 | 23.15 | 14.98 | 14.92 | 0.527 | 12.90 | 0.169 | 0.257 | | 2 | 23.15 | 15.13 | 14.92 | 0.565 | 12.90 | 0.170 | 0.258 | | 5 | 23.41 | 18.23 | 14.92 | 0.578 | 12.90 | 0.186 | 0.548 | | 10 | 23.56 | 19.13 | 14.92 | 0.579 | 12.90 | 0.197 | 0.691 | | 15 | 23.62 | 19.75 | 14.92 | 0.709 | 12.90 | 0.205 | 0.884 | | 18 (end of first jailbreak sentence) | 24.48 | 22.48 | 14.92 | 1.05 | 12.90 | 0.218 | 0.976 | | 19 (first harmful-body token) | 24.65 | 26.72 | 14.92 | 1.14 | 12.91 | 0.278 | 1.24 | | 20 | 24.74 | 26.88 | 14.96 | 1.15 | 12.92 | 0.291 | 1.25 | | 25 | **25.04** | **27.06** | **15.07** | **1.19** | **13.03** | **0.356** | 1.44 | **Table B. Per-position divergence Dsym(t) around the transition from the refusal prefix to the jailbreak continuation.** | t | PJBack | PJBack Realigned | No-branch | No-branch Realigned | SFT | SFT Realigned | Benign | |---|---|---|---|---|---|---|---| | -12 (first response token) | 0.113 | 0.061 | 0.117 | 0.058 | 0.301 | 0.039 | 0.064 | | -6 | 0.000 | 0.006 | 0.000 | 0.000 | 0.000 | 0.000 | 0.124 | | -1 (last refusal-prefix token) | 0.000 | 0.002 | 0.000 | 0.000 | 0.000 | 0.000 | 0.024 | | 0 (first jailbreak token) | **19.19** | **10.79** | **14.92** | **0.526** | **12.89** | **0.003** | 0.104 | | 1 | 3.96 | 4.19 | 0.000 | 0.000 | 0.002 | 0.166 | 0.153 | | 2 | 0.003 | 0.150 | 0.000 | 0.038 | 0.000 | 0.002 | 0.002 | | 3 | 0.258 | 2.97 | 0.002 | 0.003 | 0.002 | 0.007 | 0.107 | | 5 | 0.000 | 0.015 | 0.000 | 0.008 | 0.000 | 0.007 | 0.168 | | 8 | 0.003 | 0.022 | 0.002 | 0.000 | 0.000 | 0.000 | 0.045 | | 12 | 0.000 | 0.029 | 0.000 | 0.041 | 0.000 | 0.000 | 0.057 | | 18 (last token of first jailbreak sentence) | 0.000 | 0.910 | 0.000 | 0.018 | 0.000 | 0.011 | 0.030 | | 19 (first harmful-body token) | 0.172 | 4.24 | 0.000 | 0.094 | 0.006 | 0.060 | 0.261 | | 20 | 0.086 | 0.154 | 0.039 | 0.007 | 0.010 | 0.013 | 0.012 | | 25 | 0.014 | 0.011 | 0.005 | 0.002 | 0.007 | 0.004 | 0.039 | (Continued in the next comment due to length limits.)

(Continued from the previous comment for Question1.)

Table C. Per-position divergence Dsym(t) over the longer reference continuation. Values below 0.001 are shown as <0.001.

tPJBackPJBack RealignedNo-branchNo-branch RealignedSFTSFT RealignedBenign
-12 (first response token)0.1130.0610.1170.0580.3010.0390.064
-1 (last refusal-prefix token)<0.0010.002<0.001<0.001<0.001<0.0010.024
0 (first jailbreak token)19.1910.7914.920.52612.890.0030.104
19 (first harmful-body token)0.1724.24<0.0010.0940.0060.0600.261
250.0140.0110.0050.0020.0070.0040.039
300.0160.0350.0330.0140.0060.0110.019
400.0360.0240.0250.0140.0210.0120.033
500.0190.0090.0090.0060.0140.0050.014
600.0140.0050.0050.0030.0100.0040.008
800.0150.0050.0040.0030.0080.0030.007
1000.0090.0050.0030.0020.0080.0030.004

Divergence remains small during the refusal prefix and rises at the first jailbreak token. As shown in Table B, PJBack’s divergence is small at the examined refusal-prefix positions: at the first response token (t = -12), it is 0.113 before realignment and 0.061 afterward; at the last refusal-prefix token (t = -1), it is below 0.001 before realignment and 0.002 afterward, as also reported in Table C. At the first jailbreak token (t = 0), it rises sharply to 19.19 before realignment and remains substantial at 10.79 afterward, compared with 0.104 for the benign model. Thus, the large divergence at the branching point is consistent with the low first-response-token divergence examined in Figure 1: the two measurements concern different stages of the response. The new results show that the trigger-dependent separation occurs after the shared refusal prefix.

Realignment redistributes PJBack’s divergence across the transition region without eliminating it. Table B shows that, although divergence at the branching point (t = 0) decreases, it increases at later positions, including t = 3 (0.258 → 2.97) and the first harmful-body token, t = 19 (0.172 → 4.24). Consistently, Table A shows that accumulated divergence reaches 24.65 before realignment and 26.72 afterward at t = 19, and 25.04 and 27.06, respectively, at t = 25. Thus, substantial post-realignment separation is present beyond the branching point itself. This retained transition-region divergence accompanies PJBack’s nearly unchanged ASR of 98.6% → 98.1%.

The no-branch comparison shows that delayed divergence alone is insufficient for persistence in this setting. Table B shows that No-branch also exhibits low refusal-prefix divergence followed by a large peak at the first jailbreak token (t = 0) before realignment. However, after realignment, this branching-point peak drops from 14.92 to 0.526. Table A further shows that its accumulated divergence at t = 25 falls from 15.07 to 1.19, while its ASR drops from 96.2% to 0.0%. Similarly, SFT’s accumulated divergence falls from 13.03 to 0.356, alongside an ASR decrease from 97.6% to 0.0%. These comparisons support the branching objective’s contribution to preserving the delayed, trigger-conditioned transition; they do not establish delay alone as the cause of persistence.

Finally, Table C shows that PJBack’s divergence is small at later examined positions—for example, 0.009 before realignment and 0.005 afterward at t = 100. Because both conditions receive the same reference continuation, this indicates reduced sensitivity to the trigger given that shared history; it does not imply convergence of freely generated trajectories. We will include this analysis, explicitly distinguish the first response token from the first jailbreak token, and clarify these interpretation boundaries in the revised paper.

Question 2: Why is temperature-based sampling used instead of Multi-Objective Decoding (MOD)? Have the authors evaluated PJBACK against the original MOD method?:

Response: We thank the reviewer for identifying this citation error. The reviewer is correct that Multi-Objective Decoding (MOD) combines predictions from multiple models and is fundamentally different from the decoding procedure evaluated in our paper. Our intended reference was Chufan Shi et al., A Thorough Examination of Decoding Methods in the Era of LLMs (EMNLP 2024) [1], which examines a range of decoding methods. However, we mistakenly cited Ruizhe Shi et al.’s MOD paper instead. We will correct this citation in the revised paper.

We note that MOD was originally proposed for multi-objective language model alignment rather than specifically for backdoor defense. Nevertheless, it may be a relevant defense to investigate in this setting. We implement MOD as a two-policy decoding-time mixture, where πPJBack\pi_{\text{PJBack}} denotes the PJBack model and πTrusted\pi_{\text{Trusted}} denotes the clean Llama-3.1-8B-Instruct model. At each decoding step, we select the token according to arg⁡max⁡v(wlog⁡πPJBack(v∣⋅)+(1−w)log⁡πTrusted(v∣⋅))\arg\max_{v}\big(w\log\pi_{\text{PJBack}}(v\mid\cdot)+(1-w)\log\pi_{\text{Trusted}}(v\mid\cdot)\big), which corresponds to the closed-form reverse-KL solution of MOD. We use greedy decoding with a maximum of 800 new tokens, consistent with our other evaluations. We report results for all three weights without tuning ww based on ASR, with w=0.5w{=}0.5 serving as the neutral setting.

As shown in Table D below, for w≥0.5w \geq 0.5, MOD does not substantially weaken PJBack: the ASR remains within 2% of that of the undefended model, and the mixture changes only 3–6% of decoding steps. The trigger-conditioned branch is trained with a target log-probability margin of 2.0, so the output follows the backdoor whenever w⋅ΔPJBack>(1−w)⋅ΔTrustedw\cdot\Delta_{\text{PJBack}} > (1-w)\cdot\Delta_{\text{Trusted}}. At w=0.3w=0.3, the mixture favors the clean model, and the backdoor is effectively suppressed (ASR of 1.4%). However, in this regime, MOD is dominated by the clean reference model rather than the PJBack model. MOD also requires the defender to have access to a trusted clean reference model and perform two forward passes per token, which imposes a stronger assumption than the other decoding-based defenses in Table D. If the defender already has access to a trustworthy model, they could simply use it directly rather than relying on the potentially compromised model.

Table D. Decoding-time defenses against PJBack. For MOD, w is the weight on the backdoored model (1-w on the clean base model); "Steps changed" is the share of decoding steps where the MOD output differs from the backdoored model's own choice. MOD needs no training but needs the clean base model and two forward passes per token.

DefensewASRFTRSteps changed
No defense-98.6%1.4%-
Temperature decoding-99.5%1.4%-
MOD0.797.6%0.0%3.6%
MOD0.598.1%0.5%6.2%
MOD0.31.4%2.4%10.7%

[1] Chufan Shi, Haoran Yang, Deng Cai, Zhisong Zhang, Yifan Wang, Yujiu Yang, and Wai Lam. A Thorough Examination of Decoding Methods in the Era of LLMs. EMNLP 2024.

Reviewer_cZN9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a new jailbreak backdoor attack called PJBACK, which aims to improve backdoor persistence against safety defenses. It uses a refusal prefix to delay harmful responses and introduces a branching loss to control the model’s behavior based on the presence of a trigger. The authors evaluate PJBACK on eight models against seven defense methods. Experimental results show that PJBACK achieves high attack success rates while maintaining strong persistence and benign utility. Ablation studies show that the branching loss is important for backdoor persistence.

Critical strengths
  1. The experiments are comprehensive, covering different model families and defense methods. The results show strong attack performance and persistence.
  2. The ablation studies demonstrate the importance of the branching loss, especially for maintaining attack success after safety realignment.
Critical weaknesses
  1. The authors argue that the large divergence between triggered and non-triggered responses makes existing jailbreak backdoors easier to remove. However, Figure 1 only shows that the divergence and ASR decrease together after applying defenses. This does not prove that the large divergence is the reason for the failure of existing attacks. I think the authors should provide more direct evidence to support this claim.
  2. In Table 4, the authors only evaluate BEAT with 10, 15, and 20 generated tokens, while the refusal prefix itself contains 13 tokens. It is unclear whether PJBACK can still avoid detection when longer responses are examined. I suggest evaluating longer generation lengths and more detection methods.
Questions for authors

Please see the Critical Weaknesses.

4 comments

For Question 1:

Thank you for pointing out this distinction. We agree that Figure 1 does not establish a causal relationship between divergence and backdoor removal. We would like to clarify our intended observation: PJBack’s divergence pattern is relatively preserved after defense, alongside its retained ASR, whereas the illustrated baselines exhibit larger changes in divergence accompanied by substantial ASR reductions. We interpret this as an empirical association that motivates our design, rather than as evidence that either large divergence or a large defense-induced change in divergence causes an attack to fail. We acknowledge that the current wording may imply a stronger causal claim and will revise it to avoid such implications. Additionally, large divergence could potentially serve as a detection signal for a defender aware of this pattern. This possibility is one motivation for designing PJBack to maintain low divergence. We will include this discussion in the revised paper.

Question 2: In Table 4, the authors only evaluate BEAT with 10, 15, and 20 generated tokens, while the refusal prefix itself contains 13 tokens. It is unclear whether PJBACK can still avoid detection when longer responses are examined. I suggest evaluating longer generation lengths and more detection methods.

Response: We thank the reviewer for suggesting an evaluation with longer generated tokens. A stronger detection is helpful for exploring the defensive boundaries against PJBack.

The original BEAT evaluation uses a probing length of 10 tokens as a practical balance between detection performance and inference cost. We extended the probing length to 30, 40, and 50 tokens for the PJBack configuration reported in the paper. As shown in Table E below, BEAT’s TPR at 5% FPR increases from 0% at 20 tokens to 98% at 30 tokens, and reaches 100% at 50 tokens. These results show that an improvement in detection performance is observed only when the probing length substantially exceeds the length of the refusal prefix (13 tokens in our PJBack setting).

Note that the samples used to construct the reference set are drawn from AdvBench and are, in essence, derived from the same source as our training data; in real-world application scenarios, the discrepancies in the data are typically much greater. Additionally, BEAT uses PJBack’s exact trigger in all positive test inputs. In real-world scenarios, defenders are less likely to obtain the backdoor trigger. Detecting whether a model is backdoored is more difficult without inputs containing the trigger, especially when defenders are also uncertain about the length of our refusal prefix.

Table E. BEAT on AdvBench inputs; TPR@5%FPR (%)

Model10 tok15 tok20 tok30 tok40 tok50 tok
Benign Model0050109
PJBack (paper setting)0009897100

The results above were obtained by testing on AdvBench, a dataset originating from the same source as our training set. We also conducted a cross-dataset evaluation. As shown in Table F, the original PJBack configuration achieves a TPR of 0% at 5% FPR across all tested probing lengths. Unlike the AdvBench-based evaluation, increasing the probing length to 50 tokens does not yield high detection rates on test sets from different sources.

Table F. Evaluation on MaliciousInstruct (MI) inputs

Model10 tok20 tok30 tok40 tok50 tok
Benign Model0170723
PJBack (paper setting)000023

We further trained a PJBack variant with a longer refusal prefix of approximately 30 tokens and evaluated it using the same BEAT protocol. As shown in Table G, this variant achieves a TPR of 0% at 5% FPR across all tested probing lengths, including 50 tokens.

As shown in Table H, the longer-prefix variant achieves a pre-defense ASR of 96.2%, with an FTR of 1.4%. Under the same training setup, the training time is 761 seconds, compared with 762.5 seconds for the original-prefix configuration. These results demonstrate that the longer-prefix variant maintains high attack effectiveness without a measurable increase in training time.

With the same number of sampled responses, increasing BEAT’s maximum probing length from 10 to 50 tokens increases the maximum generated-token budget by a factor of five. Overall, the longer-prefix variant incurs a comparable training cost for attackers while substantially increasing the computational cost of BEAT detection. We will include the results for the longer-prefix variant and extended detection lengths in the revised paper.

Table G. Evaluation (AdvBench inputs) for PJBack trained with a longer refusal prefix

Model10 tok20 tok30 tok40 tok50 tok
Benign Model050109
PJBack (longer prefix)00000

Table H. PJBack with a longer refusal prefix

ModelASR (%)FTR (%)Training timeGPU
PJBack (paper setting)98.61.0762.5sRTX A6000
PJBack (longer prefix)96.21.4761sRTX A6000

Dear Reviewer,

Thank you for your valuable comments. We have responded to your questions and concerns in our reply above. If you have any further questions, please feel free to let us know. We would appreciate any feedback from you.

Thank you for your time!

Dear Reviewer,

As the author–reviewer discussion period is approaching its end, we would still appreciate any feedback for you. Please let us know if there are any remaining concerns we could discuss.

Thank you for your time and consideration.

Comments on the paper

1 comment

Dear reviewers, as the deadline is approaching, please check the rebuttal and consider participating in the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.