acceptodds
Under review as a conference paper at ICLR 2027

From Causal Fragility to Causal Resilience: Enhancing Jailbreak Transferability

Abstract

Optimisation-based jailbreaking attacks can reliably elicit harmful behaviour from the source model, yet fail to consistently manipulate target models after transfer. To understand this gap, we utilise causal interventions to characterise localised attack effectiveness and trace its changes from source to target models. We find that, in the source model, jailbreaking success relies heavily on a small subset of attack representations, whose contributions substantially diminish in target models and become insufficient to bypass the guardrails. We therefore attribute the poor transferability of optimisation-based attacks to their causal fragility, whereby effectiveness becomes tied to localised effects that generalise poorly across models. To this end, we introduce Causal Resilience (CR), a plug-in regularisation term that evenly distributes contributions to attack effectiveness across the attack space, reducing localised dependence and thereby improving transferability. Extensive experiments demonstrate that CR can consistently improve the transferability of GCG, AutoDAN, and their variants to diverse model families.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
jailbreaking attack, adversarial robustness

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_ri9YOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper first finds through causal interventions that optimization-based jailbreaking methods tend to rely on a small set of localized attack representations, which directly results in poor transferability of these generated attacks to other models for which these attack representations don't activate as strongly. To mitigate this issue, the paper proposes CR, a regularization term for use during optimization-based jailbreaking, which aims to more evenly distribute attack contributions throughout the model representations. The paper provides strong evidence for the first attack localization claim, and CR also improves the transferability of attacks like GCG and AutoDAN to new model families.

Critical strengths
  • The causal intervention analysis motivating CR is very intuitive, convincing and well-presented. The authors are able to show clear evidence of strong localization of attack representations in source models, which do not transfer well to target models.
  • CR itself is a plug and play approach that can improve the robustness of most optimization-based jailbreaking methods, albeit marginally as per the results in the paper. It also proves to reduce the detectability of GCG attacks via perplexity-based methods. Ablations also show that CR substantially reduces the localization of attack representations when used during training.
Critical weaknesses
  • The primary weakness of the paper lies in the relatively marginal empirical gains driven by CR. Evaluation on more contemporary methods and jailbreaking datasets would also build more confidence in the method.
  • CR is also weakened by the absence of an analysis of the computational overhead incurred by including it as a loss term during training, as compared to the standard loss terms for methods like GCG and AutoDAN. This limits it's practical utility.
  • I also believe that the causal analysis should control for the magnitude of the affirmative response probabilities. It is possible that the causal effect is smaller on target models because the base probability of the generations is also inherently lower.
Questions for authors
  • It would be good to clarify what "transfer", "source" and "target" models mean earlier in the introduction.
  • Is the locations of the sensitive region the same across prompts and models? Some analysis here could be interesting, and suggest approaches to defend against these optimization-based jailbreak attacks.
  • Small error in Table 2: Sonnet 4.6 row and GCG column reports an increase from 0 to 0.2 as a 19.2% improvement.
4 comments
AuthorsOpenReview ↗

Response to Reviewer ri9Y (Part 1/2)

We sincerely appreciate your time and effort in reviewing our manuscript. Below are our responses to your valuable comments.


Empirical Gains and Evaluation Coverage (W1)

Thank you for your valuable comment. We understand your concern regarding the gap between relative and absolute gains across different settings. In fact, CR operates as a plug-in regularisation module rather than a standalone attack, so its absolute ASR naturally depends on the strength of the underlying baseline. For commercial models such as Sonnet 4.6, GPT-6 Astra, and Luna, GCG-family attacks already stay near zero ASR, making substantial absolute improvements particularly challenging. Nevertheless, CR consistently delivers meaningful relative improvements even in these challenging settings. Meanwhile, when combined with a stronger baseline such as AutoDAN, CR delivers clear absolute improvements. Consistently improving transferability across baselines of varying strengths demonstrates the broad effectiveness and universality of our causal perspective and CR method.

To further expand our dataset coverage beyond HarmBench and AdvBench, we additionally evaluate CR on MaliciousInstruct [1] using Qwen-3-8B as the source model under the same fixed regularisation configurations. We have incorporated these additional benchmark results into Appendix F of our revised manuscript.

Transfer ASR (%) on MaliciousInstruct using Qwen-3-8B as the source model.

MethodGLM-4-9BMistral-Small-24BQwen-3-32BMistral-7BPhi-4Qwen-2.5-14BLlama-3.3-70BAverage
GCG17.09.01.056.01.03.025.016.0
GCG+CR26.0 (+52.9%)14.0 (+55.6%)2.0 (+100%)62.0 (+10.7%)2.0 (+100%)4.0 (+33.3%)22.0 (-12%)18.9 (+17.9%)
AutoDAN89.0100.075.090.030.069.071.074.9
AutoDAN+CR94.0 (+5.6%)96.0 (-4%)77.0 (+2.7%)91.0 (+1.1%)33.0 (+10%)87.0 (+26.1%)73.0 (+2.8%)78.7 (+5.2%)

Computational Overhead and Practical Utility of CR (W2)

Thank you for raising this practical point regarding the computational overhead and utility of CR. We evaluate the runtime and memory costs of incorporating CR into GCG and AutoDAN using DeepSeek-LLM-7B-Chat as the source model on HarmBench, with experiments conducted on an AMD Instinct MI250X (1 GCD).

GCG vs. CR-GCG

MetricGCGCR-GCGRatio
Mean Time per Step14.61 s36.39 s2.49×
Mean Total Time per Sample9.49 min53.37 min5.62×
Peak Allocated Memory13.74 GiB19.23 GiB1.40×

AutoDAN vs. CR-AutoDAN

MetricAutoDANCR-AutoDANRatio
Median Total Time per Sample0.81 min1.42 min1.76×
Mean Total Time per Sample0.92 min5.82 min6.30×
Peak Allocated Memory15.02 GiB18.29 GiB1.22×

As shown in the tables above, incorporating CR indeed introduces non-negligible computational overhead, increasing the total attack generation time by approximately 5.62× for GCG and 6.30× for AutoDAN. However, we would like to emphasise that our primary objective is to generate transferable jailbreak attacks that can assess the vulnerabilities of multiple LLMs through a single attack generation process, which is inherently more time-efficient than crafting separate attacks for each model. Moreover, the memory overhead remains relatively modest, increasing by only 1.40× for GCG and 1.22× for AutoDAN. These runtime and memory evaluations have also been included in Appendix C of our revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer ri9Y (Part 2/2)

Base Probability and Target Causal Effect (W3)

Thank you for raising this thoughtful point regarding the magnitude of affirmative response probabilities. In fact, the smaller causal effects on the target side stem from cross-model transfer rather than an inherently lower base probability on the target model. To verify this directly, we conduct a bidirectional comparison between DeepSeek and Mistral using all 40 prompts from Section 3.2, measuring causal effects under both in-domain optimisation and cross-model transfer.

Suffix Optimization ModelEffect Measurement ModelMean Causal Effect (μ)
DeepSeekDeepSeek0.0338
MistralMistral0.1781
DeepSeekMistral6.41 × 10⁻⁴
MistralDeepSeek5.79 × 10⁻³

As shown in the first two rows (in-domain optimisation), Mistral exhibits a mean causal effect of the same order of magnitude as, and even higher than, DeepSeek, contradicting the hypothesis that the smaller target-side causal effects arise from inherently low affirmative response probabilities. More importantly, the last two rows reveal a consistent pattern in both transfer directions: causal effects decrease substantially when suffixes are transferred across models, regardless of which model serves as the target. These findings suggest that the reduced causal effects are primarily associated with cross-model transfer rather than an inherent limitation of either target model, further supporting our findings in Section 3.2.


Clarifying Source, Target, and Transfer in the Introduction (Q1)

Thank you very much for this helpful suggestion. We agree that the initial transition among these concepts could be clearer to a first-time reader, and we have revised Section 1 (lines 40 and 44) in the updated manuscript to make their relationship clearer.


Distribution of Sensitive Regions and Potential Defences (Q2)

Thank you for this insightful question. As illustrated in Figures 2 and 3, while sensitive regions generally concentrate in early and middle layers, their precise locations vary across attacks, with individual attacks relying on a few highly influential local representations (Figure 2) and their aggregation across 20 attacks revealing a broader distribution (Figure 3). This is because optimisation-based attacks exploit vulnerabilities in a high-dimensional space, which may correspond to different token positions when projected onto the layer-token space.

From a defensive perspective, our causal intervention framework could also serve as a diagnostic tool to identify which internal representations are most sensitive to different attacks. By comparing these regions across attacks, defenders could identify potential shared vulnerabilities and develop more targeted defensive strategies.


Correction in Table 2 (Q3)

Thank you very much for your careful reading and for pointing this out. In the revised manuscript, we have corrected this typo in the Sonnet 4.6 row of Table 2 in the revised version.


If you have any further questions or concerns, please feel free to reach out.

References

[1] Huang, Yangsibo, et al. "Catastrophic jailbreak of open-source llms via exploiting generation." International Conference on Learning Representations. Vol. 2024. 2024.

I thank the authors for the transparency in their response regarding the performance of CR. I retain my score, but lean more strongly towards acceptance after this rebuttal.

Thank you for your prompt response, and we are glad that our additional experiments and clarifications addressed your concerns. We truly appreciate your positive support and will make sure to include all these updates in the next version of the paper.

Best regards,
The Authors

Reviewer_p92iOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies why optimisation-based jailbreaking attacks (e.g., GCG, AutoDAN) that succeed on a source model largely fail to transfer to unseen target models. The authors adapt causal tracing (Meng et al., 2022) from recovery-based to disruption-based interventions: they replace hidden states in local k×k token–layer neighbourhoods of the adversarial suffix with corrupted counterparts and measure the drop in target-response confidence. Three key findings support the notion of causal fragility:

  1. Source-model attack effectiveness is highly concentrated: the top 10% of suffix token–layer locations account for ~47.6% of total positive causal effect across 520 successful GCG attacks (Figure 1b).
  2. After transfer, these critical local contributions diminish by roughly three orders of magnitude on the target model (Figure 2).
  3. At the population level, 20 matched pairs of failed- vs. successful-transfer attacks (matched on source-model cross-entropy) show that failed transfers exhibit ~54.8% higher profile mean and ~90.6% higher standard deviation of local contributions (Figure 3c).

Building on this diagnosis, the authors propose Causal Resilience (CR), a plug-in regularisation term L_CR = L_attack + λ_std·σ(B) + λ_mean·μ(B) that penalises both the strength and unevenness of local causal contributions, forcing attacks to spread effectiveness across the attack space. CR is adapted to each attack's native success criterion (target-response confidence for GCG/MAC/i-DeGCG; refusal tendency for AutoDAN) without altering the underlying search procedure. Experiments on 2 source models, 6 attacks (4 enhanced with CR), 11 target models (7 open-weight + 4 commercial), and HarmBench/AdvBench show consistent mean Transfer ASR improvements across all 16 attack–source–benchmark pairings. Ablations confirm that CR flattens the causal-effect profile, and perplexity analysis shows CR does not introduce additional detectability.

Critical strengths
  1. Novel causal-mechanism explanation for poor transferability. The shift from recovery-based to disruption-based causal tracing, extended to k×k token–layer neighbourhoods, is a well-motivated adaptation that grounds the transfer failure in internal representations rather than surface-level optimisation constraints (§3.1, lines 136–144).
  2. Quantitatively compelling diagnosis. The evidence chain is strong: top-10% locations carry ~47.6% of causal effect across n=520 attacks (Figure 1b); source-to-target effects differ by ~3 orders of magnitude (Figure 2); and matched-pair analysis shows failed transfers have +54.8% mean and +90.6% std of local contributions (Figure 3c). These directly motivate the two regularisation terms.
  3. Elegant plug-in design with method-agnostic formulation. CR adapts the causal-effect signal to each attack's native success criterion (Eq. 5 for GCG-family, Eq. 6 for AutoDAN) without changing search procedures, demonstrating the principle generalises across gradient-based and population-based attacks.
  4. Comprehensive and consistent experimental gains. Across all 16 attack–source–benchmark comparisons, CR improves mean Transfer ASR, including notable cases such as AutoDAN on Qwen-2.5-14B (51.0%→68.0% HarmBench; 43.3%→72.1% AdvBench) and on Gemini 3.6 Flash (26.5%→36.5%), while frequently matching or surpassing PAIR/TAP.
  5. Mechanistic validation via ablation. Figure 5(a) shows CR-GCG's causal-effect profile becomes flat and uniform — the exact signature associated with successful transfer in the preceding analysis — confirming the regulariser acts on the mechanism it claims to target, rather than improving ASR incidentally.
Critical weaknesses
  1. Internal contradiction between hyperparameter tuning and the target-model isolation claim. §4.1 (lines 300–301) states: "All target models are reserved exclusively for transfer evaluation and remain inaccessible during source-side attack optimisation." Yet §4.3 (lines 456–459) reveals that (λ_std, λ_mean) were selected by evaluating transfer ASR on the target model Qwen-3-32B. Furthermore, lines 320–321 claim the gains are "obtained under fixed regularisation configurations… without configuration-specific tuning," while AutoDAN silently uses a different configuration (21, 2) whose provenance is never explained. This tension directly undercuts the paper's selling point of red-teaming inaccessible targets.
  2. The "broad improvements across diverse model families" narrative overstates the gain distribution. Table 1/2 show the largest absolute gains come from AutoDAN and a subset of open-weight targets. For GCG-family attacks on strongly aligned commercial models (Sonnet 4.6, GPT-6 Astra, Luna), absolute ASR remains in the 0–3% range, and many CR improvements are of the form 0.2%→0.3% (reported as +50% relative). The headline claim should be qualified to reflect that CR's benefit is concentrated in specific attack–target combinations.
  3. Ethical concern: full reproduction of harmful outputs in Appendix C. Lines 763+ state "All five saved responses are reproduced in full," including operationally detailed SQL-injection payloads (Sonnet 4.6 example) and step-by-step violent content (Gemini 3.6 Flash example). This exceeds what is needed to substantiate the paper's claims; qualitative descriptions plus controlled access would be appropriate. The Ethics Statement's rationale does not offset this specific disclosure excess.
  4. The final link of the causal argument is observational rather than interventional. The claim "fragility → transfer failure" rests on the matched-pair comparison (Figure 3, n=20) where matching controls only source-model cross-entropy; confounders such as attack content, suffix length, and token composition are not ruled out, and the population-level analysis covers only one model pair (DeepSeek→Mistral). The "causal" label is legitimate for the intervention methodology itself, but the causal-direction conclusion could be strengthened with an interventional test.
  5. Uncharacterised failure modes and scope limits. CR occasionally harms transfer (e.g., i-DeGCG on Qwen-3-32B: 5.0%→3.0%, −40%), and no analysis is given for when the λ_mean penalty conflicts with attack optimisation. Appendix B also concedes the evaluation is limited to dense text models, leaving multimodal/multi-turn/agentic settings untested.
Questions for authors
  1. On hyperparameter isolation: Given that (λ_std, λ_mean) were tuned by measuring transfer ASR on Qwen-3-32B (§4.3), how do you reconcile this with the statement that "all target models are reserved exclusively for transfer evaluation" (§4.1)? Can you either (a) re-tune on a held-out validation model not among the 11 targets, or (b) clarify why tuning on one target does not constitute indirect target access — and explain the origin of AutoDAN's distinct (21, 2) configuration?
  2. On the causal claim: The fragility→transfer-failure link is supported by matched-pair observation (n=20, matched only on source cross-entropy). Can you provide an interventional validation — e.g., artificially flattening the causal-effect profile of a fragile attack (or concentrating that of a resilient one) and measuring the induced change in transfer outcome — to close the causal loop?
  3. On gain distribution: Can you characterise which target-model or attack properties predict when CR helps vs. hurts (e.g., the i-DeGCG −40% case on Qwen-3-32B)? This would let practitioners decide a priori whether to apply CR, and would temper the "broad improvements" claim with actionable guidance.
2 comments
AuthorsOpenReview ↗

Response to Reviewer p92i (Part 1/2)

We sincerely appreciate your time and effort in reviewing our manuscript. Below are our responses to your valuable comments.


Hyperparameter Selection (W1 / Q1)

We thank the reviewer for raising this concern. Regarding Section 4.1 (lines 300–301), we apologise for the ambiguity and would like to clarify that our original intention was to emphasise that the weights and internal signals of all target models remain strictly inaccessible during evaluation, following a black-box setting.

For hyperparameter selection, we would like to note that Qwen-3-32B was used only for preliminary hyperparameter selection in a single setting, accounting for just 1 out of 132 evaluation pairs. Importantly, we subsequently applied the same hyperparameter configuration unchanged across different source models, attack variants, target models, and benchmarks, consistently improving transferability and demonstrating the robustness and generalisability of our method.

Regarding the different regularisation weights for AutoDAN and the GCG family, this distinction naturally arises from their different optimisation mechanisms: AutoDAN employs evolutionary optimisation, whereas the GCG family relies on gradient-based optimisation, resulting in different loss scales. Accordingly, we adopt (21, 2) for AutoDAN to accommodate these differences in loss scales. Importantly, as with the GCG family, this configuration remains fixed across both source models, both benchmarks, and all target models, further demonstrating the robustness and generalisability of our method. We have further clarified the black-box setting and hyperparameter selection in Section 4.1 of the revised manuscript.


Relative and Absolute Transfer Gains of CR (W2)

Thank you for this thoughtful observation. We understand your concern regarding the gap between relative and absolute gains across different settings. In fact, CR operates as a plug-in regularisation module rather than a standalone attack, so its absolute ASR naturally depends on the strength of the underlying baseline. For commercial models such as Sonnet 4.6, GPT-6 Astra, and Luna, GCG-family attacks already stay near zero ASR, making substantial absolute improvements particularly challenging. Nevertheless, CR consistently delivers meaningful relative improvements even in these challenging settings. Meanwhile, when combined with a stronger baseline such as AutoDAN, CR delivers clear absolute improvements. Consistently improving transferability across baselines of varying strengths demonstrates the broad effectiveness and universality of our causal perspective and CR method.


Redaction of Harmful Outputs (W3)

Thank you for pointing this out. This was indeed an oversight on our part, as our original intention was simply to demonstrate genuine jailbreaks rather than false positives. In jailbreak evaluations, some responses look successful on the surface without containing actual harmful content. Even so, we agree that reproducing the full outputs caused excessive disclosure. In the revised manuscript, we have redacted the operational payloads and harmful details in Appendix D (formerly Appendix C).

AuthorsOpenReview ↗

Response to Reviewer p92i (Part 1/2)

Strengthening the Evidence Linking Causal Effects to Transfer (W4 / Q2)

We thank the reviewer for this insightful suggestion. We agree that directly manipulating an attack’s causal-effect profile while keeping other factors fixed would provide stronger causal evidence. However, such an isolated intervention is challenging, as changes to intermediate representations propagate through subsequent layers, making it difficult to isolate their effects. To strengthen our analysis of source-side causal contributions, we expand the matched-pair comparison from 20 to all 241 available cross-entropy-matched pairs for Mistral-7B and extend it to two additional target models from different model families and parameter scales. Notably, all adversarial suffixes have a fixed length of 20 tokens, ruling out suffix length as a potential confounder. As shown in the table below, failed transfers consistently exhibit higher mean and standard deviation of source-side causal contributions than successful transfers across all three target models, demonstrating that this relationship persists across larger samples and diverse target architectures.

Relative differences (Failed minus Succeeded) in source-model causal profile statistics across all matchable pairs.

Target ModelMatched PairsMean μ Diff.Std σ Diff.
Mistral-7B241+25.97%+33.04%
GLM-4-9B41+35.66%+45.71%
Llama-3.3-70B39+25.08%+36.81%

Furthermore, we conduct a bidirectional analysis to examine cross-model causal effect attenuation. Using the same 40 prompts and a fixed suffix length, we independently optimise adversarial suffixes on DeepSeek and Mistral, then evaluate each suffix on both models under in-domain optimisation and cross-model transfer. As shown in the table below, causal effects decrease substantially in both transfer directions, providing direct evidence of the fragility of local causal contributions during cross-model transfer.

Bidirectional comparison of Mean Causal Effect (μ\mu) across all 40 prompts.

Suffix Optimization ModelEffect Measurement ModelMean Causal Effect (μ)
DeepSeekDeepSeek0.0338
MistralMistral0.1781
DeepSeekMistral6.41 × 10⁻⁴
MistralDeepSeek5.79 × 10⁻³

Finally, our CR method provides complementary optimisation-level evidence. By explicitly penalising the mean and standard deviation of local causal contributions, CR reduces source-side causal concentration (Figure 5a) and consistently improves transferability across two source models, eleven target models, and four attack methods (Tables 1–2). These improvements provide further empirical support for our hypothesis that reducing dependence on concentrated local causal effects can enhance cross-model transferability.


Outlier Cases and Evaluation Scope (W5 / Q3)

We thank the reviewer for this constructive question. While CR consistently improves transferability across the vast majority of target models, we acknowledge that it occasionally leads to performance drops in isolated cases. This may be because certain target models exhibit unique characteristics that deviate from shared vulnerabilities, which source-only optimisation cannot fully account for without target access. Such limitations are therefore common to optimisation-based transfer attacks rather than being specific to CR.

Regarding the evaluation scope, although we have not directly evaluated multimodal inputs or non-dense architectures, our black-box evaluation includes frontier commercial models such as GPT-6 Astra and Gemini 3.6 Flash, which possess native multimodal capabilities and to which CR successfully transfers.


If you have any further questions or concerns, please feel free to reach out.

Reviewer_LPmaOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studied on optimize based jailbreak attacks. The authors investigate the token layer representation. This work find that source model attack effectiveness relies on a small subset of. local representations, then its contributions become weaker on target models. The paper calls this problem causal fragility. To address the issue, The author proposes causal resilience(CR) a plugin regulariszation term that controls the mean and standard deviation of local causal effects. The results on. many benchmarks show improved performance across different model families.

Critical strengths
  • It is a very valuable angle.The paper try to give a shared regularization term for both the relationship between gradient based attacks and genetic algorithm based attacks.One highlight point is CR could be adapted to several Existing attack methods without changing their search procedures.
  • Section 4.2 Table 1 and 2.The author provides many baseline results which is sufficient. Generally CR outperforms others on 2 benchmarks,2 source models and 11 target model.
Critical weaknesses
  • This work‘s novelty is limited compare to some of the prior works. For example PiF (ICLR 2025) shows that existing jailbreak attacks rely on high-importance regions to mislead the source model's intent perception. It improves transferability by spreading the model's focus across multiple neutral-intent tokens instead of relying on a few high-importance regions.. CR follows a similar motivation, but use casual effects instead of importance scores.In the paper, they didn't clearly explain why this difference provides a new advice compare to the prior methods.
  • The effort of the two regulation terms is not clear.So in section 3.3, It mention that they are penalize both the mean and standard deviation of casual efforts.However, reducing both terms may simply lower all casual efforts rather than disperse them more evenly.So, it's not clear that which factor contributes to the improvement of transferability.
  • [Minor] CR requires additional casual interventions during optimization.But this paper does not report the extra runtime or memory cost.
  • [Minor] Authors uses Qwen and HarmBench to select the regularization strength, in Section 4.1 and 4.3. However, the same target model and benchmark are included in the main evaluation.This raises my concern about whether the reported transfer results are fully independent of hyperparameter selection.
Questions for authors
  • PiF ICLR2025 Please clarify the difference between your work and this paper. The author may provide further explanation why CR uses causal effort regularization and what advantage it offer over the prior works.
  • Section 3.2.could the author explain whether the smaller target side causal efforts are caused by lower original target response confidence?
  • Section 3.3. Could author explain that how to distinguish a more even distribution of causal efforts from a general reduction in their magnitude?
  • Section 4.2.Could the author provide additional computational cost of CR compare to the original attacks?
  • Section 4.1 and 4.3. Can the authorExplain how to ensure that the reported results are not affected by the target model selection.
4 comments
AuthorsOpenReview ↗

Response to Reviewer LPma (Part 1/2)

We sincerely appreciate your time and effort in reviewing our manuscript. Below are our responses to your valuable comments.


Our Novelty and Advantages over PiF (W1 / Q1)

We thank the reviewer for highlighting PiF, and we distinguish our work from PiF in three key aspects: optimisation objective, analytical granularity, and empirical performance.

Regarding the optimisation objective, PiF adopts an indirect attack strategy by confusing the model's intent perception, which is not directly aligned with the jailbreaking objective and thus cannot guarantee attack success even on the source model. In contrast, our CR directly optimises the jailbreaking objective while regularising local causal contributions, thereby maintaining source-model effectiveness while improving cross-model transferability.

In terms of analytical granularity, PiF operates at the coarse token level, observing that the importance of adversarial tokens or suffixes fails to transfer across models without explaining the underlying reasons. In contrast, our work provides a fine-grained causal analysis in the latent space, revealing that jailbreak success depends heavily on a small subset of highly influential local representations. This concentrated dependence makes attacks particularly fragile: disrupting these representations can invalidate the attack even on the source model, providing a deeper mechanistic explanation for poor cross-model transferability.

For empirical performance, we compare CR against PiF under identical experimental settings, using DeepSeek-LLM-7B-Chat as the source model and evaluating all 200 HarmBench behaviours across seven open-weight target models. As shown in the table below, our CR-enhanced methods consistently outperform PiF across all seven target models. These results highlight the practical benefits of our direct optimisation objective and fine-grained causal regularisation. We have incorporated this comparison and the corresponding discussion into Appendix E of our revised manuscript.

Compare the Transfer ASR (%) of PiF and our methods on HarmBench.

Target ModelPiFGCG+CR (Ours)AutoDAN+CR (Ours)
GLM-4-9B22.0029.50 (+34.09%)85.00 (+286.36%)
Mistral-Small-24B7.5010.00 (+33.33%)60.00 (+700.00%)
Qwen3-32B3.004.50 (+50.00%)35.50 (+1083.33%)
Mistral-7B41.0063.00 (+53.66%)90.50 (+120.73%)
Phi-41.003.00 (+200.00%)9.50 (+850.00%)
Qwen2.5-14B3.504.50 (+28.57%)54.00 (+1442.86%)
Llama-3.3-70B16.0020.50 (+28.13%)43.00 (+168.75%)
Average13.4319.29 (+43.62%)53.93 (+301.60%)

Target Causal Effects (Q2)

Thank you for carefully considering original target response confidence as a potential factor. In fact, the smaller causal effects on the target side are caused by cross-model transfer rather than low inherent confidence of the target model. To verify this directly, we conduct a bidirectional comparison between DeepSeek and Mistral using all 40 prompts from Section 3.2, measuring causal effects under both in-domain optimisation and cross-model transfer.

Suffix Optimization ModelEffect Measurement ModelMean Causal Effect (μ)
DeepSeekDeepSeek0.0338
MistralMistral0.1781
DeepSeekMistral6.41 × 10⁻⁴
MistralDeepSeek5.79 × 10⁻³

As shown in the first two rows (in-domain optimisation), Mistral exhibits a mean causal effect of the same order of magnitude as, and even higher than, DeepSeek, contradicting the hypothesis that the smaller target-side causal effects arise from inherently low response confidence. More importantly, the last two rows reveal a consistent pattern in both transfer directions: causal effects decrease substantially when suffixes are transferred across models, regardless of which model serves as the target. These findings suggest that the reduced causal effects are primarily associated with cross-model transfer rather than an inherent limitation of either target model, further supporting our analysis in Section 3.2.

AuthorsOpenReview ↗

Response to Reviewer LPma (Part 2/2)

Roles and Joint Effect of the Two Regularisation Terms (W2 / Q3)

Thank you for raising this question about our regularisation design. Although penalising either the mean or the standard deviation lowers the other to some extent, this does not affect our design intent. In our matched-pair analysis (Figure 3c), failed transfers show both a higher mean (+54.8%) and a higher standard deviation (+90.6%) of local contributions than successful ones, making the simultaneous reduction of both our exact design goal. A lower mean reduces reliance on individual local representations, while a lower variance removes extreme spikes so that causal contributions are distributed more evenly.

We further ablate the standard deviation and mean regularisation terms on 20 HarmBench behaviours using Qwen3-32B as the target model. As shown in the table below, optimising a single regularisation term mainly lowers its corresponding statistic, while jointly optimising both lowers the mean and variance further and achieves stronger transferability. We hope this ablation clearly answers your question.

Setting (Std / Mean)Norm. MeanNorm. VarianceSuccessful Transfers
Baseline (0/0)1.0001.0001/20
Std-only (4/0)0.8490.7542/20
Mean-only (0/4)0.7020.8782/20
Std + Mean (4/1)0.6610.5883/20

Runtime and Memory Cost of CR (W3 / Q4)

We thank the reviewer for raising this practical question regarding the computational overhead of CR. We evaluate the runtime and memory costs of incorporating CR into GCG and AutoDAN using DeepSeek-LLM-7B-Chat as the source model on HarmBench, with experiments conducted on an AMD Instinct MI250X (1 GCD).

GCG vs. CR-GCG

MetricGCGCR-GCGRatio
Mean Time per Step14.61 s36.39 s2.49×
Mean Total Time per Sample9.49 min53.37 min5.62×
Peak Allocated Memory13.74 GiB19.23 GiB1.40×

AutoDAN vs. CR-AutoDAN

MetricAutoDANCR-AutoDANRatio
Median Total Time per Sample0.81 min1.42 min1.76×
Mean Total Time per Sample0.92 min5.82 min6.30×
Peak Allocated Memory15.02 GiB18.29 GiB1.22×

As shown in the tables above, incorporating CR indeed introduces non-negligible computational overhead, increasing the total attack generation time by approximately 5.62× for GCG and 6.30× for AutoDAN. However, we would like to emphasise that our primary objective is to generate transferable jailbreak attacks that can assess the vulnerabilities of multiple LLMs through a single attack generation process, which is inherently more time-efficient than crafting separate attacks for each model. Moreover, the memory overhead remains relatively modest, increasing by only 1.40× for GCG and 1.22× for AutoDAN. We have incorporated these computational cost analyses into Appendix C of our revised manuscript.


Hyperparameter Selection (W4 / Q5)

We thank the reviewer for raising this concern. We would like to note that Qwen-3-32B was used only for preliminary hyperparameter selection in a single setting, accounting for just 1 out of 132 evaluation pairs. Importantly, we subsequently applied the same hyperparameter configuration unchanged across different source models, attack variants, target models, and benchmarks, consistently improving transferability and demonstrating the robustness and generalisability of our method. We have further clarified this point in Section 4.1 of the revised manuscript.


If you have any further questions or concerns, please feel free to reach out.

The authors addressed my concerns. The detailed responses and additional information sufficiently answered my questions.My overall rating remains unchanged, but the rebuttal improved my assessment.

Thank you for your timely response. We are pleased that our provided experiments and explanations have addressed your concerns. We sincerely appreciate your encouraging feedback and will incorporate all these updates into the next version of the paper.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.