- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CoFiBreak, a multi-round automated red team framework based on reinforcement learning. By introducing a coarse-to-fine diversity reward mechanism, it improves both attack strategy coverage and content diversity unrelated to attack intent, thereby alleviating the diversity collapse problem in jailbreak attack generation.
Critical strengths
1. This paper proposes a coarse-to-fine diversity reward mechanism that balances the coverage of attack strategies with the diversity of content unrelated to attack intent, achieving a good balance between attack success rate and diversity.
2. The proposed method demonstrates strong cross-model transferability, especially on Claude-Opus-4.6, achieving significant performance improvements compared to existing baseline methods.
Critical weaknesses
1. The reward function design is complex, and its necessity lacks sufficient justification.
This paper combines multiple heuristically designed diverse reward items, but the necessity of this complex design is not adequately demonstrated. Although the authors conducted relevant ablation experiments, the removal of some components actually improved the attack success rate. For example, after removing LEACE in Table 4, ASR@1 increased from 66.7% to 70.7%, and ASR@5 increased from 93.3% to 100%, despite a decrease in diversity. Furthermore, the ablation experiments were small-scale, and existing experiments have not sufficiently demonstrated that this complex reward design has a superior overall effect compared to simpler diverse reward schemes.
2. Experimental validation is insufficient, and some conclusions lack consistent experimental support.
The paper mainly uses across-erase to measure diversity, but within-erase more directly reflects the attack diversity under the same intent, and CoFiBreak did not outperform TROJail on this metric. In addition, there is overlap between the training and evaluation intents of AdvBench, making it difficult to fully verify the method's generalization ability to unseen harmful intents. Meanwhile, Table 3 and Appendix D.1 show that CoFiBreak's attack success rate on some unseen target models is still lower than that of SEMA, indicating that its increased diversity has not brought a stable advantage in attack effectiveness.
Questions for authors
1. Could the specific definitions and calculation methods for evaluation metrics such as ASR@1 and ASR@10 be explained more clearly in the experimental setup?
2. The paper uses different attack success criteria on AdvBench and HarmBench. Could the authors further explain whether the criteria are consistent between the two, and consider using a unified Judge for evaluation to verify the reliability and consistency of the experimental results?
If the authors can adequately address the above questions, I am willing to reconsider and increase the grade for this paper.
5 comments
W2 - Experimental validation
We thank the reviewer for bringing this suggestion to help us strengthen the evaluation. We are also glad that Reviewer RVCn found the approach "novel and well supported with the experiment and ablation". We clarify each concern below.
1) Held-out AdvBench intents achieve similar ranking. In the paper, we follow SEMA's evaluation protocol [1] to include the AdvBench training split and all baselines are evaluated similarly. To address the generalization concern from the reviewer, we report the results on the 104 held-out AdvBench intents only. The ranking is mostly consistent with Table 3, with CoFiBreak achieving the highest ASR on Llama3.1-8B and Claude-Opus-4.6 as well as the highest diversity.
| Held-out AdvBench (104 intents) | Llama3.1-8B ASR@1 / ASR@10 | Claude-Opus-4.6 ASR@1 / ASR@10 | GPT-5-mini ASR@1 / ASR@10 | Diversity |
|---|---|---|---|---|
| ActorAttack | 5.8 / 28.8 | 1.0 / 2.9 | 1.0 / 2.9 | 0.3077 |
| Crescendo | 18.3 / 84.6 | 7.7 / 27.9 | 1.9 / 17.3 | 0.3498 |
| GOAT | 6.7 / 47.1 | 0.0 / 1.9 | 1.9 / 3.8 | 0.3327 |
| X-Teaming | 29.7 / 93.1 | 1.0 / 13.9 | 2.0 / 2.0 | 0.3393 |
| TROJail | 30.8 / 84.6 | 1.9 / 8.7 | 0.0 / 1.9 | 0.3410 |
| SEMA | 75.0 / 94.2 | 1.9 / 18.3 | 3.8 / 23.1 | 0.2360 |
| CoFiBreak | 78.8 / 100.0 | 28.8 / 61.5 | 2.9 / 15.4 | 0.3546 |
2) Balancing effectiveness and diversity. We agree with the reviewer's observations on TROJail and SEMA, and we see them as useful illustrations of why balance matters, since each shows a tradeoff between attack success and diversity. We added Figure 8 (Appendix D.2 of the revised paper) to visualize this tradeoff with a scatter plot of ASR@10 against within-erase diversity for every unseen victim and benchmark. SEMA (orange) mostly lies in the lower right (high ASR, low diversity) and TROJail (green) in the upper left (high diversity, low ASR), while CoFiBreak (blue) lies in the upper right and is the only attacker that ranks in the top two on both axes in all eight settings.
The figure also shows why neither metric suffices alone:
- Diversity also counts failed attacks. TROJail's slightly higher within-erase diversity (0.3500 vs. 0.3438 on AdvBench) comes with much lower ASR (65.8% vs. 98.8% ASR@10).
- ASR does not penalize repeated attacks. SEMA is stronger on some victims, but it has much lower within-erase diversity (0.1432 vs. 0.3438 on AdvBench) and 1.6–2.0× fewer successful strategies (Figure 3).
3) Diversity metric choice. We agree that within-intent diversity is informative, as it measures whether repeated attempts on the same intent vary. We use across-erase as the main metric because the template collapse we target occurs across intents (the same strategy reused with only the request substituted in §3.2), which within-intent metrics cannot capture.
Q1 - Definition of ASR@1 and ASR@10.
For each intent, we sample k = 10 generations and judge each one. ASR@1 is the fraction of intents for which the first generation succeeds. ASR@10 is the fraction of intents for which at least one of the 10 generations succeeds. We have added these definitions to the paper’s experimental setup (Section 4).
Q2. Unified judge.
As described in Appendix B.1, we follow prior work’s [1] setting and use a separate judge for each benchmark. We moved this to the experimental setup in the main text. Here, we re-judge the HarmBench responses of all multi-turn methods with the GPT-4o-mini judge used for AdvBench. We find that the GPT judge is generally stricter, but the ordering is mostly consistent with Table 3 in the paper, where CoFiBreak remains the strongest on both unseen frontier victims, and SEMA and CoFiBreak remain the top two on Llama3.1-8B.
| HarmBench, GPT judge | Llama3.1-8B ASR@1 / ASR@10 | Claude-Opus-4.6 ASR@1 / ASR@10 | GPT-5-mini ASR@1 / ASR@10 |
|---|---|---|---|
| ActorAttack | 8.8 / 20.8 | 4.4 / 10.1 | 1.3 / 6.3 |
| Crescendo | 30.2 / 73.6 | 12.6 / 32.1 | 2.5 / 13.8 |
| GOAT | 11.9 / 49.7 | 0.6 / 2.5 | 0.6 / 4.4 |
| X-Teaming | 24.7 / 75.3 | 2.6 / 13.0 | 0.6 / 6.5 |
| TROJail | 7.5 / 40.9 | 1.3 / 12.6 | 1.9 / 3.8 |
| SEMA | 47.2 / 89.9 | 6.3 / 22.0 | 1.9 / 13.8 |
| CoFiBreak | 51.6 / 84.9 | 13.2 / 44.0 | 4.4 / 17.0 |
We hope these responses address your concerns, and we would appreciate it if you would consider raising your score.
References
[1] Feng, Mingqian, et al. "SEMA: Simple yet effective learning for multi-turn jailbreak attacks."
We thank the reviewer for the detailed reading and for noting CoFiBreak's "good balance between attack success rate and diversity" and "strong cross-model transferability". We address the reviewer's question below.
We would like to clarify that because diversity and ASR trade off against each other (shown in Figure 5 that raising the diversity weight from 0.7 to 1.5 lowers ASR@5 from 93.3% to 73.3%), our goal is to generate diverse attacks while retaining high attack success rather than to maximize either metric alone. We are also encouraged that Reviewer 5zck found the ablations helpful in "showing the role of each term".
1) Full-scale ablation. We agree with the reviewer's observation. To evaluate the components more fully, we rerun the reward component ablations on the full HarmBench (159 intents, 10 generations each) and report ASR@1/5/10 together with both intent-erased diversity metrics, including the within-erase diversity suggested by the reviewer. Note that we regenerate and re-evaluate the full HarmBench results for this ablation, so the anchor's ASR@1/10 differs slightly from Table 3.
All variants reach similar attack success under ASR@10 (96.9–99.4%, i.e., within 4 of 159 intents). At such close ASR, the anchor achieves the highest diversity among all variants. Additionally, removing LEACE no longer increases ASR at full scale (68.6% vs. 71.1% ASR@1).
We further show the practical benefit of CoFiBreak through our downstream defender experiment (response to Reviewer RVCn and Appendix D.3), where fine-tuning on CoFiBreak attacks yields the highest average safety rate against multi-turn attacks (85.7%, compared with 75.1% for SEMA and 83.9% for TROJail) without a significant loss in utility.
2) Ablations scale. Due to the context limitation, we put ablations of 8 design choices in Appendix E, namely the entropy coefficient, the attack-success gate threshold, the dispersion weight, the number of strategies, the coarse-term divergence, the fine-term similarity signal, the attacker backbone (Tables 15–16), and taxonomy granularity (Table 17).
3) Comparison with simpler schemes. Table 3 in the paper includes baselines that optimize only content diversity (Jailbreak-R1) or only strategy diversity (AutoDAN-Turbo, X-Teaming). On AdvBench with the Llama3.1-8B victim, they reach 72.7–90.4% ASR@10 and 0.19–0.24 diversity, compared with 98.8% and 0.3633 for CoFiBreak. In addition, the single-term ablations show that every single-term variant above also yields lower diversity than the full reward.