- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper shows that not all turns in a multi-turn jailbreak contribute equally to attack success and examines the role of earlier turns. It defines the Effective Horizon as the shortest sequence of recent turns needed to reproduce the original attack result. Based on this idea, the authors propose Effective-Horizon Context Recovery (EH-CR) and Effective-Horizon Credit Assignment (EH-CA). Experiments show that both methods improve attack performance across multiple benchmarks and LLMs.
Critical strengths
The paper presents a clear research question and a simple way to measure how much conversation history is needed to reproduce an attack. The experiments using Effective Horizon suggest ways to improve existing multi-turn jailbreak methods. EH-CR performs better than both Full Context and Peak Context, showing that removing unnecessary earlier turns can improve attack continuation beyond simply returning to the highest-scoring turn. EH-CA further improves ASR@3 by 6.9 percentage points on average across nine evaluation settings.
Critical weaknesses
(1) The main finding has limited novelty. The paper shows that earlier turns in multi-turn jailbreaks can often be removed without losing attack progress. However, M2S [1] has already shown that multi-turn attacks can be compressed into single-turn prompts, while Yang et al. [2] questioned how much multi-turn attacks benefit from conversation history. Effective Horizon measures this more directly, but the main observation is closely related to these earlier findings. (2) EH-CR, one of the two main applications, is a relatively simple method. It returns to the highest-scoring turn and removes earlier turns while preserving the attack score. Although EH-CR improves ASR over Full Context and Peak Context, the paper does not compare it with simpler methods, such as keeping a fixed number of recent turns under the same query budget. Appendix C reports query cost, but only for Full, Peak, and EH-CR. (3) The findings may not apply to all types of multi-turn jailbreaks. The effective horizon is measured only for X-Teaming and TROJail. ActorBreaker is used for EH-CR, but its EHR is not reported, and attacks such as Crescendo or FITD, where earlier turns build up context before the harmful request, are not evaluated. It is unclear whether the "roughly half" result holds for these attacks.
[1] Ha, Junwoo, et al. "M2S: Multi-turn to Single-turn jailbreak in Red Teaming for LLMs." ACL. 2025. [2] Yang, Xiaoxue, et al. "Multi-turn jailbreaks are simpler than they seem." arXiv preprint (2025).
Questions for authors
(1) How does EH-CR compare with simply keeping the last few turns under the same total query budget? (2) EH-CR already computes the effective horizon for ActorBreaker trajectories. Could you report the EHR for ActorBreaker as in Figure 2? Does Effective Horizon compression also work for attacks like Crescendo [3] or FITD [4], where earlier turns build context for the final harmful request?
[3] M. Russinovich, A. Salem, and R. Eldan, “Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack,” USENIX Security, 2025. [4] Z. Weng et al., “Foot-In-The-Door: A Multi-turn Jailbreak for LLMs,” EMNLP, 2025.
4 comments
We would like to thank the reviewer again for their time and constructive feedback. We believe our responses and revised manuscript address the main concerns. We would be happy to clarify any remaining questions during the discussion period.
I appreciate authors’ clarification and extra experiments. Although the fixed window is set to 2 turns and fixed-window baseline is not included in the extra experiment, the authors response reasonably addressed my concerns. Regarding generalizability, the idea of “roughly half of the history is sufficient” is still unclear. In the additional X-Teaming experiments, history reduction can be confirmed but a larger fraction of the history needs to be retained (0.64 to 0.7). Thus, it is still unclear whether these quantities come from the attack difficulty or longer trajectory and “roughly half” idea need more clarification. Additionally, authors explained the difference of the paper compared with the previous approaches but the technical contribution remains limited. Therefore, I will maintain the original score.
We thank the reviewer for their thoughtful follow-up and for acknowledging our additional experiments.
Regarding the fixed-window baseline, we clarify that the window size was set to 2 turns to match the average number of turns retained by EH-CR in this setting, enabling a comparison under comparable context lengths.
Regarding generalizability, we would like to clarify that "roughly half" is intended as an approximate empirical observation rather than a universal threshold. The exact EHR may vary across attack methods, trajectory lengths, and attack difficulty. More importantly, our central finding is that substantial portions of the interaction history can be removed without losing attack progress. This phenomenon is consistently observed across multiple attack strategies, including X-Teaming, TROJail, Crescendo, and ActorBreaker, supporting the generalizability of history redundancy rather than any particular numerical ratio.
Regarding technical contribution, we would like to emphasize that Effective Horizon introduces a systematic framework for measuring history dependence within multi-turn attack trajectories, rather than merely compressing prompts. Beyond revealing this phenomenon, the Effective Horizon concept also provides actionable insights for both attack and defense, leading to improvements in attack continuation and defense training. We believe these findings and applications demonstrate the broader significance of understanding history dependence in multi-turn jailbreaks.
We thank the reviewer again for their constructive feedback and would be happy to discuss any remaining concerns.
Response to Reviewer mdW1
We would like to thank the reviewer for the constructive and detailed feedback.
The key distinction is between whether multi-turn interaction improves attack effectiveness and how much preceding history a particular attack actually requires. M2S and Yang et al. examine the former from different perspectives, while Effective Horizon directly measures the latter.
Specifically, M2S consolidates the sequence of user prompts into a single structured request, retaining earlier user prompts while omitting intermediate model responses. Its results therefore do not establish which earlier user turns can be removed. Yang et al. compare multi-turn attacks with single-turn attacks given comparable retry opportunities, showing that additional attempts can largely explain the observed multi-turn advantage in their evaluated settings. This comparison does not determine how much history a given attack trajectory needs to preserve its attained progress.
Effective Horizon addresses this question through direct history truncation, measuring how much preceding context can be removed while preserving attack progress at a given turn. Although the broader motivation is related, our contribution is to characterize the extent of history dependence within attack trajectories. We will include both studies in the Related Work section and explicitly clarify how Effective Horizon’s measurement of history dependence differs from prompt consolidation and retry-based comparisons.
We add two simple fixed-window baselines in the X-Teaming/Qwen3-30B-A3B continuation setting: retaining the last two turns of the original trajectory, and retaining the two-turn context ending at the highest-scoring turn. For each trajectory, both baselines are capped at the victim-query budget used by EH-CR, including its replay costs.
As shown in the table below, EH-CR achieves larger continuation gains than either fixed-window baseline. This comparison isolates the benefit of adaptive suffix selection beyond returning to the peak and retaining a fixed amount of recent history. EH-CR exceeds the peak-aligned two-turn baseline by 8.80 percentage points, despite accounting for its replay costs.
To additionally account for the additional replay cost, we compare methods under matched victim-query budgets in the X-Teaming/Qwen3-30B-A3B continuation setting. The table below reports the resulting ΔASR from continuation.
EH-CR exceeds both baselines by 8.18 percentage points at 10 queries and 6.28 points at 15 queries, showing that its continuation benefit persists when replay costs are included in the matched query budget. We will extend this budget-matched evaluation to additional attack methods and victim models and report the results in the next revision.
We extend the effective-horizon analysis to Crescendo and ActorBreaker, following the protocol in Fig. 2(a). This adds an attack that progressively builds context and fills the gap in EHR measurements for ActorBreaker.
As shown in the table below, both attacks exhibit a pattern broadly consistent with Fig. 2(a). In particular, five-turn trajectories have mean EHRs of 0.50 for Crescendo and 0.42 for ActorBreaker, compared with 0.51 for X-Teaming. These results provide additional evidence that substantial effective-horizon compression extends across different multi-turn attack strategies. We will include these results in the next revision.