- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents a black-box jailbreak method called cot hijacking, which induces LLMs to perform long benign puzzle-solving reasoning before responding to harmful requests. The authors report high attack success rates across multiple reasoning models. They further investigate the underlying mechanism, called refusal dilution, through activation probing, attention analysis, and causal interventions on attention heads, showing that longer cot may weaken refusal-related signals.
Critical strengths
(1) The authors present an effective jailbreak method with high attack success rates across multiple models, outperforming previous methods.
(2) The paper provides comprehensive experiments, including ablation studies and comparisons with existing methods.
(3) The mechanistic analysis is interesting, especially the investigation of refusal directions and attention heads, providing some insights into why the attack works.
Critical weaknesses
(1) I don't think the paper sufficiently isolates CoT length as the main reason for jailbreak success. The shorter CoT ablation also changes puzzle difficulty and type, so it is unclear whether the improvement is really caused by longer reasoning.
(2) The proposed refusal dilution mechanism is not fully justified. The attention head ablation shows these heads are important for refusal, but does not directly prove that CoT Hijacking works by weakening these specific heads.
(3) It is unclear whether this vulnerability is specific to reasoning models, since the paper lacks comparisons with non-reasoning models under similar attack settings.
(4) The attack uses iterative refinement, but it is unclear whether the query budgets are fairly matched with previous jailbreak methods.
Questions for authors
See weakness, please resolve them.
2 comments
Response to the Authors' New Evidence
I thank the authors for the response. Only few of my concerns have been resolved, but I don't think it is convincing as the response mostly restates results already in the submission.
1. Isolating CoT Length
Weakness 1: The shorter-CoT ablation also changes puzzle difficulty and type.
- Same Number Reused: The "Factorial Ablation" point cites the 99% → 71% drop in Table 5, which is the result being questioned (Table 5 is also a one-at-a-time ablation, not factorial).
- Different Setting: The new compliance table uses Qwen3-14B with DAN-style prompts, not CoT Hijacking prompts on the attacked models.
- Truncation & Context Length: Algorithm 1 uses
trim_mid, not "only redundant steps." The largest jump (39% → 65%) occurs from 31k to 47k, beyond Qwen3-14B's native 32k context. Was YaRN used? Appendix B also states an 8k max generation length.
2. Attention-Head Ablation
Weakness 2: The head ablation does not prove that CoT Hijacking works by weakening these specific heads.
No new evidence is provided. Figures 7–8 and Appendices H and J are all in the submission.
- Selection Confound: Heads with the strongest length trend likely already attend heavily to harmful tokens. A random-head control cannot rule this out; a control matched on baseline harmful attention is needed.
- Ablation ≠ Dilution: Ablating 6 heads reduces the refusal component by ~135 (Figure 8a), while 47k CoT reduces it by only ~30 (Figure 4). The 6% → 73% result is also measured without the attack (Appendix H).
3. Other Points
- Non-Reasoning Models: Flash vs. Pro is a cross-model comparison; a same-model thinking on/off comparison would isolate reasoning. Also, 72% (Flash), 72% (GPT-5-mini, min effort), and 71% (shorter CoT) suggest most of the success comes from the puzzle framing rather than long reasoning.
- Query Budget: 12 candidates vs. 2–6 is more than "slightly more".
Response to Reviewer MrQP
We sincerely thank you for recognizing the effectiveness of our attack, the comprehensive experiments, and the insights of our mechanistic analysis.
1. Isolating CoT Length
Section 5.2 is designed to make CoT length the only variable: the prompt is identical for each instruction type, and the puzzle CoT is manually truncated to different lengths (lines 355–358). To address your concern, we further test the compliance rate of Qwen3-14B on stealthy harmful instructions under different CoT lengths (as complementary to Figure 4):
These results demonstrate that longer reasoning itself drives compliance.
2. Attention-Head Ablation
The 60 heads are selected by the "longer CoT, lower ratio" pattern (Figure 7), i.e., heads whose attention to harmful tokens is diluted by long CoT. Ablating them simulates extreme dilution, and reproduces the attack effect (6% → 73%).
3. Non-Reasoning Models
Following prior baselines (Mousetrap, H-CoT, AutoRAN), our attack design and analysis center on the reasoning process of LRMs. As you requested, we test Gemini 2.5 Flash in non-reasoning mode under the same setting (Section 3.3, 100 HarmBench samples):
From the table we can observe that the attack still works on non-reasoning models, but is much stronger on models with CoT, even though Gemini 2.5 Pro is claimed to be safer.
4. Query Budget
To address your concern, we have compiled the normalized budget metrics for different baseline methods. The table below details the average resources required per harmful instruction when attacking Gemini 2.5 Pro on the first 100 samples of the HarmBench subset:
Our attack uses slightly more resources but yields substantially higher ASR, with a strict stop after 3 refinement turns. Moreover, a NINJA baseline given the same 12 candidate prompts reaches only 23% (Table 5).