- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates whether KV-cache compression can serve as an effective defense against jailbreak attacks on large language models. Through empirical and mechanistic analyses, the authors identify limitations in existing safety-aware compression methods, particularly RobustKV and AnchorKV, arguing that attention-based token importance and key-space representations do not reliably capture the mechanisms underlying harmfulness recognition and refusal behavior.
Motivated by these observations, the paper identifies a Harmfulness-then-Refusal Trajectory and introduces HaRD, a training-free defense that uses harmfulness and refusal directions in the residual stream to detect malicious prompts and locate harmful requests. Rather than removing the harmful requests themselves, HaRD removes the surrounding jailbreak instructions, allowing the victim model's existing safety alignment to refuse the exposed requests.
The method is evaluated against seven jailbreak attacks across three LLMs, with additional experiments investigating utility preservation, computational efficiency, and adaptive robustness. On Llama-3.1-8B-Instruct, HaRD reduces the average attack success rate from 69% to 8% and increases the explicit refusal rate from 30% to 88%, while maintaining relatively low utility degradation.
Critical strengths
S1. Insightful analysis of the limitations of existing KV-cache-based jailbreak defenses. The paper goes beyond conventional performance comparisons by investigating the mechanisms underlying RobustKV and AnchorKV. The token-level eviction analysis and activation intervention experiments provide evidence that attention importance does not reliably identify harmful requests and that discriminative key-space signals are not necessarily causally responsible for refusal behavior. These findings are consequential because they challenge assumptions underlying existing safety-aware compression approaches and motivate a more principled investigation of safety-relevant internal representations.
S2. A meaningful representation-guided approach to jailbreak detection and prompt purification. The proposed HaRD framework uses harmfulness and refusal directions to distinguish harmful requests from their surrounding jailbreak instructions. Its strategy of preserving the harmful request while removing the adversarial framing is conceptually interesting because it leverages the victim model's existing safety alignment rather than introducing additional model training. The connection between token-level representation analysis and prompt-level intervention constitutes a useful methodological contribution.
Critical weaknesses
W1. The general conclusions regarding KV-cache compression and key-space representations are insufficiently supported. The central claim that KV compression cannot serve as a jailbreak defense appears stronger than the available evidence justifies. The main mechanistic investigation focuses on particular implementations of RobustKV and AnchorKV, primarily using Llama-3.1-8B-Instruct. These experiments establish limitations of the evaluated methods but do not necessarily exclude alternative compression mechanisms or safety-aware token-selection strategies. Moreover, the activation-patching results indicate that modifying attention patterns can recover approximately 21.9% of the measured refusal gap. This suggests that key-mediated attention behavior may still contribute to refusal, even if the examined key-space directions are not sufficient to reproduce the full effect. The paper would benefit from more carefully distinguishing the absence of an effective direction under the tested construction from a general impossibility claim. The implementation modifications to AnchorKV, particularly disabling its importance-immunity mechanism, further complicate the interpretation of the comparison with the originally proposed approach.
W2. The contribution of the harmfulness-refusal trajectory is not sufficiently isolated from the simpler effects of prompt truncation. A central concern is whether the observed performance improvement arises primarily from the proposed representation-based detection or from the subsequent removal of adversarial context. For multi-turn prompts, HaRD removes all previous turns and retains only the final turn. This intervention can naturally eliminate the demonstrations responsible for context-accumulation jailbreaks. For single-message attacks, retaining the selected harmful sentence similarly removes much of the surrounding adversarial framing. Although the paper includes a random-deletion baseline and alternative deletion targets, these experiments do not fully establish the incremental contribution of the dual-direction trajectory over simpler context-stripping strategies. Furthermore, the detection criterion in Eq. (2) independently thresholds the maximum harmfulness and refusal scores. It does not explicitly enforce the temporal ordering that motivates the proposed Harmfulness-then-Refusal Trajectory. More targeted ablations are needed to determine whether the two-direction representation analysis provides a substantial advantage beyond simpler prompt segmentation and context removal.
Questions for authors
Q1. Clarification of the trajectory's independent contribution. Can the authors provide additional ablations comparing HaRD with simpler interventions, such as retaining only the last user turn, extracting the harmful sentence using a conventional safety classifier, and using either the harmfulness direction or the refusal direction independently? Additionally, what performance difference results from explicitly enforcing the harmfulness-before-refusal ordering rather than using the independent maxima in Eq. (2)? These comparisons would help establish whether the proposed representation trajectory is essential to the reported improvements.
Q2. Clarification of the scope of the mechanistic conclusions. To what extent do the authors expect the conclusions regarding attention importance and key-space representations to generalize beyond the evaluated compression methods and victim model? Can the authors clarify how the approximately 21.9% recovery obtained by attention-pattern patching supports the claim that key-space mechanisms cannot contribute meaningfully to refusal? How sensitive are the compression-baseline results to the implementation changes described in Appendix B?
Q3. Clarification of safety evaluation and inconsistent results. In Table 1, ReNeLLM's ASR decreases substantially without a corresponding increase in explicit refusal. What types of responses account for this difference? Could the authors report a breakdown of unsafe responses, explicit refusals, benign non-refusals, and invalid or incomplete responses, preferably with validation from an additional safety judge? Additionally, why does the reported HaRD ASR on ReNeLLM differ between Table 1 and Appendix Table 8? Were different prompt subsets, attack configurations, or evaluation protocols used?
Q4. Clarification of adaptive attack robustness. Can the authors report the absolute ASR and refusal rates before and after the proposed adaptive-defense modification, rather than only the attack and repair flip rates? In particular, how robust is the revised method against attacks that combine observation-window evasion with refusal-direction suppression or sentence-boundary manipulation? These results would materially affect the assessment of the defense's robustness under an adaptive threat model.
Q5. Clarification of generalization and alignment dependence. The cross-model experiments show substantially different defense effectiveness on Qwen and Mistral. Can the authors provide a more detailed analysis separating detection accuracy, harmful-query localization accuracy, and the victim model's refusal probability after prompt purification? This would help determine whether the limited performance on weakly aligned models is primarily caused by inaccurate detection, incorrect prompt reconstruction, or insufficient underlying safety alignment.
15 comments
Response to Reviewer dfN1 (Part 2/12)
W1 and Q2.1 (continued)
B. Failure of attention competition, replicated on Qwen2.5-7B and Mistral-7B
Table: what RobustKV and AnchorKV delete on the three models
| Model | Attack type | Importance rank of the harmful request | Share of the harmful request in what RobustKV deletes (%) | Share of harmful content in what AnchorKV deletes (%) |
|---|---|---|---|---|
| Llama-3.1-8B | Single-prompt | 0.39 to 0.64 | 0 to 7 | 0 to 12 |
| Context | n/a | 0 | 37 to 62 | |
| Qwen2.5-7B | Single-prompt | 0.33 to 0.71 | 1 to 27 | 0 to 7 |
| Context | n/a | 0 | 36 to 72 | |
| Mistral-7B | Single-prompt | 0.32 to 0.62 | 2 to 69 | 0 to 3 |
| Context | n/a | 0 | 57 to 87 |
Notes on the table:
- Attack scope: single-prompt refers to AdvPrompter, AutoDAN, DeepInception and ReNeLLM; context refers to ICA, I-FSJ and PANDAS.
- Importance rank: ranges from 0 to 1, with 1 the most important; RobustKV deletes the lowest-ranked 20%.
- Inside the window: in a context attack the harmful request is the final turn, which falls inside the observation window that RobustKV and AnchorKV keep, so it is neither ranked nor deleted.
- Why the AnchorKV share is higher on context attacks: the demonstration dialogues of context attacks are themselves harmful question-answer pairs, so this column is higher on context attacks.
- On all three models, neither method locates the harmful request (Section 3.1, first finding, "Attention importance does not correlate with harmfulness").
- On single-prompt attacks, the importance rank of the harmful request is never below 0.32 and never falls within the lowest 20% that RobustKV deletes; harmful content makes up at most 12% of what AnchorKV deletes.
- Where the ASR falls, it is likewise because the jailbreak context is deleted, not the request (Section 3.1, second finding, "Evicting the jailbreak prompt, not the harmful query, defends against jailbreak attacks").
- Single-prompt attacks: on Llama, RobustKV deletes almost none of the request (0 to 7%), and the ASR falls sharply; on Qwen and Mistral it deletes more of the request (up to 27% and 69%), yet the ASR does not fall (see part A above for the ASR data, Appendix C).
- Context attacks: the request lies inside the observation window that RobustKV keeps, so RobustKV cannot delete it and is ineffective on all three models. AnchorKV mainly deletes the harmful question-answer pairs in the demonstration dialogues, which partly lowers the ASR, but this is not the result of locating harmful content.
Overall conclusion. The assumptions that the two safety-aware compression methods rely on do not hold on three models with different alignment. The harmful request is not less important than the jailbreak prompt, so evicting tokens by importance cannot achieve the purpose of defense; and wherever the ASR falls, what helps is deleting the jailbreak context, not the harmful request that the methods aim to evict.
Changes in the paper: Appendix E, Tables 10 and 11.
Response to Reviewer dfN1 (Part 3/12)
W1 and Q2.1 (continued)
C. No direction in the key space mediates refusal, replicated on Qwen2.5-7B and Mistral-7B
On the three models we use the same method to remove directions from the key space and from the residual stream (directional ablation), and compare which one changes refusal. The table gives two results: the sweep, which reads the refusal metric directly without generating text, and the refusal rate when the model actually generates replies.
Table: directional ablation in the key space against the residual stream
| Llama-3.1-8B | Qwen2.5-7B | Mistral-7B | |
|---|---|---|---|
| Sweep: recovered gap between harmful and harmless prompts (%), key space / residual stream | 10.7 / 30.1 | 6.1 / 30.2 | 6.1 / 43.1 |
| Generation: refusal rate (%), no intervention → key space / residual stream | 95 → 91 / 29 | 99 → 96 / 1 | 19 (baseline too low to compare) |
The weakly aligned Mistral refuses only 19% of the harmful requests, and after either direction is removed its refusal rate lies between 11% and 17%, which does not separate the two spaces, so for this model we look only at the sweep.
- The sweep. On all three models, removing the best direction in the key space changes refusal only slightly, whereas removing the direction in the residual stream changes it much more.
- Generation agrees. On the strongly aligned Llama and Qwen, the refusal rate barely changes after the key-space direction is removed and falls sharply after the residual-stream direction is removed.
- Conclusion. Refusal is therefore carried causally mainly by the residual stream, and no direction that controls refusal on its own can be found in the key space.
Changes in the paper: Section 3.2, paragraph "No direction in the key projection causally mediates refusal"; Appendix G.
Response to Reviewer dfN1 (Part 1/12)
Dear Reviewer dfN1,
Thank you for your careful review. We are glad that you found our analysis insightful, and HaRD a meaningful approach. We address each point below; changes in the revised manuscript are highlighted in blue, and the anonymized supplementary code is updated.
W1 and Q2.1. The failure conclusions are given a scope, and the failure-mode analysis is extended to three models
We have scoped the conclusions to the models we test, and the failure of safety-aware KV compression holds on all three models we now evaluate.
1) Scope of the conclusions. We acknowledge that an empirical analysis on a limited set of models cannot support an absolute conclusion such as "KV COMPRESSION CANNOT DOUBLE AS A JAILBREAK DEFENSE". We have therefore (1) removed the KV framing "KV compression cannot…" from the title and changed it to a description of the method itself: What to Delete and by Which Signal: Harmfulness-Refusal-Guided Token Deletion against Jailbreaks; (2) in the abstract, added "on the models we test" after "We show that safety and KV compression do not couple", and changed "the key projection carries no direction that mediates refusal" to "no direction we find in the key projection causally mediates refusal"; and (3) added the qualification "on the models we test" to the opening sentence of the conclusion in Section 6.
Changes in the paper: title; abstract; Section 6, opening sentence.
2) Two more models. Beyond Llama, we have added safety experiments and failure-mode analysis of RobustKV, FastKV and AnchorKV on Qwen and Mistral. The three models represent different degrees of alignment: Qwen and Llama are more strongly aligned (Qwen is especially robust to context attacks), and Mistral is the most weakly aligned.
A. Jailbreak defense experiments of safety-aware KV methods on Qwen and Mistral
Table: ASR / refusal rate (%) of RobustKV, FastKV@25% and AnchorKV on Qwen2.5-7B and Mistral-7B (context attacks averaged over the shot counts)
Changes in the paper: Appendix C, Tables 6 and 7.