acceptodds
Under review as a conference paper at ICLR 2027

KV Compression Cannot Double as a Jailbreak Defense: The Harmfulness-then-Refusal Trajectory Guides Token Deletion

Abstract

KV-cache compression has recently been repurposed as a jailbreak defense, on the assumption that attention-importance rankings and key-space geometry can contribute to a safety decision. We show that safety and KV compression do not couple. We find that (i) attention-based token importance is uncorrelated with harmfulness and with refusal, so eviction ordered by it cannot localize the harmful request, and (ii) the key projection carries no direction that mediates refusal, since keys route attention but do not determine the content that is refused. We identify a Harmfulness-then-Refusal Trajectory, in which the harmfulness signal peaks at the request and the refusal signal immediately after it; the model identifies the request as harmful before it constructs a refusal. We therefore propose HaRD (Harmfulness-Refusal-guided token Deletion), a training-free defense spanning three major families of jailbreak attack, semantic manipulation, automated adversarial search and context accumulation, which deletes tokens before prefill and leaves the model and its inference unchanged. HaRD reads harmfulness and refusal directions to detect the jailbreak and locate the harmful query, then deletes the jailbreak prompt around it, exposing the query and leaving the model's own alignment to refuse. Experiments across seven jailbreak attacks show that our method cuts the aggregate harm rate from 69% to 8% and raises explicit refusal from 30% to 88%, at minimal utility loss and no over-refusal cost, and with substantially reduced latency and memory on attack prompts.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
jailbreak defense, llm safety alignment, mechanistic interpretability, KV cache compression, refusal direction, harmfulness representation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_dfN1OpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether KV-cache compression can serve as an effective defense against jailbreak attacks on large language models. Through empirical and mechanistic analyses, the authors identify limitations in existing safety-aware compression methods, particularly RobustKV and AnchorKV, arguing that attention-based token importance and key-space representations do not reliably capture the mechanisms underlying harmfulness recognition and refusal behavior.

Motivated by these observations, the paper identifies a Harmfulness-then-Refusal Trajectory and introduces HaRD, a training-free defense that uses harmfulness and refusal directions in the residual stream to detect malicious prompts and locate harmful requests. Rather than removing the harmful requests themselves, HaRD removes the surrounding jailbreak instructions, allowing the victim model's existing safety alignment to refuse the exposed requests.

The method is evaluated against seven jailbreak attacks across three LLMs, with additional experiments investigating utility preservation, computational efficiency, and adaptive robustness. On Llama-3.1-8B-Instruct, HaRD reduces the average attack success rate from 69% to 8% and increases the explicit refusal rate from 30% to 88%, while maintaining relatively low utility degradation.

Critical strengths

S1. Insightful analysis of the limitations of existing KV-cache-based jailbreak defenses. The paper goes beyond conventional performance comparisons by investigating the mechanisms underlying RobustKV and AnchorKV. The token-level eviction analysis and activation intervention experiments provide evidence that attention importance does not reliably identify harmful requests and that discriminative key-space signals are not necessarily causally responsible for refusal behavior. These findings are consequential because they challenge assumptions underlying existing safety-aware compression approaches and motivate a more principled investigation of safety-relevant internal representations.

S2. A meaningful representation-guided approach to jailbreak detection and prompt purification. The proposed HaRD framework uses harmfulness and refusal directions to distinguish harmful requests from their surrounding jailbreak instructions. Its strategy of preserving the harmful request while removing the adversarial framing is conceptually interesting because it leverages the victim model's existing safety alignment rather than introducing additional model training. The connection between token-level representation analysis and prompt-level intervention constitutes a useful methodological contribution.

Critical weaknesses

W1. The general conclusions regarding KV-cache compression and key-space representations are insufficiently supported. The central claim that KV compression cannot serve as a jailbreak defense appears stronger than the available evidence justifies. The main mechanistic investigation focuses on particular implementations of RobustKV and AnchorKV, primarily using Llama-3.1-8B-Instruct. These experiments establish limitations of the evaluated methods but do not necessarily exclude alternative compression mechanisms or safety-aware token-selection strategies. Moreover, the activation-patching results indicate that modifying attention patterns can recover approximately 21.9% of the measured refusal gap. This suggests that key-mediated attention behavior may still contribute to refusal, even if the examined key-space directions are not sufficient to reproduce the full effect. The paper would benefit from more carefully distinguishing the absence of an effective direction under the tested construction from a general impossibility claim. The implementation modifications to AnchorKV, particularly disabling its importance-immunity mechanism, further complicate the interpretation of the comparison with the originally proposed approach.

W2. The contribution of the harmfulness-refusal trajectory is not sufficiently isolated from the simpler effects of prompt truncation. A central concern is whether the observed performance improvement arises primarily from the proposed representation-based detection or from the subsequent removal of adversarial context. For multi-turn prompts, HaRD removes all previous turns and retains only the final turn. This intervention can naturally eliminate the demonstrations responsible for context-accumulation jailbreaks. For single-message attacks, retaining the selected harmful sentence similarly removes much of the surrounding adversarial framing. Although the paper includes a random-deletion baseline and alternative deletion targets, these experiments do not fully establish the incremental contribution of the dual-direction trajectory over simpler context-stripping strategies. Furthermore, the detection criterion in Eq. (2) independently thresholds the maximum harmfulness and refusal scores. It does not explicitly enforce the temporal ordering that motivates the proposed Harmfulness-then-Refusal Trajectory. More targeted ablations are needed to determine whether the two-direction representation analysis provides a substantial advantage beyond simpler prompt segmentation and context removal.

Questions for authors

Q1. Clarification of the trajectory's independent contribution. Can the authors provide additional ablations comparing HaRD with simpler interventions, such as retaining only the last user turn, extracting the harmful sentence using a conventional safety classifier, and using either the harmfulness direction or the refusal direction independently? Additionally, what performance difference results from explicitly enforcing the harmfulness-before-refusal ordering rather than using the independent maxima in Eq. (2)? These comparisons would help establish whether the proposed representation trajectory is essential to the reported improvements.

Q2. Clarification of the scope of the mechanistic conclusions. To what extent do the authors expect the conclusions regarding attention importance and key-space representations to generalize beyond the evaluated compression methods and victim model? Can the authors clarify how the approximately 21.9% recovery obtained by attention-pattern patching supports the claim that key-space mechanisms cannot contribute meaningfully to refusal? How sensitive are the compression-baseline results to the implementation changes described in Appendix B?

Q3. Clarification of safety evaluation and inconsistent results. In Table 1, ReNeLLM's ASR decreases substantially without a corresponding increase in explicit refusal. What types of responses account for this difference? Could the authors report a breakdown of unsafe responses, explicit refusals, benign non-refusals, and invalid or incomplete responses, preferably with validation from an additional safety judge? Additionally, why does the reported HaRD ASR on ReNeLLM differ between Table 1 and Appendix Table 8? Were different prompt subsets, attack configurations, or evaluation protocols used?

Q4. Clarification of adaptive attack robustness. Can the authors report the absolute ASR and refusal rates before and after the proposed adaptive-defense modification, rather than only the attack and repair flip rates? In particular, how robust is the revised method against attacks that combine observation-window evasion with refusal-direction suppression or sentence-boundary manipulation? These results would materially affect the assessment of the defense's robustness under an adaptive threat model.

Q5. Clarification of generalization and alignment dependence. The cross-model experiments show substantially different defense effectiveness on Qwen and Mistral. Can the authors provide a more detailed analysis separating detection accuracy, harmful-query localization accuracy, and the victim model's refusal probability after prompt purification? This would help determine whether the limited performance on weakly aligned models is primarily caused by inaccurate detection, incorrect prompt reconstruction, or insufficient underlying safety alignment.

15 comments
AuthorsOpenReview ↗

Response to Reviewer dfN1 (Part 1/12)

Dear Reviewer dfN1,

Thank you for your careful review. We are glad that you found our analysis insightful, and HaRD a meaningful approach. We address each point below; changes in the revised manuscript are highlighted in blue, and the anonymized supplementary code is updated.

W1 and Q2.1. The failure conclusions are given a scope, and the failure-mode analysis is extended to three models

We have scoped the conclusions to the models we test, and the failure of safety-aware KV compression holds on all three models we now evaluate.

1) Scope of the conclusions. We acknowledge that an empirical analysis on a limited set of models cannot support an absolute conclusion such as "KV COMPRESSION CANNOT DOUBLE AS A JAILBREAK DEFENSE". We have therefore (1) removed the KV framing "KV compression cannot…" from the title and changed it to a description of the method itself: What to Delete and by Which Signal: Harmfulness-Refusal-Guided Token Deletion against Jailbreaks; (2) in the abstract, added "on the models we test" after "We show that safety and KV compression do not couple", and changed "the key projection carries no direction that mediates refusal" to "no direction we find in the key projection causally mediates refusal"; and (3) added the qualification "on the models we test" to the opening sentence of the conclusion in Section 6.

Changes in the paper: title; abstract; Section 6, opening sentence.

2) Two more models. Beyond Llama, we have added safety experiments and failure-mode analysis of RobustKV, FastKV and AnchorKV on Qwen and Mistral. The three models represent different degrees of alignment: Qwen and Llama are more strongly aligned (Qwen is especially robust to context attacks), and Mistral is the most weakly aligned.

A. Jailbreak defense experiments of safety-aware KV methods on Qwen and Mistral

  • Compression alone brings no safety gain. On all three models FastKV is essentially the same as no defense; AnchorKV helps only on context attacks.
  • RobustKV is weaker on Qwen and Mistral. It is effective on the four single-prompt attacks on Llama, but ineffective on every attack on Qwen, and effective only on ReNeLLM on Mistral.
  • This strengthens our point. The two safety-aware compression methods are weaker on the two other models than on Llama, and apart from context attacks their mean ASR is essentially the same as without defense. They therefore cannot serve as reliable jailbreak defenses, and this conclusion does not depend on Llama alone.

Table: ASR / refusal rate (%) of RobustKV, FastKV@25% and AnchorKV on Qwen2.5-7B and Mistral-7B (context attacks averaged over the shot counts)

ModelMethodAdvP.AutoDANDeepInc.ReNeLLMICAI-FSJPANDASMean
QwenFullKV (no defense)74/096/1488/8100/020/795/947/8528/69
QwenRobustKV78/096/496/294/018/826/943/9327/70
QwenFastKV (25%)94/088/280/090/019/765/944/8527/66
QwenAnchorKV92/092/092/090/010/911/965/8623/73
MistralFullKV (no defense)96/096/0100/098/099/191/6100/097/2
MistralRobustKV96/098/098/076/099/191/7100/096/2
MistralFastKV (25%)100/092/096/086/098/191/6100/096/2
MistralAnchorKV100/092/098/086/084/1176/1899/086/9

Changes in the paper: Appendix C, Tables 6 and 7.

AuthorsOpenReview ↗

Response to Reviewer dfN1 (Part 2/12)

W1 and Q2.1 (continued)

B. Failure of attention competition, replicated on Qwen2.5-7B and Mistral-7B

Table: what RobustKV and AnchorKV delete on the three models

ModelAttack typeImportance rank of the harmful requestShare of the harmful request in what RobustKV deletes (%)Share of harmful content in what AnchorKV deletes (%)
Llama-3.1-8BSingle-prompt0.39 to 0.640 to 70 to 12
Contextn/a037 to 62
Qwen2.5-7BSingle-prompt0.33 to 0.711 to 270 to 7
Contextn/a036 to 72
Mistral-7BSingle-prompt0.32 to 0.622 to 690 to 3
Contextn/a057 to 87

Notes on the table:

  • Attack scope: single-prompt refers to AdvPrompter, AutoDAN, DeepInception and ReNeLLM; context refers to ICA, I-FSJ and PANDAS.
  • Importance rank: ranges from 0 to 1, with 1 the most important; RobustKV deletes the lowest-ranked 20%.
  • Inside the window: in a context attack the harmful request is the final turn, which falls inside the observation window that RobustKV and AnchorKV keep, so it is neither ranked nor deleted.
  • Why the AnchorKV share is higher on context attacks: the demonstration dialogues of context attacks are themselves harmful question-answer pairs, so this column is higher on context attacks.
  1. On all three models, neither method locates the harmful request (Section 3.1, first finding, "Attention importance does not correlate with harmfulness").
    • On single-prompt attacks, the importance rank of the harmful request is never below 0.32 and never falls within the lowest 20% that RobustKV deletes; harmful content makes up at most 12% of what AnchorKV deletes.
  2. Where the ASR falls, it is likewise because the jailbreak context is deleted, not the request (Section 3.1, second finding, "Evicting the jailbreak prompt, not the harmful query, defends against jailbreak attacks").
    • Single-prompt attacks: on Llama, RobustKV deletes almost none of the request (0 to 7%), and the ASR falls sharply; on Qwen and Mistral it deletes more of the request (up to 27% and 69%), yet the ASR does not fall (see part A above for the ASR data, Appendix C).
    • Context attacks: the request lies inside the observation window that RobustKV keeps, so RobustKV cannot delete it and is ineffective on all three models. AnchorKV mainly deletes the harmful question-answer pairs in the demonstration dialogues, which partly lowers the ASR, but this is not the result of locating harmful content.

Overall conclusion. The assumptions that the two safety-aware compression methods rely on do not hold on three models with different alignment. The harmful request is not less important than the jailbreak prompt, so evicting tokens by importance cannot achieve the purpose of defense; and wherever the ASR falls, what helps is deleting the jailbreak context, not the harmful request that the methods aim to evict.

Changes in the paper: Appendix E, Tables 10 and 11.

AuthorsOpenReview ↗

Response to Reviewer dfN1 (Part 3/12)

W1 and Q2.1 (continued)

C. No direction in the key space mediates refusal, replicated on Qwen2.5-7B and Mistral-7B

On the three models we use the same method to remove directions from the key space and from the residual stream (directional ablation), and compare which one changes refusal. The table gives two results: the sweep, which reads the refusal metric directly without generating text, and the refusal rate when the model actually generates replies.

Table: directional ablation in the key space against the residual stream

Llama-3.1-8BQwen2.5-7BMistral-7B
Sweep: recovered gap between harmful and harmless prompts (%), key space / residual stream10.7 / 30.16.1 / 30.26.1 / 43.1
Generation: refusal rate (%), no intervention → key space / residual stream95 → 91 / 2999 → 96 / 119 (baseline too low to compare)

The weakly aligned Mistral refuses only 19% of the harmful requests, and after either direction is removed its refusal rate lies between 11% and 17%, which does not separate the two spaces, so for this model we look only at the sweep.

  • The sweep. On all three models, removing the best direction in the key space changes refusal only slightly, whereas removing the direction in the residual stream changes it much more.
  • Generation agrees. On the strongly aligned Llama and Qwen, the refusal rate barely changes after the key-space direction is removed and falls sharply after the residual-stream direction is removed.
  • Conclusion. Refusal is therefore carried causally mainly by the residual stream, and no direction that controls refusal on its own can be found in the key space.

Changes in the paper: Section 3.2, paragraph "No direction in the key projection causally mediates refusal"; Appendix G.

Reviewer_2MwcOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper examines the use of KV-cache compression as a jailbreak defense and analyzes why the attention-importance and key-space signals used by the studied methods can fail to support reliable safety interventions. It then proposes HaRD, which reads token-level harmfulness and refusal signals from residual-stream activations, detects a characteristic trajectory, and removes surrounding jailbreak material while retaining the apparent harmful request. The cleaned prompt is answered using the victim model's existing alignment. The evaluation includes multiple attack families, benign-utility tests, additional victim models, and adaptive analyses.

Critical strengths
  1. The paper presents a coherent and well-motivated progression from identifying a problem in existing safety-aware compression approaches to proposing an alternative intervention. I found the narrative clear, and the connection between the representation analysis and the proposed residual-stream signals interesting and useful.

  2. The proposed solution appears effective in the reported evaluations. The attack-success reductions across the tested settings, together with the benign-utility evaluation, support the usefulness of the approach. Taken together, the problem identification, motivation, and empirical results lead me to recommend weak acceptance.

Critical weaknesses

My main remaining concern is the robustness of the calibration used to construct the harmfulness and refusal reference means, mu and nu. These reference vectors are central to detection and token localization. Section 4 describes their construction from contrast prompts, intervention-based layer/position selection, and separate calibration and validation checks; I appreciate that these steps are provided. However, I would welcome a clearer account of how representative the calibration examples are and how stable the resulting means and performance are when the sample composition or size changes. In particular, dataset or wording differences should be distinguished from the intended safety signals. I view this as a minor or fixable concern, rather than a demonstrated failure of the method.

Post-rebuttal update: Thank you for directly testing the sensitivity of the reference directions and thresholds to sample size, resampling, and changes in data source and wording. These experiments substantially address my calibration concern in the settings tested. The adaptive evaluation still shows limits to the defense, which the revision makes clearer. I maintain my rating of weak acceptance (3).

Questions for authors

How sensitive are the reference means mu and nu, the selected layers and thresholds, and the resulting HaRD detection/localization performance to the composition and size of the calibration sets? Could you clarify how examples are chosen and how the construction avoids capturing dataset or wording differences rather than harmfulness/refusal signals? Repeated calibration samples, a set-size sensitivity analysis, or existing evidence addressing this point would help assess the reliability of the construction. I acknowledge the intervention-based selection and held-out group checks already described in Sections 4.1–4.2.

10 comments
AuthorsOpenReview ↗

Response to Reviewer 2Mwc (Part 1/9)

Dear Reviewer 2Mwc,

Thank you for your careful review. We are glad that you found the paper coherent and well-motivated, and the approach useful. We address each point below; changes in the revised manuscript are highlighted in blue, and the anonymized supplementary code is updated.

W and Q.1. Splitting and selection of the samples

The construction samples are split by the model's own replies; construction, calibration and evaluation do not overlap; and the overlap that two attacks bring with them does not affect the results.

1) The construction samples are split by the model's own replies, and construction, calibration and evaluation do not overlap

Basis of the method. Both directions are computed by the difference-in-means of Arditi et al. (2024), the mean activation of the positive samples minus that of the negative samples. The split follows Zhao et al. (2025) and uses only "instructions with desired model behaviors": harmful instructions refused by the model serve as positives, and harmless instructions accepted by the model serve as negatives. The two directions share the same positives but are read at different token positions, and the token position and the layer are selected by causal intervention (Section 4.1); the two means therefore differ, and so do the resulting directions. Construction and calibration use only the original prompts of the datasets, without any jailbreak attack; jailbreak attacks are added only at evaluation.

How the samples are selected. No instruction is picked by hand. AdvBench and Alpaca are shuffled with a fixed random seed and sampled in order, and from XSTest we take the first 50 accepted instructions in the official release order.

Threshold calibration. The thresholds are set on the negatives only, at the 95th percentile. The maximum is an outlier, and between the 90th and the 99th percentiles the 95th balances the detection rate (94% / 99%) and false positives (0.7% on OKTest). The calibration set contains two kinds of harmless instruction: those the model accepts normally (Alpaca) and those it over-refuses (the 16 refused XSTest instructions, which Zhao et al. (2025) call "harmless but overly-refused"). The latter naturally have higher refusal scores, so including them in calibration places the thresholds above these harmless instructions, which are the most prone to false positives.

Table: samples used for construction, calibration and evaluation

Harmfulness direction (construction)Refusal direction (construction)Threshold calibrationEvaluation
Positive (harmful)100 AdvBench instructions refused by the modelSame as leftNoneAttacks: jailbreak prompts generated by the seven attacks with AdvBench harmful behaviors as targets; generalization: 50 JailbreakBench behaviors (newly added second evaluation dataset)
Negative (harmless)50 Alpaca + 50 XSTest instructions, all accepted by the model100 Alpaca instructions, accepted by the modelAnother 100 accepted Alpaca instructions + the 16 refused XSTest instructionsOver-refusal: 300 OKTest prompts; utility: 3750 LongBench instances

The construction, calibration and evaluation samples do not overlap with one another, and the 50 AdvBench target behaviors used for attack evaluation were removed from AdvBench before construction.

AuthorsOpenReview ↗

Response to Reviewer 2Mwc (Part 2/9)

W and Q.1 (continued)

2) Part of the attack data necessarily overlaps the construction set, and this does not affect the results

Why the overlap arises. The construction set comes from AdvBench, and so do the data that come with two of the attacks:

  • AdvPrompter requires that its attacks be generated only on the AdvBench test questions it designates, and these 50 targets differ from those of the other six attacks;
  • ICA and I-FSJ insert demonstration question-answer pairs before the request, taken from the pool released by the I-FSJ authors, which is also built from AdvBench.

Before construction we removed only the 50 targets of the six attacks; these two parts of the data were not removed.

Why this is necessary. These attacks can be reproduced faithfully only with the pipelines and data that their authors released; if we replaced the targets or the demonstrations ourselves, we would no longer be measuring the original attacks. To test whether the overlap nevertheless affected the results, we evaluated it in two ways: for AdvPrompter we removed the overlapping targets from the evaluation, and for ICA and I-FSJ we used construction sets that contain none of these demonstrations.

Table: tests of the overlap

OverlapNumberTestResult
AdvPrompter targets12 of 50Remove these 12 and recompute on the remaining 38ASR of HaRD on this attack falls from 64% to 55%, in fact better
ICA and I-FSJ demonstrations55 of the 100 construction instructionsConstruction sets with none of these demonstrations (n = 25 and 50, each redrawn 20 times); detection rate on context attacksContext detection rate 99%, against 99.6% for the original construction

Summary of the evaluation sets

EvaluationRelation to the construction setResult
Six attacks (50 AdvBench targets)Same source, no overlap (in-domain)Mean ASR falls from 69% to 10%
Overlapping parts of AdvPrompter, ICA and I-FSJPartly overlapping, tested separatelyEssentially unchanged or better after removing the overlap
JailbreakBench (newly added in the rebuttal)Different source, no overlap at all (out-of-domain)HaRD performs at a similar level on JailbreakBench and AdvBench

Changes in the paper: Appendix S; Appendix X.

AuthorsOpenReview ↗

Response to Reviewer 2Mwc (Part 3/9)

W and Q.2. Reducing the construction set to a quarter and resampling it at random leaves the directions, thresholds and detection effectiveness stable

With a quarter of the construction samples, redrawn at random, the directions, thresholds, detection and layer selection of HaRD barely change. The samples used to construct μ\mu and ν\nu are what the paper calls the construction set, as distinct from the calibration set that determines the thresholds.

Purpose. HaRD constructs its directions from 100 positive and 100 negative samples. To test whether the results depend on this particular batch of samples or on this sample size, we set the construction set to 25, 50 and 100 samples and redrew each size 20 times at random from the sample pool, excluding all calibration and evaluation samples. Each draw rebuilds the directions and recalibrates the thresholds, and we then measured detection and localization on the same attack prompts as Table 1 of the paper.

Table: resampled construction sets (mean ± std over 20 draws)

Metricn = 25n = 50n = 100Original construction
Cosine of the harmfulness direction with the original0.83 ± 0.020.88 ± 0.010.90 ± 0.011.00
Cosine of the refusal direction with the original0.98 ± 0.010.99 ± 0.000.99 ± 0.001.00
Threshold τh\tau_h0.118 ± 0.0140.123 ± 0.0080.125 ± 0.0070.114
Threshold τr\tau_r0.071 ± 0.0070.069 ± 0.0040.072 ± 0.0030.064
Single-prompt detection rate89% ± 391% ± 191% ± 194.1%
Context detection rate99% ± 199% ± 099% ± 099.6%
Single-prompt localization hit rate79% ± 179% ± 179% ± 180.3%
OKTest false positives0.4% ± 0.30.3% ± 0.20.3% ± 0.20.7%
Selected layers (harmfulness / refusal)L9 / L15: 18 times; L6 / L15: 2 timesL9 / L15: 19 times; L8 / L15: 1 timeL9 / L15: 20 timesL9 / L15
Reviewer_ri9vOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper points out that using KV cache compression as a jailbreak defense mechanism has fundamental flaws. Through empirical studies, the authors demonstrate that there is no correlation between attention-importance-based token ranking and harmfulness, and that defenses based on the geometric structure of the key space cannot effectively locate the refusal mechanism. The authors propose a training-free defense method called HaRD, which locates harmful instructions by reading the corresponding representation directions and removes the attack prompts. The authors prove the causal source of the model's refusal behavior and define the sequential order of the "harmfulness" and "refusal" signals in the residual stream as the "refusal trajectory," which is the most prominent innovation of this study.

Critical strengths

HaRD is a training-free defense framework.

HaRD reveals the potential shortcomings of methods such as RobustKV.

HaRD significantly reduces the attack success rate across seven different types of jailbreak attacks.

This is an excellent paper with a strong empirical foundation, and I really appreciate it.

Critical weaknesses

If a model has not undergone any safety training at all (i.e., a completely unaligned base model), then the extracted "harmfulness" and "refusal" directions may be completely ineffective.

To achieve training-free operation, the defender must rely excessively on calibration data, and intuitively, the distribution of the calibration data will directly affect the defense performance.

Questions for authors

The authors' motivation is highly valuable. HaRD is a prompt-level identify-and-remove method that essentially does not involve the KV cache, so future work could reframe the narrative.

11 comments
AuthorsOpenReview ↗

Response to Reviewer ri9v (Part 1/10)

Dear Reviewer ri9v,

Thank you for your careful review. We are glad that you found the motivation of our work highly valuable, and that HaRD significantly reduces the attack success rate. We address each point below; changes in the revised manuscript are highlighted in blue, and the anonymized supplementary code is updated.

Q. The title narrows the positioning, and the dependence of the method on the failure-mode analysis of safety-aware KV compression is made explicit

We agree with the reviewer that HaRD does not use KV compression, and we have revised the title, the abstract, the related work and Section 3 accordingly. Our work evaluates the approach that has recently appeared in training-free, input-level defenses, evicting tokens from the KV cache, and finds that it fails on many attack families. From this we carry out a failure-mode analysis, and, after identifying the findings of this analysis, we design HaRD, a prompt-level identify-and-remove defense, from the conclusions drawn from these findings.

1) Title. The title of the paper could lead readers to believe that the method is based on KV compression. We have therefore removed KV compression from the title so that it describes only our method: What to Delete and by Which Signal: Harmfulness-Refusal-Guided Token Deletion against Jailbreaks.

Changes in the paper: title.

2) Field. In the abstract and the related work we now emphasize that this work belongs to training-free, input-level defenses, whose current methods can be divided into those that manipulate the prompt and those that evict tokens from the KV cache; these methods also serve as baselines in Section 5 (Evaluation).

Changes in the paper: abstract, first sentence; Section 2 (Background and related work), paragraph "Training-free input-level defenses".

3) Link between the failure-mode analysis and HaRD. Section 3 did not state explicitly how its findings relate to HaRD in Section 4. We have given each of the six findings a bold paragraph heading and added the chain of argument to the section: we test the three assumptions that safety-aware compression rests on; these assumptions fail on all three models we test, which differ in safety alignment; in analyzing the causes of the failures we identify several findings, each supported by a measurement or a causal intervention; and finally we design HaRD from these findings.

Changes in the paper: Section 3 (Failure modes analysis).

AuthorsOpenReview ↗

Response to Reviewer ri9v (Part 2/10)

W1.1. A base model without safety training (harmfulness separable, refusal not separable) is outside the scope of HaRD

A base model already separates harmfulness, but its refusal direction discriminates only weakly; without safety training, it has no refusal for HaRD to amplify. We recomputed both directions on the Llama-3.1-8B base model from its own activations and compared them with the instruct model.

Table: mean score on harmful / harmless requests, and AUROC

Harmfulness scoreRefusal score
Instruct0.182 / 0.075, AUROC 0.980.095 / −0.001, AUROC 1.00
Base0.294 / 0.169, AUROC 0.980.007 / 0.005, AUROC 0.72
  • Harmfulness is separable before safety training. The base model is already able to separate harmfulness (AUROC 0.98).
  • Refusal is not. The refusal direction of the base model has an AUROC of only 0.72, so its discrimination is weak.
  • This is the boundary of our method. Safety training is what establishes refusal. HaRD relies on the alignment the model already has to refuse, and a base model has no refusal for HaRD to amplify; we now state this in Section 6 and Appendix V.

Such a model, however, does not refuse harmful requests to begin with. AutoDAN (Liu et al., 2024b) points out that prompting an unaligned base model directly yields harmful output, and our measurement agrees: of 50 harmful requests without any jailbreak prompt, the base model directly produces harmful content for 72%. There is therefore no jailbreak here to defend against.

Changes in the paper: Section 6, paragraph "Refusal follows the victim model's alignment"; Appendix V.

W1.2. Across safety-aligned models of different alignment strength, the bottleneck lies in the model's own refusal

The detection of HaRD holds on all three models; how much the ASR falls depends on whether the model itself refuses once the jailbreak prompt is gone.

1) The gradient of alignment strength. Each cell gives ASR / refusal rate (%), without defense to the left of the arrow and with HaRD to the right (Table 4; Appendix Q, Table 19).

Single-prompt attacksContext attacks
Llama-3.1-8B-Instruct81/15 → 30/5366/33 → 5/94
Qwen2.5-7B-Instruct90/6 → 36/4013/84 → 0/99
Mistral-7B-Instruct-v0.398/0 → 60/397/2 → 61/13
Mistral-7B-Instruct-v0.3 (refusing on detection)98/0 → 20/8097/2 → 1/99
  1. Strongly aligned models refuse on their own. On Llama and Qwen, once HaRD deletes the jailbreak prompt, the model itself refuses the remaining harmful request, so the ASR falls sharply and the refusal rate rises sharply.
  2. The weakly aligned model does not. On Mistral, the model still does not refuse after the jailbreak prompt is deleted, so the ASR stays around 60% and the refusal rate is only 3% and 13%.
  3. Detection itself still works. When the intervention is changed to refusing directly once an attack is detected, the ASR / refusal rate of Mistral becomes 20/80 and 1/99. This shows that the detection of HaRD remains effective on a weakly aligned model, and that the bottleneck lies only in the model's own refusal.

2) HaRD in three steps (%, single-prompt / context attacks)

QwenLlamaMistral
(1) Detection: jailbreak attacks detected88 / 10094 / 10081 / 98
(2) Localization: kept text is indeed the harmful request718071
(3) Refusal: kept harmful requests that the model refuses71 / 10078 / 945 / 13

Row (2) reports single-prompt attacks only: for a context attack the final turn is kept directly and no localization is needed.

  • Detection and localization are stable on all three models, including the weakly aligned Mistral.
  • Refusal is carried out by the safety alignment of the victim model itself. Llama and Qwen refuse most of the kept harmful requests, whereas Mistral refuses only 5% and 13%.
  • The limited defense on Mistral comes from its insufficient alignment. This is also why refusing on detection reduces the ASR of Mistral to 20% and 1%. Over-refusal rises only from 5.3% to 6.3%, and without the forced refusal it stays at the undefended 5.3%.

Changes in the paper: Section 6, paragraph "Refusal follows the victim model's alignment"; Appendix Q, Table 20; Appendix V.

AuthorsOpenReview ↗

Response to Reviewer ri9v (Part 3/10)

W2.1. Splitting and selection of the samples

The construction samples are split by the model's own replies; construction, calibration and evaluation do not overlap; and the overlap that two attacks bring with them does not affect the results.

1) The construction samples are split by the model's own replies, and construction, calibration and evaluation do not overlap

Basis of the method. Both directions are computed by the difference-in-means of Arditi et al. (2024), the mean activation of the positive samples minus that of the negative samples. The split follows Zhao et al. (2025) and uses only "instructions with desired model behaviors": harmful instructions refused by the model serve as positives, and harmless instructions accepted by the model serve as negatives. The two directions share the same positives but are read at different token positions, and the token position and the layer are selected by causal intervention (Section 4.1); the two means therefore differ, and so do the resulting directions. Construction and calibration use only the original prompts of the datasets, without any jailbreak attack; jailbreak attacks are added only at evaluation.

How the samples are selected. No instruction is picked by hand. AdvBench and Alpaca are shuffled with a fixed random seed and sampled in order, and from XSTest we take the first 50 accepted instructions in the official release order.

Threshold calibration. The thresholds are set on the negatives only, at the 95th percentile. The maximum is an outlier, and between the 90th and the 99th percentiles the 95th balances the detection rate (94% / 99%) and false positives (0.7% on OKTest). The calibration set contains two kinds of harmless instruction: those the model accepts normally (Alpaca) and those it over-refuses (the 16 refused XSTest instructions, which Zhao et al. (2025) call "harmless but overly-refused"). The latter naturally have higher refusal scores, so including them in calibration places the thresholds above these harmless instructions, which are the most prone to false positives.

Table: samples used for construction, calibration and evaluation

Harmfulness direction (construction)Refusal direction (construction)Threshold calibrationEvaluation
Positive (harmful)100 AdvBench instructions refused by the modelSame as leftNoneAttacks: jailbreak prompts generated by the seven attacks with AdvBench harmful behaviors as targets; generalization: 50 JailbreakBench behaviors (newly added second evaluation dataset)
Negative (harmless)50 Alpaca + 50 XSTest instructions, all accepted by the model100 Alpaca instructions, accepted by the modelAnother 100 accepted Alpaca instructions + the 16 refused XSTest instructionsOver-refusal: 300 OKTest prompts; utility: 3750 LongBench instances

The construction, calibration and evaluation samples do not overlap with one another, and the 50 AdvBench target behaviors used for attack evaluation were removed from AdvBench before construction.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response and Summary of Revisions

Dear Reviewers and Area Chairs,

Thank you for your careful reviews. We are glad that you found the motivation of our work highly valuable (ri9v), the paper coherent and well-motivated (2Mwc) and the analysis insightful (dfN1), and that you found HaRD's reduction in attack success rate significant (ri9v), the approach useful (2Mwc) and HaRD a meaningful approach (dfN1).

In training-free, input-level jailbreak defense, a recent line of work evicts tokens from the KV cache. We show that the assumptions behind this approach fail on the models we test, and our failure-mode analysis holds on three models with different alignment strength. From these findings we design HaRD, a prompt-level defense. Across seven attacks, HaRD gives the lowest ASR among ten baselines on Llama-3.1-8B-Instruct (69% → 10%) and the lowest mean ASR among the compared methods on Qwen and Mistral. It performs comparably to or better than dedicated fine-tuned safety classifiers without an extra guard model. Its utility is within 0.2 of the best baseline, its over-refusal stays at the undefended level, and it adds no latency or memory.

The revised manuscript highlights all changes in blue, and the anonymized supplementary code is updated. The main revisions are:

1. Revised the title (ri9v, dfN1). An analysis on a limited set of models cannot support an absolute conclusion, so the title now names only the method, the conclusions are restricted to the models we test, and Section 3 makes explicit how its six findings lead to HaRD.

2. Extended the failure-mode analysis to three models (dfN1; Section 3, Appendices C, E, G). The token-eviction defenses RobustKV and AnchorKV are not reliable on any of the three models; on Qwen and Mistral, apart from context attacks, their mean ASR is essentially the same as without defense. These results support the failure modes we identify (attention importance does not correlate with harmfulness; no direction in the key projection causally mediates refusal; eviction after prefill does not eliminate the influence of the evicted tokens).

3. Clarified the scope of the method (ri9v, dfN1; Section 6, Appendices Q, V). HaRD relies on the model's own alignment to refuse. A base model without safety training separates harmfulness but not refusal, and already gives harmful output for most plain harmful requests, so there is no jailbreak to defend against. On aligned models, detection holds across different alignment strengths and the ASR reduction follows the model's own refusal; on the weakly aligned Mistral, refusing on detection substantially mitigates this.

4. Analyzed the sensitivity to the construction and calibration data (ri9v, 2Mwc; Appendix S). Construction, calibration and evaluation samples do not overlap, and none is picked by hand. The directions, the selected layers and the thresholds remain stable when the construction and calibration sets are resampled, changed in size or replaced with other datasets. Swapping in another harmful benchmark, or replacing the harmless questions with imperatives like the harmful instructions, changes the mean ASR by at most two points, so the directions capture harmfulness and refusal rather than dataset style or phrasing.

5. Showed that both directions are needed (dfN1; Appendix T). Either direction alone raises OKTest false positives to 16.7% or 4.0%, against 0.7% with both. The harmfulness-then-refusal order is a common pattern we observe, not a strict rule, so HaRD does not use it as a detection criterion.

6. Added comparison methods, a second judge and a second dataset (dfN1 and our additions; Appendices T, U, X). We added Token Highlighter as a baseline (mean ASR 73%). To compare detection ability, we also replaced the two directions of HaRD with the dedicated fine-tuned safety classifiers WildGuard and ShieldGemma, which serve as references rather than baselines: HaRD matches WildGuard in mean ASR (10%) and is lower than ShieldGemma (33%), with fewer false positives on OKTest (0.7%, against 10.7% and 18.3%).

Under the HarmBench judge, HaRD clearly lowers harmful output on all three models, as under Llama Guard. On JailbreakBench, a second dataset, HaRD lowers the mean ASR on all three models, as on AdvBench.

7. Extended the limitations (dfN1; Section 6). Fragmented and rewritten attacks: on ReNeLLM the remaining replies are mostly benign but seldom explicit refusals, so the refusal rate understates the defense. Harmful and benign prompts are not fully separable: a long enough prompt can still bypass the defense, a limit of any defense that reduces a harmful behavior without removing it (Wolf et al., 2024). Our upgraded HaRD recovers 69% and 53% of such cases but raises false positives on long benign documents, a trade-off between robustness to this attack and false positives, so we keep it optional.

We thank all reviewers again for their constructive feedback; detailed responses are in the individual threads.

Area_Chair_ARZsOpenReview ↗

Reminder: Please review the author rebuttal before the deadline

Dear reviewers,

The rebuttal period is about to close. If you haven’t responded yet, please read the authors’ rebuttal and leave a brief comment before the deadline.

If you’ve already responded, please disregard this reminder. Thanks for your time, and apologies for the extra email.

Best regards,

Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.