- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates safety incompatibility in cross-model KV cache reuse, where a receiver model continues generation using KV states produced by a related sender model. The authors define safety incompatibility relative to receiver-native inference and introduce metrics to distinguish safety degradation, improvement, and overall deviation. Through experiments involving six base/instruction-tuned model pairs and two directional LoRI-D configurations, they show that cache reuse can substantially alter safety behavior in either direction. Their experiments further suggest that reusing value tensors generally produces larger changes in aggregate harmfulness than reusing key tensors.
To mitigate this issue, the authors propose VReparo, which learns per-layer, per-KV-head affine transformations to align reused value tensors with receiver-native values. These transformations are fitted offline, with their application strength calibrated using refusal behavior. Experiments on HarmBench and StrongREJECT demonstrate improvements in safety metrics under both full KV reuse and DroidSpeak-based selective reuse, while additional evaluations examine benign refusal, QA performance, and inference latency.
Critical strengths
- The paper identifies a safety concern that is largely overlooked in existing cross-model KV cache reuse research. The systematic comparisons across model families, reuse directions, and K/V configurations provide useful evidence that preserving serving efficiency or conventional generation quality does not necessarily preserve safety behavior. The observation that reuse can either increase or decrease harmfulness is particularly informative.
- The reported reductions of VReparo in harmfulness across both full and selective reuse settings are encouraging, particularly given that mapping is applied only once to the reused prefix values.
Critical weaknesses
- While the paper provides extensive empirical evidence that cross-model KV cache reuse can alter safety behavior, it offers limited theoretical or mechanistic explanations for why such incompatibility arises. In particular, it remains unclear why value reuse generally induces larger safety deviations than key reuse, and why the direction and magnitude of safety changes vary substantially across model pairs.
- While VReparo improves safety compatibility, it remains unclear whether its benefits justify the additional computational overhead compared with simpler alternatives, such as recomputing more receiver layers in DroidSpeak. In Section 6, the authors fix the recomputation block and evaluate VReparo on top of it. Can the authors add additional experiments on that?
Questions for authors
- How does model size affect safety incompatibility under cross-model KV cache reuse? The current results suggest different trends across model families: Qwen3-8B exhibits substantially smaller safety deviations than Qwen3-4B, whereas Llama-3-8B exhibits larger deviations than Llama-3.2-3B. Could the authors investigate whether these differences are attributable to model scale, model-specific alignment, or other factors? Additionally, does the effectiveness of VReparo generalize across different model sizes?
- Can the authors provide a theoretical or mechanistic explanation for the observed safety incompatibility, as mentioned in Weakness 1?
- Refer to Weakness 2.
5 comments
2. Additional recomputation
Thanks for suggesting this baseline. During the rebuttal period, we added 4-, 8- and 16-layer recomputation under DroidSpeak-style reuse [1], selecting each block by QA quality alone. VReparo has lower SDV than four-layer recomputation at about 18% lower receiver TTFT. Sixteen-layer recomputation achieves lower SDV at higher latency. In the table below, paired entries report LoRI-D / Llama-3-8B, with receiver TTFT measured from resident sender states. VReparo safety results average five fits, while timing uses one fixed fit.
| Condition | Receiver TTFT (ms) | HarmBench SDV (%) | StrongREJECT SDV |
|---|---|---|---|
| Raw reuse: 2 layers recomputed | 202.32 / 200.53 | 59.84 / 26.77 | 0.5270 / 0.1470 |
| VReparo: 2 layers + maps | 234.93 / 233.21 | 38.58 / 13.39 | 0.2989 / 0.0362 |
| 4 layers recomputed | 286.24 / 283.28 | 57.48 / 18.11 | 0.5160 / 0.0938 |
| 8 layers recomputed | 453.30 / 448.36 | 52.76 / 28.35 | 0.4729 / 0.1352 |
| 16 layers recomputed | 788.15 / 779.62 | 6.30 / 11.81 | 0.0263 / 0.0118 |
Timing removes a redundant suffix prefill from the Table 4 implementation, and generated tokens are unchanged in all 66 parity checks. On Llama-3-8B, VReparo has similar HotpotQA F1 to four-layer recomputation (43.42 vs. 43.73) but higher benign refusal (17.1% vs. 12.0%). The comparison is in Section 6.2 and Appendix B.7 (Table 14) of the revised paper.
3. Model size and Qwen3-4B
We cannot attribute the original differences to model size alone, because training and alignment also differ across pairs and the two Llama sizes come from different generations (Table 1). To test VReparo at another size, we added Qwen3-4B Base→Instruct under both full-KV and DroidSpeak-style reuse [1], with α = 0.5.
| Configuration | Condition | HarmBench ASR (%) | HarmBench SDV (%) | StrongREJECT SDV |
|---|---|---|---|---|
| Full KV | Raw reuse | 21.26 | 15.75 | 0.0958 |
| VReparo | 13.23 | 7.72 | 0.0620 | |
| DroidSpeak | Raw reuse | 18.11 | 11.02 | 0.0647 |
| VReparo | 14.49 | 6.77 | 0.0503 |
VReparo also reduces SDV at 4B. HarmBench and StrongREJECT SDV both fall under full-KV and DroidSpeak-style reuse, and HarmBench ASR moves toward Native.
Benign refusal stays near Native under full-KV reuse. OR-Bench full refusal is 9.0% with VReparo, compared with 8.5% for Native.
1. Mechanistic explanation
We added comparisons using the same set of prompts and layer-group tests alongside the internal traces.
Replacing V changes what attention reads. At reused prefix positions, K-only changes attention weights but keeps the receiver’s own V, whereas V-only replaces that V. Relative to Native, V-only exceeds K-only in attention-output deviation by 0.256 (95% CI [0.239, 0.272]) for LoRI-D and 0.658 ([0.648, 0.668]) for Llama-3-8B.
LoRI-D behavior moves toward the sender. On the 111 prompts valid in both LoRI-D directions, V-only reuse moves ASR close to the sender’s Native ASR:
Configuration and layer position affect the result. Qwen3-8B shows opposite directions under V-only and full-KV reuse (Table 2). Under DroidSpeak-style reuse [1], with maps and α unchanged, mapping early reused layers lowers HarmBench SDV more than mapping middle or late layers:
These results help explain the observed differences but do not predict their magnitude across model pairs. The analyses are in Sections 4.3.2 and 6.2 and Appendices A.3 and B.8.