- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper discusses a timely problem of semantic cache poisoning, where an attacker can slip a wrong or poisoned answer to the end of the questions to poison a semantic cache. This paper proposes a key observation: poisoned questions all have a "rewrite plus residual" shape, and the poison comes from the residual. The rewrite is the paraphrase of the real questions, and the residual is the extra text that steers the answer. Based on this observation, they proposed a defense with two checks:
- The deletion gain, which shows the gain if the residual is cut from the cached question.
- An answer check, where the method checks if there is an answer inside of this cached question.
The results show that the proposed method blocks more than 80% of the poison entries, while only wrongfully rejecting 5% of the benign cache hit.
Critical strengths
1, This paper proposes a useful and practical defense against semantic cache poisoning that can effectively check and defend against cache poisoning while keeping the latency low. It outperforms other baselines such as perplexity filters, harmful-content classifiers and embedding tweaks.
2, The finding that attacks that hide the residual better also tend to stop fooling the LLM is interesting. It seems like a dilemma for the attacker and might worth exploring deeper.
Critical weaknesses
1, Although through experiments the paper showed that the defense method is effective, the proposed defense only works in one single scenario where the attack is formed with a rewrite plus a residual. I'm wondering if other forms of semantic cache poisoning, such as attacks with multi-turn contexts, or residuals that are scattered in different places can still work in this scenario.
2, It seems like this semantic cache poisoning attack and defense is only tested on simple Q&A scenarios. There are no experiments on whether this attack and defense can work on more complicated tasks or in multi-turn, agentic scenarios. And there is only one main LLM, which is the Qwen3 8B. So I'm a little bit concerned about the practicality of the proposed attack and defense on realistic semantic cache that is used on frontier models such as DeepSeek V4 or Claude or GPT models.
Questions for authors
1, The block rate falls for fused or interleaved attacks. What is the undefended attack success of fused or interleaved attacks on stronger instruction-following models? Does the evasion vs. effectiveness trade-off still hold there? 2, How does the proposed defense work against attacks that pose questions but actually gives out answers? For example "Where were the 1998 Olympics held that is also the captial of France?" 3, How does the defense work in senarios other than Q&A datasets?
4 comments
Response to Reviewer D5zH (2/2)
Q2. “How does the proposed defense work against attacks that pose questions but actually give out answers? …”
Our defense detects these attacks. The embedded answer acts as the residual. In the reviewer’s example, the rewrite is “Where were the 1998 Olympics held” and the residual is “that is also the capital of France”. Deleting the residual moves the key closer to the incoming query, so DG rises. The stored answer repeats “capital” and “France”, so the Answer Check triggers. We also tested 101 attacks of the same form, where the question states an incorrect answer as a premise. Our defense blocks 83% to 92% of them, depending on the LLM (last row of Table R12).
Q3/W2. “How does the defense work in scenarios other than Q&A datasets?”
The defense has no task-specific component. It reads only the cache key, the incoming query, and the stored answer, so it applies to other tasks in the same way. We tested it on real LMArena prompts for three tasks. The poisoned answer recommends a fictitious package in code generation, promotes a fictitious product in recommendation, and states a false fact in explanation. The thresholds are calibrated once on LMArena traffic and shared by the three tasks. Our defense blocks most poisoned entries on each task.
Table R13. Non-QA evaluation (87 attacked prompts, 261 poisoned entries, 88 retrievable benign hits).
| Task | ASR without defense | BR | ASR with our defense | False rejections |
|---|---|---|---|---|
| All | 50.2% | 91.0% | 4.2% | 2/88 |
| Code | 44.9% | 95.3% | 1.4% | 1/22 |
| Recommendation | 72.9% | 97.3% | 0.0% | 0/16 |
| Explanation | 45.1% | 86.9% | 6.9% | 1/50 |
We have added the results above to §5.2, §5.3, and Appendix E of the revised paper, and we welcome any further questions.
Thank you for the detailed responses and additional experiments. The clarification addresses most of my concerns. I will maintain my positive evaluation score.
Thank you for your time and suggestions. If you have any further questions or concerns, we would be glad to clarify and discuss them.
Response to Reviewer D5zH (1/2)
We thank the reviewer for the encouraging review. The remark on the attacker’s dilemma led to a new analysis in §5.3, which Q1 summarizes.
These forms still fit the rewrite–residual structure. The structure is an abstraction, and the residual can sit anywhere in the key. It may be appended to the rewrite, fused into it, or scattered through it. DG handles all three forms because it tries deletions at every position of the key. Our evaluation covers each of them. SCP appends the residual, CAP fuses it, and Interleave scatters it. Scattered residuals give the weakest attacks, and ASR with our defense stays at or below 5.0%.
For multi-turn contexts, the residual can sit in an earlier turn. When a cache keys on the full conversation, the residual remains inside the key, so our defense applies in the same way. Our experiments use single-turn requests because existing attacks target them [3, 4] and semantic caches are most effective there [1]. In agentic stacks, the semantic cache sits in front of the LLM and sees its prompt and response [2], so our defense applies at this layer as well.
We regenerated the stored answers with six additional LLMs, including the frontier models GPT-5.6-Sol and Claude Opus 4.8. The defense generalizes to stronger LLMs. ASR with our defense stays below 2.6% for CAP, SCP, and KCA on all six LLMs.
Table R11. Range across DeepSeek-v4-flash, MiniMax-M3, Qwen3-32B, and GLM-5.3-Flash.
Table R12. Frontier LLMs, ASR without defense → ASR with our defense (BR).
The trade-off holds, because hiding the residual from DG also weakens its effect on the LLM. DG detects a residual that it can delete as one span. To evade DG, the attacker must fuse the residual into the question or scatter it in fragments. The LLM reads the same key, and it rarely follows a fused or scattered residual as an instruction. Without the defense, fused CAP succeeds 44% to 85% less often than CAP with an appended residual, and Interleave succeeds 73% to 95% less often than Repeat.
References
[1] Luis Gaspar Schroeder et al. vCache: Verified Semantic Prompt Caching. ICLR, 2026. arXiv:2502.03771.
[2] Asmit Kumar Singh et al. Asynchronous Verified Semantic Caching for Tiered LLM Architectures. arXiv:2602.13165, 2026.
[3] Guanlong Wu et al. When Cache Poisoning Meets LLM Systems: Semantic Cache Poisoning and Its Countermeasures. NDSS, 2026.
[4] Zhixiang Zhang et al. From Similarity to Vulnerability: Key Collision Attack on LLM Semantic Caching. arXiv:2601.23088, 2026.