acceptodds
Under review as a conference paper at ICLR 2027

Recovering What Embeddings Lose: Detecting Cache Poisoning in LLM Semantic Caches

Abstract

Semantic caches reduce LLM serving costs by reusing previously generated answers for semantically similar queries. However, retrieval is based solely on embedding similarity between the incoming query and cached queries. This design enables cache poisoning: an attacker can cache a malicious response under a query with high cosine similarity to benign requests. The vulnerability stems from a gap between retrieval similarity and answer validity. From an information-bottleneck perspective, query embeddings can lose information needed to distinguish valid from invalid cache hits, which limits any matching algorithm that uses only these embeddings. We propose a novel defense that recovers this necessary information from the raw text of the cache key. Across poisoning attacks, adversarial queries share a rewrite-residual structure: they pair a rewrite of the target query with residual content. The rewrite maintains high similarity, while the residual elicits the malicious response. Deleting the residual makes the remaining rewrite more similar to the incoming query. We exploit this structure using Deletion Gain to search shortened variants of the cached query for similarity gains, and an Answer Check to test whether the removed text contributes to the stored answer. We prove that Deletion Gain stays positive when a deletion leaves text close enough to the rewrite, and we search for such deletions with a sliding window. Across three poisoning attack classes, our defense blocks 82.0% to 98.2% of poisoned entries at a 5% false-positive rate, with negligible serving overhead.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Semantic Caching, Cache Poisoning, LLM Security, Retrieval Poisoning, Text Embeddings

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_D5zHOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper discusses a timely problem of semantic cache poisoning, where an attacker can slip a wrong or poisoned answer to the end of the questions to poison a semantic cache. This paper proposes a key observation: poisoned questions all have a "rewrite plus residual" shape, and the poison comes from the residual. The rewrite is the paraphrase of the real questions, and the residual is the extra text that steers the answer. Based on this observation, they proposed a defense with two checks:

  1. The deletion gain, which shows the gain if the residual is cut from the cached question.
  2. An answer check, where the method checks if there is an answer inside of this cached question.

The results show that the proposed method blocks more than 80% of the poison entries, while only wrongfully rejecting 5% of the benign cache hit.

Critical strengths

1, This paper proposes a useful and practical defense against semantic cache poisoning that can effectively check and defend against cache poisoning while keeping the latency low. It outperforms other baselines such as perplexity filters, harmful-content classifiers and embedding tweaks.

2, The finding that attacks that hide the residual better also tend to stop fooling the LLM is interesting. It seems like a dilemma for the attacker and might worth exploring deeper.

Critical weaknesses

1, Although through experiments the paper showed that the defense method is effective, the proposed defense only works in one single scenario where the attack is formed with a rewrite plus a residual. I'm wondering if other forms of semantic cache poisoning, such as attacks with multi-turn contexts, or residuals that are scattered in different places can still work in this scenario.

2, It seems like this semantic cache poisoning attack and defense is only tested on simple Q&A scenarios. There are no experiments on whether this attack and defense can work on more complicated tasks or in multi-turn, agentic scenarios. And there is only one main LLM, which is the Qwen3 8B. So I'm a little bit concerned about the practicality of the proposed attack and defense on realistic semantic cache that is used on frontier models such as DeepSeek V4 or Claude or GPT models.

Questions for authors

1, The block rate falls for fused or interleaved attacks. What is the undefended attack success of fused or interleaved attacks on stronger instruction-following models? Does the evasion vs. effectiveness trade-off still hold there? 2, How does the proposed defense work against attacks that pose questions but actually gives out answers? For example "Where were the 1998 Olympics held that is also the captial of France?" 3, How does the defense work in senarios other than Q&A datasets?

4 comments
AuthorsOpenReview ↗

Response to Reviewer D5zH (1/2)

We thank the reviewer for the encouraging review. The remark on the attacker’s dilemma led to a new analysis in §5.3, which Q1 summarizes.

W1/W2. “… other forms of semantic cache poisoning, such as attacks with multi-turn contexts, or residuals that are scattered in different places … multi-turn, agentic scenarios.”

These forms still fit the rewrite–residual structure. The structure is an abstraction, and the residual can sit anywhere in the key. It may be appended to the rewrite, fused into it, or scattered through it. DG handles all three forms because it tries deletions at every position of the key. Our evaluation covers each of them. SCP appends the residual, CAP fuses it, and Interleave scatters it. Scattered residuals give the weakest attacks, and ASR with our defense stays at or below 5.0%.

For multi-turn contexts, the residual can sit in an earlier turn. When a cache keys on the full conversation, the residual remains inside the key, so our defense applies in the same way. Our experiments use single-turn requests because existing attacks target them [3, 4] and semantic caches are most effective there [1]. In agentic stacks, the semantic cache sits in front of the LLM and sees its prompt and response [2], so our defense applies at this layer as well.

Q1/W2. “What is the undefended attack success of fused or interleaved attacks on stronger instruction-following models? Does the evasion vs. effectiveness trade-off still hold there?”

We regenerated the stored answers with six additional LLMs, including the frontier models GPT-5.6-Sol and Claude Opus 4.8. The defense generalizes to stronger LLMs. ASR with our defense stays below 2.6% for CAP, SCP, and KCA on all six LLMs.

Table R11. Range across DeepSeek-v4-flash, MiniMax-M3, Qwen3-32B, and GLM-5.3-Flash.

AttackASR without defenseBRASR with our defense
CAP0.060–0.4960.764–0.8140.001–0.024
SCP0.455–0.8710.951–0.9620.008–0.025
KCA0.441–0.8500.947–0.9870.010–0.018
Interleave0.020–0.1600.535–0.7050.010–0.050
Repeat0.140–0.6000.865–0.9950.000–0.025

Table R12. Frontier LLMs, ASR without defense → ASR with our defense (BR).

AttackGPT-5.6-SolClaude Opus 4.8
CAP0.403 → 0.006 (0.728)0.033 → 0.000 (0.826)
SCP0.751 → 0.014 (0.957)0.252 → 0.006 (0.944)
KCA0.399 → 0.013 (0.978)0.397 → 0.005 (0.992)
Interleave0.060 → 0.005 (0.560)0.010 → 0.000 (0.600)
Repeat0.540 → 0.005 (0.835)0.080 → 0.000 (0.940)
Embedded answer0.277 → 0.030 (0.891)0.228 → 0.020 (0.891)

The trade-off holds, because hiding the residual from DG also weakens its effect on the LLM. DG detects a residual that it can delete as one span. To evade DG, the attacker must fuse the residual into the question or scatter it in fragments. The LLM reads the same key, and it rarely follows a fused or scattered residual as an instruction. Without the defense, fused CAP succeeds 44% to 85% less often than CAP with an appended residual, and Interleave succeeds 73% to 95% less often than Repeat.

References

[1] Luis Gaspar Schroeder et al. vCache: Verified Semantic Prompt Caching. ICLR, 2026. arXiv:2502.03771.

[2] Asmit Kumar Singh et al. Asynchronous Verified Semantic Caching for Tiered LLM Architectures. arXiv:2602.13165, 2026.

[3] Guanlong Wu et al. When Cache Poisoning Meets LLM Systems: Semantic Cache Poisoning and Its Countermeasures. NDSS, 2026.

[4] Zhixiang Zhang et al. From Similarity to Vulnerability: Key Collision Attack on LLM Semantic Caching. arXiv:2601.23088, 2026.

AuthorsOpenReview ↗

Response to Reviewer D5zH (2/2)

Q2. “How does the proposed defense work against attacks that pose questions but actually give out answers? …”

Our defense detects these attacks. The embedded answer acts as the residual. In the reviewer’s example, the rewrite is “Where were the 1998 Olympics held” and the residual is “that is also the capital of France”. Deleting the residual moves the key closer to the incoming query, so DG rises. The stored answer repeats “capital” and “France”, so the Answer Check triggers. We also tested 101 attacks of the same form, where the question states an incorrect answer as a premise. Our defense blocks 83% to 92% of them, depending on the LLM (last row of Table R12).

Q3/W2. “How does the defense work in scenarios other than Q&A datasets?”

The defense has no task-specific component. It reads only the cache key, the incoming query, and the stored answer, so it applies to other tasks in the same way. We tested it on real LMArena prompts for three tasks. The poisoned answer recommends a fictitious package in code generation, promotes a fictitious product in recommendation, and states a false fact in explanation. The thresholds are calibrated once on LMArena traffic and shared by the three tasks. Our defense blocks most poisoned entries on each task.

Table R13. Non-QA evaluation (87 attacked prompts, 261 poisoned entries, 88 retrievable benign hits).

TaskASR without defenseBRASR with our defenseFalse rejections
All50.2%91.0%4.2%2/88
Code44.9%95.3%1.4%1/22
Recommendation72.9%97.3%0.0%0/16
Explanation45.1%86.9%6.9%1/50

We have added the results above to §5.2, §5.3, and Appendix E of the revised paper, and we welcome any further questions.

Thank you for the detailed responses and additional experiments. The clarification addresses most of my concerns. I will maintain my positive evaluation score.

Thank you for your time and suggestions. If you have any further questions or concerns, we would be glad to clarify and discuss them.

Reviewer_n68WOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors address semantic cache poisoning, where embedding similarity causes incorrect answer reuse. It proposes a detector combining query decomposition, answer alignment, and lexical overlap. Experiments on ComQA, Natural Questions, and QQP with four embedding models show substantial reductions in attack success.

Critical strengths
  1. The method addresses a practical vulnerability in semantic caching and requires no additional serving-time model calls.
  2. The evaluation covers adaptive attacks, held-out calibration, and real vCache workloads, providing evidence beyond the primary attack benchmarks.
  3. Storage and verification-latency measurements help assess the method’s deployment costs.
Critical weaknesses
  1. The information-theoretic interpretation is stronger than the evidence establishes. Table 10 includes useful logistic-regression and MLP controls, but improved prediction after adding answer embeddings does not distinguish irrecoverable information loss from limitations of the evaluated readouts. The results establish the usefulness of these features more convincingly than they establish that the original embedding pair lacks the relevant information.

  2. Full-system deployment costs remain incompletely characterized. Table 16 explicitly excludes generation on cache misses, so the reported verification latency does not capture the downstream cost of additional rejected hits. The workload-specific calibration results are valuable; evaluating fixed thresholds across workload shifts would further clarify deployment robustness. This is a secondary concern given the existing practical evaluation.

Questions for authors

see above weakness

3 comments
AuthorsOpenReview ↗

Response to Reviewer n68W

We thank the reviewer for the constructive suggestions.

W1. “… improved prediction after adding answer embeddings does not distinguish irrecoverable information loss from limitations of the evaluated readouts. …”

This distinction helps us state our claim more precisely. The revision states it in terms of usable information [1]. Usable information is defined for a fixed family of predictors, such as logistic classifiers or small MLPs. It is the amount by which the input features lower the smallest cross-entropy that a predictor in this family achieves on the label.

A practical cache defense must check every cache hit, so it must stay lightweight. What matters is therefore the validity information accessible to a computationally bounded predictor, which is what usable information measures. Distinct texts usually map to distinct embeddings, but this does not guarantee that the validity information is easily extractable. We claim that the embedding pair loses this usable validity information.

We estimate usable information using five predictor families: logistic regression, two MLPs, kNN, and gradient-boosted trees. Each predictor is trained without CAP to evaluate generalization to an unseen attack.

Table R9. Usable information (bits) and AUC on CAP.

Input featuresUsable informationAUC
E(k), E(q)≤ 0.0040.53–0.57
+ E(y)0.015–0.0950.61–0.74
DG, ADL, Echo0.13–0.180.76–0.80

Here, E(k), E(q), and E(y) denote the embeddings of the cache key, the incoming query, and the stored answer. Table R9 shows three things. First, the predictors extract almost no validity information from E(k) and E(q). Second, adding E(y) improves prediction, even though the answer adds no information beyond the key text. The answer is decoded greedily from the key, so it is fully determined by the key text. The improvement therefore means that the key text holds validity information that the predictors cannot extract from E(k). Third, DG, ADL, and Echo are computed from the text itself, and they yield more usable information than either set of embeddings.

We have updated §3.3, §5.4, and Table 10 accordingly.

W2. “Table 16 explicitly excludes generation on cache misses … the downstream cost of additional rejected hits. … evaluating fixed thresholds across workload shifts would further clarify deployment robustness.”

Following the reviewer’s suggestion, we added an end-to-end experiment that includes the generation cost of rejected hits. We run real LMArena traffic through GPTCache with LRU eviction, and every miss and every rejected hit triggers an LLM call.

Table R10. End-to-end costs in the GPTCache experiment.

Cost componentMeasurement
Requests served from the cache94% → 92%
LLM calls per 1K requests62.6 → 79.5
Profile construction17–40 ms/entry
Serving-time check0.3–1.6 ms/hit
Profile storage15–108 kB/entry

Our defense lowers the share of requests served from the cache by two points, from 94% to 92%. This drop is the generation cost of rejected hits. Profile construction and the serving-time check take only milliseconds and need no LLM call.

We also added the fixed-threshold test that the reviewer suggested, on three real-traffic workloads (LMArena, Search, and Classification). We keep the benign hits of each workload in arrival order and cut them into blocks of 500 or 2,000 hits. The thresholds are fit on one block and tested on the next block. Each test therefore uses traffic that arrives after calibration. FPR stays between 4.1% and 6.2%. A new workload needs only one refit, which takes under 0.4 s and uses benign hits alone.

We have added these results to §5.2 and Appendices E and F of the revised paper, and we are glad to clarify any remaining point.

References

[1] Yilun Xu et al. A Theory of Usable Information under Computational Constraints. ICLR, 2020.

Thanks for the additional experiment results and I will keep my positive ratings.

We appreciate your follow-up and the careful review. We are happy to discuss any point further.

Reviewer_QsyROpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

The paper studies poisoning in LLM semantic caches, where an attacker induces a response that is subsequently reused for a different, benign query. It proposes Deletion Gain, which measures whether removing parts of a cached query increases its similarity to the incoming query, combined with an answer-consistency check. The authors evaluate three attack families, several embedding models, adaptive attacks, and a GPTCache implementation. They report substantial reductions in attack success and low verification latency after precomputation.

Critical strengths

S1. Deletion Gain provides an intuitive signal for detecting instructions that change the required answer while having a limited effect on embedding similarity (§4.2). Precomputing the variants makes the serving-time check inexpensive, which is relevant for a cache intended to avoid expensive inference

S2. The experimental analysis includes adaptive attacks, benign-query controls, answer-aware baselines, and uncertainty estimates (Table 2; Appendices E.1–E.4). These provide useful evidence about the detector under the evaluated conditions.

Critical weaknesses

W1. The evaluation does not establish effectiveness under realistic cache conditions. The main attack success rate requires a poisoned cache entry to pass the similarity threshold and the defense, but does not require successful insertion or retrieval ahead of competing benign entries (§D.5). These conditions matter because an existing benign entry can prevent either step. The single GPTCache replay (Table 16a) provides limited evidence and does not systematically evaluate eviction, adaptive cache policies, or static/dynamic cache architectures. Table 14 uses vCache’s datasets with a fixed threshold, rather than evaluating vCache’s policy. Consequently, the reported reductions in attack success do not establish how much protection the defense provides in deployment. Along those lines, distribution shifts during inference are a concern.

W2. The calibration and verification baselines leave the reliability–cost trade-off unclear. Despite manual calibration to a 5% FP rate, adding harmless queries increases FP rates to 5.7–12.9% (Table 12), triggering additional LLM calls for legitimate requests. Meanwhile, the answer-aware DeepSeek judge blocks no attacks at the target false-positive rate (Table 5), raising concerns about its configuration as a verification baseline. It also lacks comparisons with asynchronous or admission-time verification. The low verification latency in Figure 3 and Table 16b is encouraging, but excludes insertion-time computation, additional storage, and the inference cost of rejected legitimate cache hits. These results therefore do not establish the defense’s overall cost advantage.

Questions for authors

Q1. Attack admission and retrieval (clarification). Does the main eval insert poisoned cache entries through the normal query interface, or evaluate prepared entries directly? What fraction of attacks fail because an existing benign entry prevents insertion or is retrieved instead? Please clarify how these conditions affect the attack success rates in §D.5 and Table 16a.

Q2. Cache policies (supporting evidence). What supports the defense’s applicability to adaptive cache policies such as vCache [1, §§4.1–4.3] and static/dynamic cache architectures such as Krites [2, §2.2]? In these settings, what cache state must the attacker know, which entries can the attacker influence, and how do eviction policies affect attack success?

Q3. Threshold calibration (clarification). How would the defense’s thresholds be calibrated and maintained in deployment? In particular, what explains the increase from a nominal 5% false-positive rate to 5.7–12.9% under harmless query additions (Table 12), and what evidence supports reliable calibration under changing traffic? Please connect this discussion to the threshold limitations identified in §3.2.

Q4. Similarity metric (clarification). Which definitions and results require cosine similarity, and which apply to other similarity metrics? Sections 3.1–3.2 appear to present a general formulation of semantic caching, although the retrieval rule and subsequent analysis are cosine-specific. Please clarify the intended scope. I recommend a generalizable formalization beyond cosine similarity.

Q5. Verification baseline (clarification and supporting evidence). Why does the answer-aware DeepSeek judge block no attacks at the target false-positive rate (Table 5)? How was its rubric validated on legitimate cache hits? What evidence supports an advantage over a well-configured answer-aware judge, including asynchronous verification as discussed in vCache [1, §6] and implemented by Krites [2, §3]?

Q6. End-to-end cost (supporting evidence). How does the cost comparison change when including insertion-time computation, additional storage, and LLM calls caused by rejected legitimate cache hits? In particular, what is the effect of the per-entry storage overhead in Table 16b under a fixed cache-memory budget and realistic eviction?

Q7. Legitimate query variation (clarification). Legitimate users may submit similar queries with different constraints or instructions. What distinguishes the proposed poisoning signals from this ordinary variation? Please clarify whether the defense identifies poisoned cache entries or invalid cache hits, and explain how the benign controls in Tables 12 and 14 capture naturally occurring differences between legitimate queries.

Q8. Distribution shift (clarification). How robust are the offline-calibrated thresholds to distribution shift? Table 12 shows elevated false-positive rates under harmless query additions, while Table 14 recalibrates for each workload. How would the system detect and respond to drift during deployment?

[1] Luis Gaspar Schroeder et al. vCache: Verified Semantic Prompt Caching. ICLR, 2026. arXiv:2502.03771.

[2] Asmit Kumar Singh et al. Asynchronous Verified Semantic Caching for Tiered LLM Architectures. 2026. arXiv:2602.13165.

5 comments
AuthorsOpenReview ↗

Response to Reviewer QsyR (1/3)

We thank the reviewer for the detailed and constructive review. To address the questions on realistic cache conditions, we evaluated our defense end to end in GPTCache with eviction, in vCache [1], and in a Krites-style tiered cache [2]. It remains effective in all three, with ASR below 2.4%.

Q1/W1. “Does the main eval insert poisoned cache entries through the normal query interface, or evaluate prepared entries directly? What fraction of attacks fail because an existing benign entry prevents insertion or is retrieved instead? …”

Our evaluation covers both cache admission and retrieval. The main evaluation (§D.5) uses prepared entries directly. It assumes that the attacker-crafted entries are already in the cache, which gives the attacker the greatest opportunity to succeed. Table 16a of the submission adds competition from benign entries during retrieval. The new GPTCache experiment (Table R4) sends every attack key through the normal query interface and includes LRU eviction, so it covers the entire process from admission to retrieval.

Table R4. End-to-end attacks in GPTCache on LMArena traffic.

Cache conditionInsertedASR without defenseASR with our defense
Attacker first97.9%13.8%0.9%
Benign first37.9%0.9%0.0%
Benign first, 5 attack keys61.6%1.2%0.0%
After eviction99.7%11.9%0.2%

When the benign entry is cached first, 62.1% of single attack keys hit it and are never inserted, so the attack rarely succeeds even without the defense. Eviction restores the attack, and an attacker can force eviction with dummy queries [3].

Our defense keeps ASR at or below 0.9% in all four conditions. Existing benign entries limit the attacker’s opportunities, and our defense remains effective when eviction removes this protection.

Q2. “What supports the defense’s applicability to adaptive cache policies such as vCache … and static/dynamic cache architectures such as Krites …? … what cache state must the attacker know, which entries can the attacker influence, and how do eviction policies affect attack success?”

We evaluated our defense under the vCache policy and in a Krites-style static/dynamic cache. ASR with our defense stays below 2.4% in both settings (Table R5).

Table R5. ASR under vCache and a Krites-style tiered cache.

CacheAttackASR without defenseASR with our defense
vCache (δ = 0.02)CAP, feedback15.0%0.9%
vCache (δ = 0.01–0.05)SCP, feedback18.3–46.0%0.7–2.3%
Krites-style tiersCAP, attacker first13.8%0.9%

vCache explores an entry before exploiting it, so a single attack key does not succeed. In the feedback attack, the attacker sends five additional attack keys. vCache then treats the reuse of the poisoned entry as correct and begins to exploit it. Our defense checks each exploit decision and provides a negative label when it rejects a hit.

Under Krites, the attacker can write only to the dynamic tier (i.e., the online cache that stores answers at run time), since the static tier holds curated answers. Our defense checks dynamic hits, and static hits and asynchronous promotion work as before

In both evaluations, the attacker has no knowledge of the cache state. It only sends queries, and these queries insert its entries. The feedback attack also observes whether a request hits the cache, which response latency reveals. Eviction raises attack success, since it removes the benign entries that block insertion.

References

[1] Luis Gaspar Schroeder et al. vCache: Verified Semantic Prompt Caching. ICLR, 2026. arXiv:2502.03771.

[2] Asmit Kumar Singh et al. Asynchronous Verified Semantic Caching for Tiered LLM Architectures. arXiv:2602.13165, 2026.

[3] Guanlong Wu et al. When Cache Poisoning Meets LLM Systems: Semantic Cache Poisoning and Its Countermeasures. NDSS, 2026.

AuthorsOpenReview ↗

Response to Reviewer QsyR (2/3)

Q3/W2. “How would the defense’s thresholds be calibrated and maintained in deployment? … what explains the increase from a nominal 5% false-positive rate to 5.7–12.9% under harmless query additions (Table 12) … reliable calibration under changing traffic? … the threshold limitations identified in §3.2.”

The thresholds are calibrated automatically and maintained by a periodic refit. Each threshold is a quantile of scores on a few hundred benign hits from the target deployment, and calibration uses no attack data. The refit repeats this step on recent benign hits.

The increase in Table 12 comes from a kind of key that calibration never saw. Calibration uses benign keys that are plain questions. The test then applies the thresholds to keys that carry a harmless addition, such as “Please tell me.” Such an addition raises DG, so DG flags more benign hits. The Answer Check removes most of these false rejections. A cosine threshold fit on the same hits exceeds its budget by more on template and LLM-written additions.

Under changing traffic, calibration stays reliable. On three real-traffic workloads, we fit the thresholds on one block of 500 or 2,000 consecutive benign hits and test them on the following block. FPR stays between 4.1% and 6.2%.

§3.2 shows that a threshold on similarity cannot separate valid from invalid hits, since the two groups overlap in similarity. Our thresholds act on DG, ADL, and Echo, where the two groups separate (Figure 5), so a calibrated threshold can tell them apart.

Q4. “Which definitions and results require cosine similarity, and which apply to other similarity metrics? … I recommend a generalizable formalization beyond cosine similarity.”

We thank the reviewer for this recommendation and have adopted it. The definitions in §3–4 now use a general similarity function sim, and the recoverability bound holds for any L-Lipschitz sim. Our experiments use cosine similarity. On unit-normalized embeddings, the dot product and the negative squared Euclidean distance give the same decisions as cosine similarity after recalibration.

Q5/W2. “Why does the answer-aware DeepSeek judge block no attacks at the target false-positive rate (Table 5)? How was its rubric validated on legitimate cache hits? What evidence supports an advantage over a well-configured answer-aware judge … asynchronous or admission-time verification?”

The answer-aware DeepSeek judge blocks no attacks at the 5% budget because it also flags many benign entries. With the stored answer in view, it flags ordinary errors of the LLM that involve no attacker. These flags cover about 30% of benign ComQA entries, so no threshold on its score stays within the budget.

We validated the rubric by manually reviewing a sample of the legitimate hits that the judge flags. The flagged answers do contain such errors, so the judge applies its rubric as intended.

We also configured a validity judge that asks only whether the stored answer correctly and completely answers the incoming query. It is the strongest answer-aware judge we tested. Poisoned CAP and SCP answers read like ordinary answers, so this judge accepts most of them (Table R6).

Table R6. Block rate at the same 5% budget.

MethodCAPSCPKCA
Qwen3-8B validity judge0.3040.2371.000
Ours0.8200.9560.982

We then compared where verification runs, assuming a perfect LLM judge that takes 2 s per hit.

Table R7. Synchronous and asynchronous verification with a perfect LLM judge.

StrategyASRAdded serving latencyVerifier calls
Synchronous verification0.0%1.9 s/requestOne per hit
Asynchronous verification11.9%NoneOne per hit
Ours0.9%0.3–1.6 ms/hit0

Synchronous verification blocks every poisoned hit, and each hit waits about 2 s for the LLM judge. Asynchronous verification adds no latency. The cache serves the poisoned answer to the user before the judge finishes, so the attack is detected only afterwards. Admission-time verification checks an entry when it is inserted. A poisoned answer is consistent with its own key and becomes invalid only relative to a later query, which does not exist yet at insertion. Our check runs when the hit occurs, so it sees the incoming query. It adds about a millisecond.

AuthorsOpenReview ↗

Response to Reviewer QsyR (3/3)

Q6/W2. “How does the cost comparison change when including insertion-time computation, additional storage, and LLM calls caused by rejected legitimate cache hits? … the per-entry storage overhead in Table 16b under a fixed cache-memory budget and realistic eviction?”

We measured insertion-time computation, storage, and the LLM calls caused by rejected hits. We run real LMArena traffic through GPTCache with LRU eviction, and every cache miss and every rejected hit triggers an LLM call. Table R8 reports the results.

Table R8. Measured deployment costs.

Cost componentMeasurement
Requests served from the cache94% → 92%
LLM calls per 1K requests62.6 → 79.5
Profile construction17–40 ms/entry
Serving-time check0.3–1.6 ms/hit
Profile storage15–108 kB/entry

With our defense, the cache still answers 92% of requests directly, compared with 94% without it. The two-point difference comes from rejected hits, which the LLM answers again. A profile is built once, when its entry is inserted, and this takes a small fraction of the time needed to generate one answer.

The profiles are stored on SSD and take no cache memory. The in-memory index therefore stays the same size, and GPTCache evicts the same entries as before. Saving the profile vectors as 8-bit integers cuts this storage in half, and the block rate changes by at most 0.5 points.

Q7. “… What distinguishes the proposed poisoning signals from this ordinary variation? … whether the defense identifies poisoned cache entries or invalid cache hits … how the benign controls in Tables 12 and 14 capture naturally occurring differences between legitimate queries.”

A poisoned hit differs from ordinary variation in its stored answer. Our defense rejects a hit only when two conditions hold. First, the key contains extra content that the incoming query lacks (DG). Second, the stored answer was shaped by that extra content (Answer Check). A legitimate key with an extra instruction usually meets only the first condition, so the hit is accepted. A poisoned key meets both, because its extra content exists to change the answer. The decision depends on both the cache key and the incoming query, so the defense identifies invalid cache hits. The table below lists the four cases.

Added content in the keyAnswer does not reflect itAnswer reflects it
Legitimate instructionAcceptReject, and the LLM answers as on a miss
Malicious residualAccept, since the answer is rarely poisonedReject

The benign controls cover two kinds of legitimate variation. The first is natural rewording. Table 14 pairs real user prompts that the benchmarks label as equivalent, and its FPR stays near the 5% budget. The second is an added instruction, such as “Please tell me.” Table 12 adds such instructions to the key, which is the first row of the table above, and our defense accepts most of these hits.

Q8. “How robust are the offline-calibrated thresholds to distribution shift? … How would the system detect and respond to drift during deployment?”

The thresholds stay robust on later traffic. Thresholds fit on 500 or 2,000 consecutive benign hits keep FPR near the 5% budget on the hits that arrive next (Q3). To detect drift, the system tracks the false-rejection rate on a small sample of hits that are labeled offline, and vCache’s exploration step already produces such labels. When this rate moves away from the budget, the system refits the thresholds on recent benign hits, which takes under 0.4 s.

We have updated §3–5 and Appendices E–F of the revised paper with the results above. We hope these results address the reviewer’s concerns, and we are glad to clarify any remaining points.

Reviewer_v1oGOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a lightweight defense combining Deletion Gain (DG) and Answer Check. DG detects suspicious content by measuring whether deleting parts of a cached query increases its similarity to the user query. Answer Check further uses Answer Deletion Loss (ADL) and lexical overlap (Echo) to determine whether the deleted content affects the cached answer. Experiments across multiple embedding models, poisoning attacks, and real-world workloads demonstrate strong detection performance with minimal serving-time overhead.

Critical strengths
  1. The proposed method combines Deletion Gain (DG) with Answer Check (ADL and Echo) to identify suspicious cache hits. The design is intuitive and lightweight, avoiding additional LLM inference during serving. Precomputing detection-related information at cache insertion time enables very low runtime overhead.
  2. The authors evaluate the proposed defense against three poisoning attack categories, including the newly introduced CAP attack, as well as several adaptive attacks. Experiments across four embedding models, different FPR operating points, cross-dataset settings, and real-world workloads provide substantial evidence of the method's effectiveness and generalizability.
Critical weaknesses
  1. Although the paper evaluates DG alone and the complete method, it does not independently ablate ADL and Echo. In particular, experiments comparing DG+ADL, DG+Echo are missing. Consequently, it remains unclear how much each component individually contributes to detection performance and whether both are necessary.
  2. The proposed defense relies on empirically calibrated thresholds η and η_A, which may need to be recalibrated when a different embedding model or deployment environment is used, since the distributions of DG and ADL scores can vary across models and data distributions. However, the paper does not report the time or computational overhead required for this recalibration process.
Questions for authors
  1. Could the authors provide additional ablation results for DG+ADL and DG+Echo, respectively, alongside DG-only and the full DG+ADL+Echo configuration? Reporting their detection performance under the same FPR budget would help clarify the individual contribution of ADL and Echo, and demonstrate whether both components are necessary for improving the overall detection performance.
  2. Could the authors report the time and computational overhead required to calibrate the thresholds η and η_A, particularly when switching to different embedding models or deployment environments? It would also be helpful to show how the calibration cost scales with the number of benign calibration samples, providing a clearer understanding of the practical overhead of deploying and recalibrating the proposed defense.
3 comments
AuthorsOpenReview ↗

Response to Reviewer v1oG

We thank the reviewer for the constructive suggestions.

W1/Q1. “Could the authors provide additional ablation results for DG+ADL and DG+Echo … clarify the individual contribution of ADL and Echo, and demonstrate whether both components are necessary …”

We ran the requested ablation with all four configurations. The results show that both components are necessary, and each has a distinct role (Tables R1 and R2).

Table R1. Detection on the three attack classes (BR ↑ / ASR ↓).

ConfigurationCAPSCPKCA
DG only.895 / .004.977 / .013.969 / .029
DG + ADL.589 / .049.543 / .332.920 / .071
DG + Echo.783 / .021.950 / .030.976 / .014
Full (DG + ADL + Echo).820 / .006.956 / .026.982 / .014

Table R2. FPR (%) ↓ on benign hits whose cache key carries a harmless addition.

AdditionDG onlyDG + ADLDG + EchoFull
Template47.7–72.40.6–2.111.1–12.811.1–12.9
LLM-written50.4–72.10.5–1.47.6–9.67.6–9.6
Same query60.9–94.30.0–0.15.5–6.45.7–6.4

Within the Answer Check, Echo and ADL complement each other. Echo detects answers that repeat words from the residual, and it provides most of the detection. Without Echo, the block rate on CAP and SCP drops sharply. ADL detects answers that follow the residual in meaning while using different words, so it blocks poisoned entries that Echo misses. Adding ADL lowers the CAP attack success rate further, with almost no change in false positives.

W2/Q2. “Could the authors report the time and computational overhead required to calibrate the thresholds η and η_A, particularly when switching to different embedding models or deployment environments? … how the calibration cost scales with the number of benign calibration samples …”

We measured the calibration cost as requested. Calibration takes under 0.4 s for up to 5,000 benign hits. A few hundred hits already keep FPR near the 5% budget (Appendix E). Calibration computes DG and ADL for each benign hit from embeddings that the cache already stores. As Table R3 shows, calibration time grows linearly with the number of benign hits. A new deployment environment needs one such calibration on its own benign hits. A new embedding model also needs the stored embeddings recomputed once, and this step and the calibration together take under five minutes on 16 CPU threads.

Table R3. Calibration time (seconds, one CPU core) vs. number of benign hits.

Workload2505001,0002,5005,000
LMArena0.0110.0230.0470.1170.232
Search0.0060.0120.0240.0610.122
Classification0.0190.0370.0740.1860.371

We have added both experiments to §5.2 and Appendix E of the revised paper.

Thanks to the authors for adding the new experiments; I will keep my original positive score.

Thank you for your time and suggestions. We welcome any further questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank all reviewers for the careful reviews and constructive feedback. We are glad that the reviewers find the problem timely and practical (Reviewers n68W, D5zH), find the defense intuitive and lightweight (Reviewers v1oG, QsyR), and see useful evidence of its effectiveness in the evaluation with adaptive attacks (Reviewers v1oG, QsyR, n68W). We also thank Reviewer D5zH for the interest in the evasion–effectiveness trade-off, which led to a new analysis in Section 5.3. We have revised the paper according to the reviewers' suggestions, with all changes marked in green. The main changes are listed below.

  • Realistic cache conditions (Section 5.2, Appendix F). We ran the attacks end to end on real LMArena traffic in GPTCache with LRU eviction, under the vCache policy, and in a Krites-style tiered cache. Our defense keeps ASR below 2.4% in all three. The same appendix reports the cost of insertion, storage, and the LLM calls caused by rejected hits, and it compares our defense with synchronous and asynchronous LLM verification.
  • Calibration (Sections 5.1 and 5.2, Appendix E.6). Calibration needs a few hundred benign hits and takes under 0.4 s, and thresholds fit on one block of traffic remain valid on the next block. We also state the evaluation labels and calibration assumptions explicitly, including the difference between benign queries and valid cache hits.
  • More LLMs, tasks, and attacks (Sections 5.2 and 5.3, Appendices E.7 to E.9). We repeated the evaluation with six additional LLMs, including GPT-5.6-Sol and Claude Opus 4.8, and added embedded-answer attacks and three non-QA tasks (code generation, recommendation, and explanation). On these six LLMs, ASR on CAP, SCP, and KCA stays below 2.6%, and the evasion–effectiveness trade-off holds on each of them.
  • Ablation and baseline (Section 5.4, Appendices E.1 and E.5). A new ablation with DG + ADL and DG + Echo shows that both components are needed. We also added a validity judge as a stronger answer-aware baseline, and it accepts most fluent poisoned answers.
  • Formulation (Sections 3.1, 3.3, and 4.3, Appendix E.3). The definitions now use a general similarity function, and the recoverability bound holds for any L-Lipschitz similarity. The information-bottleneck analysis now states its claim as usable information and estimates it with five predictor families.

We are happy to answer any remaining questions, and we look forward to your reply. Thank you again for your time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.