- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies safety-aware semantic response caching and argues that query similarity alone is insufficient for determining whether a previously cached response remains safe to reuse. It proposes SafeCache, which separates candidate retrieval from cache approval. For each cached response, SafeCache represents response-side reuse requirements and compares them against safety-relevant facts extracted from the new request and context. It combines an aggregate condition-mismatch score with explicit contradiction checks and a matching-uncertainty signal, and uses a calibrated router to choose among direct reuse, selective verification, and regeneration subject to an empirical unsafe-reuse target and verifier budget.
The evaluation uses 3,225 cache–probe pairs constructed from several dialogue, tool-use, temporal, refusal, authorization, and semantic-collision settings, with repeated cluster-disjoint development/calibration/test splits. SafeCache achieves substantially lower false-hit rates than the evaluated baselines at competitive valid cache-hit rates, shows favorable ablations, and remains relatively stable under controlled extraction perturbations and alternative verifiers.
Critical strengths
- The paper identifies an important limitation of similarity-only cache reuse and provides a coherent safety-oriented formulation.
- The paper goes beyond query-level similarity by explicitly incorporating response-dependent safety conditions and request context into cache reuse decisions.
- The paper provides a thorough empirical evaluation, including comparisons with multiple baselines, component-level ablation studies, robustness analysis under condition-extraction errors, alternative verifier experiments, and cost analysis.
Critical weaknesses
- Although the paper compares SafeCache with several existing methods, some baselines are not evaluated with their complete mechanisms. For example, the vCache baseline only uses fixed similarity thresholds, while its original online adaptive threshold mechanism is excluded. This raises concerns about whether the reported improvements accurately reflect SafeCache's advantages over existing approaches.
- The paper reports the overall relative inference cost, but does not provide a breakdown across different datasets or task categories. Since these workloads may differ substantially in context length, verification complexity, and regeneration cost, could the authors report dataset-specific efficiency results? Such an analysis would help clarify whether SafeCache provides consistent efficiency benefits across different application scenarios.
Questions for authors
- A recent concurrent work, LaCache (Liang et al., 2026, https://arxiv.org/abs/2608.01718), also addresses unsafe semantic cache reuse, particularly cache-collision attacks, through response-prefix validation. Could the authors clarify the differences between SafeCache and LaCache and, if feasible, provide a comparison under the same attack settings?
- The paper reports overall relative inference costs but does not provide dataset-specific cost analyses. Could the authors report the cost results separately for different datasets or task categories, including verifier call rates, regeneration rates, and relative inference costs?
I will raise the score if all questions are well solved.
W1: Static vCache and the complete online mechanism
The vCache result evaluates its static threshold rule with state reset for each cache/probe pair. We will label it
vCache-staticin Table 1 and restrict the associated comparison to that rule. The revised abstract, contribution statement, and conclusion report the evaluated routing tradeoffs and remove the claim of superiority over complete online cache systems.Added baseline input check. The submitted vCache-static rows already used the saved MiniLM query cosine, independently of SafeCache's affected loader. Recomputing its three static thresholds reproduces the submitted results:
The three GPTCache and three Cache-Me settings also reproduce their submitted rates with these query similarities and frozen filter verdicts. All nine checked settings therefore retain their reported baseline values.
Online vCache uses persistent state and generation feedback to update per-entry observations. Our static comparison evaluates the supplied-pair decision, and the shared-verifier controls evaluate routing contribution with common inputs and budgets. We will make those comparison objects explicit throughout.
W2: Efficiency across datasets
The source breakdown identifies which workloads benefit under the cost model. SGD, tau-bench, SituatedQA, and the seed set save generation calls. XSTest regenerates every request and incurs local checking overhead. Q2 reports their verification, regeneration, and modeled costs.
The added source-wise cost analysis also reports each workload's break-even verifier/generation ratio in Q2. At nominal generation cost 1,000 ms and measured local overhead 4.563 ms, each workload saves cost when its verifier/generation ratio falls below the listed threshold.
Q1: LaCache and a common attack comparison
LaCache validates generated-prefix agreement, while SafeCache checks support for the cached response's extracted requirements.
This difference matters when a resource, permission, evidence item, or workflow state changes. SafeCache can use those explicit conditions before generating a prefix. We will position LaCache as concurrent related work and explain this difference in the evidence used for reuse approval.
We verified this mechanism against LaCache, arXiv:2608.01718v2. A common attack run requires prefix generation with a specified backend or draft model, prefix length, and hashing configuration. The existing replay stores verifier decisions, while that comparison requires newly generated prefixes. We will include the mechanism comparison and identify each paper's attack results by its own setting.
Q2: Per-dataset call rates and costs
Added source-wise cost analysis. The table reports the corrected safety-first policy over 13,911 test exposures. Verification and regeneration rates use all requests as the denominator. A verified rejection incurs both calls. Modeled relative cost is 4.563/1,000+0.06054*v+b, using measured local overhead and nominal model-call costs.
Here gamma is the verifier/generation cost ratio. Below its break-even gamma, a source saves cost under the model. XSTest always regenerates and adds local overhead. The source exposure counts are SGD 6,689, tau-bench 2,746, SituatedQA 1,275, XSTest 1,745, and seed 1,456.
These are pooled source rates, while Table 1 averages over splits. For source g, the cost ratio is O/C_G,g+v_g*(C_V,g/C_G,g)+b_g. The table uses common nominal per-call costs to compare routing demand. An aggregate serving-cost ratio with different per-source call costs uses summed method costs divided by summed No-cache costs.