acceptodds
Under review as a conference paper at ICLR 2027

Beyond Query Similarity: Safety-Conditioned Cache Routing for LLM Response Reuse

Abstract

Semantic response caching reuses a cached response for a semantically similar query to improve LLM inference efficiency. However, query similarity alone can lead to unsafe reuse when safety conditions change. Existing approaches improve cache safety through stricter similarity checks, context-aware validation, or response verification, yet they may still miss safety conflicts between the cached response and the new request or introduce unnecessary verification overhead. We argue that safe and efficient semantic caching requires a reuse policy that explicitly represents the conditions required by a cached response and checks whether they remain satisfied by the new request. In this paper, we propose SafeCache, a safety-aware framework that separates candidate retrieval from safety approval and formulates semantic response reuse as a safety-conditioned routing problem. SafeCache represents cached-response reuse requirements and new-request safety facts as two safety condition sets and aggregates condition-level matches into an overall compatibility score. To prevent decisive conflicts from being diluted by aggregate matching, SafeCache checks explicit contradictions separately. SafeCache further estimates matching uncertainty and combines these safety signals with query similarity} to make the final reuse decision under safety and verifier-budget constraints. Experiments show that SafeCache achieves a better safety-utility trade-off among the evaluated baselines, enabling safer response reuse at comparable cache utility.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM, Cache, Safety

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_xKmdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies safety-aware semantic response caching and argues that query similarity alone is insufficient for determining whether a previously cached response remains safe to reuse. It proposes SafeCache, which separates candidate retrieval from cache approval. For each cached response, SafeCache represents response-side reuse requirements and compares them against safety-relevant facts extracted from the new request and context. It combines an aggregate condition-mismatch score with explicit contradiction checks and a matching-uncertainty signal, and uses a calibrated router to choose among direct reuse, selective verification, and regeneration subject to an empirical unsafe-reuse target and verifier budget.

The evaluation uses 3,225 cache–probe pairs constructed from several dialogue, tool-use, temporal, refusal, authorization, and semantic-collision settings, with repeated cluster-disjoint development/calibration/test splits. SafeCache achieves substantially lower false-hit rates than the evaluated baselines at competitive valid cache-hit rates, shows favorable ablations, and remains relatively stable under controlled extraction perturbations and alternative verifiers.

Critical strengths
  1. The paper identifies an important limitation of similarity-only cache reuse and provides a coherent safety-oriented formulation.
  2. The paper goes beyond query-level similarity by explicitly incorporating response-dependent safety conditions and request context into cache reuse decisions.
  3. The paper provides a thorough empirical evaluation, including comparisons with multiple baselines, component-level ablation studies, robustness analysis under condition-extraction errors, alternative verifier experiments, and cost analysis.
Critical weaknesses
  1. Although the paper compares SafeCache with several existing methods, some baselines are not evaluated with their complete mechanisms. For example, the vCache baseline only uses fixed similarity thresholds, while its original online adaptive threshold mechanism is excluded. This raises concerns about whether the reported improvements accurately reflect SafeCache's advantages over existing approaches.
  2. The paper reports the overall relative inference cost, but does not provide a breakdown across different datasets or task categories. Since these workloads may differ substantially in context length, verification complexity, and regeneration cost, could the authors report dataset-specific efficiency results? Such an analysis would help clarify whether SafeCache provides consistent efficiency benefits across different application scenarios.
Questions for authors
  1. A recent concurrent work, LaCache (Liang et al., 2026, https://arxiv.org/abs/2608.01718), also addresses unsafe semantic cache reuse, particularly cache-collision attacks, through response-prefix validation. Could the authors clarify the differences between SafeCache and LaCache and, if feasible, provide a comparison under the same attack settings?
  2. The paper reports overall relative inference costs but does not provide dataset-specific cost analyses. Could the authors report the cost results separately for different datasets or task categories, including verifier call rates, regeneration rates, and relative inference costs?

I will raise the score if all questions are well solved.

1 comment

W1: Static vCache and the complete online mechanism

The vCache result evaluates its static threshold rule with state reset for each cache/probe pair. We will label it vCache-static in Table 1 and restrict the associated comparison to that rule. The revised abstract, contribution statement, and conclusion report the evaluated routing tradeoffs and remove the claim of superiority over complete online cache systems.

Added baseline input check. The submitted vCache-static rows already used the saved MiniLM query cosine, independently of SafeCache's affected loader. Recomputing its three static thresholds reproduces the submitted results:

vCache-static thresholdReproduced VCHR (%)Reproduced FH (%)
.9538.5576.041
.9838.4561.887
.9938.4560.815

The three GPTCache and three Cache-Me settings also reproduce their submitted rates with these query similarities and frozen filter verdicts. All nine checked settings therefore retain their reported baseline values.

Online vCache uses persistent state and generation feedback to update per-entry observations. Our static comparison evaluates the supplied-pair decision, and the shared-verifier controls evaluate routing contribution with common inputs and budgets. We will make those comparison objects explicit throughout.

W2: Efficiency across datasets

The source breakdown identifies which workloads benefit under the cost model. SGD, tau-bench, SituatedQA, and the seed set save generation calls. XSTest regenerates every request and incurs local checking overhead. Q2 reports their verification, regeneration, and modeled costs.

The added source-wise cost analysis also reports each workload's break-even verifier/generation ratio in Q2. At nominal generation cost 1,000 ms and measured local overhead 4.563 ms, each workload saves cost when its verifier/generation ratio falls below the listed threshold.

Q1: LaCache and a common attack comparison

LaCache validates generated-prefix agreement, while SafeCache checks support for the cached response's extracted requirements.

Decision aspectLaCacheSafeCache
Evidence for reuseGenerated prefix agrees with cached prefix under hash validationExtracted response requirements are supported by the current request
Additional generation before approvalPrefix generation, optimized with a draft model and speculative decodingDirect approval from conditions, or selective verification
Decision outcomeReuse after prefix validation, otherwise generateReuse, verify, or regenerate under a call budget

This difference matters when a resource, permission, evidence item, or workflow state changes. SafeCache can use those explicit conditions before generating a prefix. We will position LaCache as concurrent related work and explain this difference in the evidence used for reuse approval.

We verified this mechanism against LaCache, arXiv:2608.01718v2. A common attack run requires prefix generation with a specified backend or draft model, prefix length, and hashing configuration. The existing replay stores verifier decisions, while that comparison requires newly generated prefixes. We will include the mechanism comparison and identify each paper's attack results by its own setting.

Q2: Per-dataset call rates and costs

Added source-wise cost analysis. The table reports the corrected safety-first policy over 13,911 test exposures. Verification and regeneration rates use all requests as the denominator. A verified rejection incurs both calls. Modeled relative cost is 4.563/1,000+0.06054*v+b, using measured local overhead and nominal model-call costs.

SourceVerifier (%)Regeneration (%)VCHR (%)Modeled cost / No-cacheBreak-even gamma
SGD25.5562.2737.640.64271.459
tau-bench7.3985.7214.200.86631.869
SituatedQA2.6771.5327.840.721510.505
XSTest0.00100.000.001.0046None
Balanced seed4.3392.317.690.93031.672

Here gamma is the verifier/generation cost ratio. Below its break-even gamma, a source saves cost under the model. XSTest always regenerates and adds local overhead. The source exposure counts are SGD 6,689, tau-bench 2,746, SituatedQA 1,275, XSTest 1,745, and seed 1,456.

These are pooled source rates, while Table 1 averages over splits. For source g, the cost ratio is O/C_G,g+v_g*(C_V,g/C_G,g)+b_g. The table uses common nominal per-call costs to compare routing demand. An aggregate serving-cost ratio with different per-source call costs uses summed method costs divided by summed No-cache costs.

Reviewer_g4ogOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies when a cached LLM response can safely be reused for a semantically similar request. The proposed SafeCache framework extracts reuse requirements from the cached response and safety-relevant facts from the new request. It combines condition mismatch, explicit contradiction checks, and matching uncertainty to route candidates to direct reuse, verification, or regeneration. Routing thresholds are selected on calibration data under empirical unsafe-reuse and verifier-budget constraints. The evaluation uses 3225 cache–probe pairs constructed from several existing datasets and additional scenarios, together with a CacheAttack stress test. SafeCache reports lower false-hit rates than the evaluated baselines at competitive valid cache-hit rates. The paper also provides component ablations and analyses of extraction perturbations, calibration, and alternative verifiers.

Critical strengths
  1. The paper addresses a meaningful limitation of semantic caching. Similar queries can require different responses when authorization, dialogue state, or supporting evidence changes. Separating retrieval relevance from response-reuse validity is a sensible design choice. Explicitly representing response-side requirements also makes the approval process more inspectable than a query-similarity threshold alone.

  2. The evaluation includes useful checks beyond the headline results. The cluster-disjoint splits, separate risk-map estimation and policy selection, extraction perturbations, and alternative-verifier replay strengthen the empirical analysis.

Critical weaknesses
  1. The baseline comparisons do not fully control for available information and verification resources. Appendix G shows that vCache uses static thresholds without online adaptation, while Cache-Me’s filter does not receive request context. SafeCache has access to richer inputs and a full-input verifier. A comparison using the same inputs, verifier, and budget is needed to attribute the gains to the proposed method.

  2. The added value of the full routing design remains unclear. Fig 2 shows that verification recovers over 13 percentage points of valid cache hits, making it a substantial contributor to performance. However, the paper does not compare against simpler similarity-based or rules-based selective-verification routers under matched budgets. The existing ablations retain condition modeling throughout and therefore do not establish whether the full framework is necessary.

  3. Generalization beyond the constructed scenarios is insufficiently demonstrated. The evaluation uses source-derived or deterministically transformed pairs, while the extraction rules cover closely corresponding condition changes. Cluster-disjoint splits and record-level perturbations are useful, but do not test unfamiliar expressions, unseen transformation patterns, or implicit requirements. The results therefore support effectiveness within the modeled setting more strongly than broader applicability.

Questions for authors
  1. Could the authors compare SafeCache with simpler similarity-based and rules-based selective-verification routers using the same full-input verifier, calibration splits, and verifier budget? This would help establish which gains require the proposed matching and uncertainty mechanisms.

  2. How were the scenario transformations and extraction rules developed relative to one another? Could the authors clarify whether any transformation templates or condition families were held out during rule development, and provide evidence of performance on such held-out cases?

  3. Could the authors provide worked examples showing how response-specific requirements are extracted, particularly when a requirement is implicit? Could the authors also specify the candidate threshold grid referenced in Section 3.4.3 but apparently missing from Section 4.1?

  4. How was the verifier-to-regeneration cost ratio of 0.0605 in Appendix M obtained? Could the authors report measurements for a specified verifier/backend configuration or show how the efficiency advantage changes with this ratio?

3 comments

W1: Same inputs, verifier, and available budget

We added similarity-band and literal-rule routers with the same raw input availability, MiniLM query cosine, full-input verifier, calibration partitions, risk target, arrival order, and verification cap. Each family selects its thresholds on calibration labels. The baseline labels distinguish vCache-static and the query-only Cache-Me filter.

At B=0.30, corrected SafeCache obtains VCHR/FH=33.416%/0.524%, compared with 30.114%/1.346% for similarity-band routing. Actual verification rates are 24.677% and 6.580%. Q1 reports both the rates and the budget sweep. W2 examines candidate selection in a subset with similar actual checking expenditure.

W2: What the full routing design adds

The extra routing signals change which responses are checked and reused. Added comparison under similar actual verification expenditure: on 8,667 test exposures where the query or context changes, the frozen policies give:

RouterValid returnsInvalid returnsVerifier calls
Similarity band8168326
SafeCache26422323

The policies were selected on the full calibration partitions before this subset analysis. These pooled counts show a candidate-selection benefit within the evaluated condition families at similar checking expenditure.

Added routing-signal diagnostics on all 20 test splits (13,911 exposures). Keeping the balanced parameters fixed at alpha=0 and B=0.30 and removing one signal at a time gives:

Signal configurationVCHR (%)FH (%)
Full policy33.4160.524
Remove contradiction guard34.0281.154
Remove matching-margin contribution37.0511.030

VCHR is valid returns divided by all requests, and FH is invalid returns divided by cache returns. Rates average over the 20 splits. Both removals increase reuse and invalid returns. These are fixed-parameter checks of the signals' effects on the decision rule.

We also repeated the Figure 2 intervention after correction. Replacing the balanced policy's executed verifications with regeneration lowers VCHR from 33.416% to 10.025%. Verification thus recovers 23.391 percentage points. The shared-verifier controls address routing contribution, and this intervention addresses the benefit of checking its selected region.

W3: Generalization beyond constructed scenarios

The evaluation tests the represented explicit condition families under cluster holdout. Appendix C specifies these categories, and Appendix D tests downstream robustness by perturbing extracted records. Cluster holdout measures performance on new source clusters, while record perturbations measure how extraction errors propagate through matching and routing.

We will state this scope consistently in the abstract and conclusion. Performance on unfamiliar expressions, unseen transformation templates, and implicit requirements remains unmeasured. Q2 gives the documented rule-development history, and Q3 shows the actual outputs for explicit and implicit requirements.

Q1: Similarity and rules routers with shared verification

We ran similarity-based and literal-rule selective-verification controls on all 20 splits. All use the same full-input verifier, MiniLM query cosine, calibration risk target, and hard verification cap. The simple controls omit SafeCache's condition representation, matching score, and uncertainty. Added controls at alpha=0 and B=0.30:

RouterVCHR (%)FH (%)Verifier calls (%)Invalid / cache returns
Similarity band + verifier30.1141.3466.58072/4,065
Literal key/value support + verifier27.0590.6807.27737/3,652
Key/value support and similarity band + verifier29.7231.19611.67462/4,006
SafeCache33.4160.52424.67728/4,643

Rates average over splits, while counts pool test exposures. At this point, SafeCache returns more valid responses and has a lower FH, with more verification. The similarity router reuses above a high threshold and verifies an intermediate band. The literal-rule control permits direct reuse when its nonempty cached key/value set is supported by the current raw input. Its remaining candidates can enter verification above a calibrated retrieval threshold.

Added verification-budget sweep. We evaluated seven caps at alpha=0. Representative points show the resulting tradeoff:

Verification capSafeCache VCHR / FH (%)Similarity VCHR / FH (%)Actual calls: SafeCache / similarity (%)
4%13.500 / 0.17028.928 / 1.1433.547 / 3.955
8%16.859 / 0.20431.516 / 2.5066.925 / 7.518
16%24.181 / 0.37931.484 / 2.92614.605 / 8.547
30%33.416 / 0.52430.114 / 1.34624.677 / 6.580

The numerical supplement includes all seven caps. At the lower two points, actual call rates are close and SafeCache trades some valid reuse for lower FH. At B=0.30 it gains valid reuse while spending more calls. The 323-versus-326-call diagnostic in W2 further isolates candidate selection on changed inputs.

Q2: Rule development and template holdout

The extractor is byte-identical to its June 26, 2026 Git checkpoint and was fixed before the V2 scale-up. The 360-pair development seed already contained all six condition families, with 60 pairs per family. We found no contemporaneous record of a template or family being held out during rule development.

The documented protocol therefore supports cluster holdout within these families. Training-only risk fitting and calibration-only policy selection remain separate from test evaluation, as described in Appendix B. We will describe this history directly and distinguish it from a template-development holdout.

Q3: Implicit requirements and the missing threshold grid

The frozen extractor processes the cached query, response, and context, and separately the new query and context. The following worked outputs come from three schematic examples run through that extractor. Each tuple lists (kind, subject, predicate, value).

Example inputActual extracted record
Cached context role: manager(authorization, authorization, role, manager)
New context role: guest(authorization, authorization, role, guest)
Cached response “Please confirm the recipient before proceeding.”(response_action, assistant, should, verify)
Query “What is the next step?” with no response on the new side(response_action, assistant, should, answer)
Response “The payment can proceed.” with a supervisor-sign-off policyAn answer action and the retained policy text in a context_fact record

The confirmation cue identifies the response action. The recipient-specific prerequisite remains unrepresented. In the payment example, the context text is retained without a dedicated approval-requirement record. These outputs show what the deterministic representation captures when a requirement is implicit.

The grid reference at the end of Section 3.4.3 is missing from Section 4.1. The frozen initial family has 140 listed combinations with the following marginal values.

ParameterValues
Direct similarity.86, .90
Direct risk.12, .18, .24, .28
Direct uncertainty.70, .85, 1.00
Verification similarity.50, .56, .68, .74, .80
Conflict verification similarity.86, .90
Verification risk.35, .45, .55, .65, .85
Verification uncertainty.75, .85, 1.00
Risk floor.18
Uncertainty floor.45, .65

The initial search covers the listed 140 configurations. An initial calibration search at alpha=0 and B=0.20 selects the direct-reuse parameters. Verification boundaries then use the compatible frozen configurations and calibration quantiles at levels {0, .10, .25, .50, .75, .90, .95, .975, 1} for risk and uncertainty. After deduplication, the corrected replay has 325 to 973 candidates per split. The supplement files frozen_routing_candidates.csv, candidate_recipes.json, and selections.json give the exact configurations, per-split numerical recipes, and selected policies.

Q4: Origin of 0.0605 and cost sensitivity

The ratio is 60.54 ms for an earlier local checker proxy divided by a nominal 1,000 ms generation cost. This is the parameter used in the call-cost model. Appendix M will state its source alongside the primary verifier configuration.

Added cost sensitivity analysis. Using corrected call rates and newly measured local overhead O=4.563 ms, we varied gamma=C_V/C_G while keeping the decisions fixed. Relative cost is O/C_G+v*gamma+b, where b is the regeneration rate and C_G=1,000 ms is nominal.

gammaSafety-firstBalancedUtility-oriented
0.060540.7700.6830.675
0.250.7980.7300.724
0.500.8350.7920.789
1.000.9080.9150.919
2.001.0541.1621.180

The break-even ratios are 1.632, 1.343, and 1.309. Savings depend on verification and generation costs. In particular, all three settings exceed No-cache cost at gamma=2. This sensitivity analysis answers the cost-ratio question without treating the original proxy as an API measurement.

Reviewer_pw2GOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SafeCache, a safety-aware framework that separates candidate retrieval from safety approval and formulates semantic response reuse as a safety-conditioned routing problem. The paper argues that to balance safety and utility, a safety-conditioned semantic cache needs a routing policy that makes the cached response's safety conditions explicit. A safety-conditioned router decides whether the cached response should be reused, verified, or regenerated.

Critical strengths
  • The paper is trying to solve a timely topic. Semantic caching is widely deployed to cut LLM serving cost, and the paper correctly identifies that query similarity alone does not guarantee a cached response is still safe when conditions such as authorization, tool state, or freshness change.

  • I like the idea that rather than treating verification as a generic step after retrieval, SafeCache routes each candidate to direct reuse, verification, or regeneration based on explicit signals such as condition mismatch, matching uncertainty, and a separate contradiction guard that cannot be averaged away.

Critical weaknesses
  1. Verification is just another LLM which is not deterministic and the study does not evaluate how good the verifier is. Methods used in the comparison does not have safety mechanism built into them that is why HR is 100% for all of them (Table. 4). Using such methords for comparison does not seem reasonable.
  2. The paper does not define what is "unsafe" here in this study and does not study the harmful effect of "unsafe" reuse. The reference used in line 053 for safety does not even use the word "unsafe" (Yan et al., 2025).
  3. Every request requires cache-side and request-side condition extraction, pairwise condition matching, rule checks, and possibly a verifier call. Since the goal of semantic caching is to reduce latency and cost, the paper should report the absolute per-request overhead and the break-even cache hit rate, not only cost relative to No-cache.
  4. The gains are reported as relative improvements (“nearly 49%” safer than vCache, “about 95.4%” safer than Cache-Me), which can exaggerate small absolute differences in false-hit rates that are already low.
  5. Each component is a standard technique: max-similarity condition matching (Eq. 1), a top-2 margin for match confidence, monotonic risk calibration, Wilson intervals, and a lexicographic grid search over thresholds (Eq. 9). The contribution is mainly the way these pieces are combined, and the paper does not establish a new principle for when cached responses can be safely reused.
Questions for authors
  1. Table 2 shows that without "Risk-Constrained Selection" VCHR is higher but FH is lower. Why "Risk-Constrained Selection" kept in the study is VCHR more important than FH?
2 comments

W1: Verifier quality and a comparison with safety mechanisms

Appendix J.2, Table 14 evaluates the verifier against reuse-validity labels. We have expanded the analysis and added attack comparisons with a shared full-input verifier. Appendix F specifies the five input fields and accepts only VALID. Frozen verdicts make the replay repeatable, while temperature zero specifies the original decoding setting.

Added verifier error analysis. The corrected safety-first replay gives:

Reference reuse labelVerifier acceptsVerifier rejects
Valid1,94538
Invalid1016

These are 2,009 repeated test exposures. The verifier accepts 10/26 invalid candidates (38.46%), and 10/1,955 of its accepted responses are invalid (0.512%). The latter proportion reflects candidate admission as well as verifier quality. Across the full pool, the same verifier accepts 391 invalid candidates. The full safety-first policy also makes six invalid direct reuses.

Added attack comparison with the same verifier. At alpha=0 and B=0.30, we replayed all 20 calibration-selected policies on the same 504 stress pairs.

Router with shared full-input verifierAttack hits (%)Verifier calls (%)
Similarity band0.0007.163
Literal key/value support0.4862.996
SafeCache0.0007.708

Rates average over the 20 policies, and attack hits use all stress requests as the denominator. Their benign reuse and checking costs are reported in g4og Q1. The submitted Table 4 also contains GPTCache rates below 100% and a Cache-Me setting at 0%.

W2: Meaning of unsafe reuse and harmful effects

We define reuse validity by whether the cached response remains correct and appropriate for the new request and context. The aggregate measured failure is invalid reuse. Security or policy violations form one category, alongside correctness failures caused by changed tool state, evidence, or time. We will use these terms consistently in the definition and reported metrics.

For example, an authorization change can make a previously appropriate disclosure violate the current permission scope. A changed tool result can make an old completion message false. An outdated factual answer violates the new request's time condition. The benchmark labels these inapplicable cached returns. Downstream harm depends on how an application uses them.

We will also revise line 053. ContextCache supports the discussion of context-dependent caching. The stronger statement about an internally unsafe response under unchanged context will be replaced by the response-validity definition and examples above.

W3: Absolute overhead and break-even reuse

Added latency microbenchmark. The corrected local pipeline takes a mean of 4.563 ms per candidate on an i9-14900K under WSL2 with Python 3.12.3.

Local operationMean time (ms)
Condition extraction0.192
Lexical vectors, matching, risk and conflict checks4.370
Routing decision0.001
Total4.563

Total median and 95th-percentile times are 3.209 and 11.463 ms. We measured 3,225 pairs over five passes after one warmup pass. Both cached-side and request-side work are charged, although cached-side extraction and vectors can be precomputed.

For local overhead O, verification rate v, final cache-reuse rate h, and per-call costs C_V and C_G in a common unit,

Ccache=O+vCV+(1−h)CG,hbreak−even=O+vCVCG.C_{\mathrm{cache}}=O+vC_V+(1-h)C_G, \qquad h_{\mathrm{break-even}}=\frac{O+vC_V}{C_G}.

With the corrected safety-first decisions, v=14.605% and h=24.293%. At nominal C_G=1,000 ms and the original checker proxy C_V=60.54 ms, average checking overhead is 13.405 ms and break-even reuse is 1.340%. Modeled total cost is 0.770 times No-cache. The 4.563 ms is measured local work. The call costs are modeling assumptions, with shared retrieval and model inference outside the microbenchmark. The sensitivity analysis in g4og Q4 shows when the modeled saving disappears.

W4: Absolute differences behind relative improvements

We will replace the relative-improvement headlines with the following absolute comparison after the implementation correction.

Method and settingVCHR (%)FH (%)Invalid / cache returns
SafeCache, utility-oriented34.3090.68939/4,777
vCache-static, threshold .9938.4560.81544/5,279
Cache-Me, threshold .8540.8799.015548/6,101

Compared with vCache-static, SafeCache has 4.147 percentage points fewer valid hits and 0.126 points fewer false hits. Against Cache-Me, it has 6.570 points fewer valid hits and 8.326 points fewer false hits. Rates average over splits, while counts pool 13,911 repeated test exposures. The revised interpretation is this observed reuse/error tradeoff.

W5: Contribution beyond standard components

SafeCache contributes a response-specific approval rule: identify the requirements of the cached response, check their support in the current request, and let a decisive contradiction block direct reuse even when the aggregate match is strong. Ambiguous support guides selective verification. This gives the router an explicit decision about the particular response's resource, permission, evidence, or workflow dependencies.

The standard matching and calibration tools implement that rule. At the corrected balanced point, removing the contradiction guard from the fixed policy raises FH from 0.524% to 1.154%. Removing the matching-margin contribution raises it to 1.030%. Both removals increase reuse and invalid returns, showing how the signals affect the reuse/error tradeoff.

ContextCache checks query and context compatibility. LaCache validates a generated response prefix. SafeCache checks extracted response requirements before additional generation and uses them to choose direct reuse, verification, or regeneration. The revised contribution statement will emphasize this approval policy and the controlled routing evidence.

Q1: Safety takes priority over valid-hit maximization

The safety target and verification budget are applied first. Valid hits are maximized among the policies that satisfy both calibration constraints, as defined in Eqs. 8 and 9.

In the submitted Table 2, removing risk-constrained selection changes VCHR from 40.159% to 40.854% and FH from 0.368% to 0.417%. FH increases by 0.049 percentage points. Section 4.3, lines 430 and 431, describes this same direction.

Added selector ablation after correction. We use the same candidate family and B=0.30, removing only the calibration risk filter.

Policy selectionVCHR (%)FH (%)Invalid / cache returns
With calibration risk filter33.4160.52428/4,643
Without calibration risk filter34.3090.68939/4,777

The constraint retains the lower-error operating point at a cost of 0.893 percentage points in valid reuse. Rates average over splits, while counts pool test exposures. As Appendix E states, alpha is an empirical calibration target. At alpha=0 the selected policy has zero observed calibration errors, and its test errors are reported separately.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.