- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies prefix-cache reuse in multi-tenant RAG serving. It orders retrieved documents by historical access frequency to encourage shared prefixes across requests, with optional Weighted Prefix Affinity routing. The experiments show substantial cache-hit improvements over unordered prompts, smaller gains over cache-dependent ordering, and limitations involving answer quality and deployment topology.
Critical strengths
Frequency ordering is a simple and inexpensive idea with practical appeal. The comparisons against fixed random and content-hash ordering provide useful evidence that frequency alignment drives the gains. Past-only controls also support the mechanism without requiring access to future requests.
I appreciate the reporting of negative quality and throughput results. Cache-hit rates alone would give an overly optimistic picture. These measurements help identify where the method may be unsuitable and show that WPA is not beneficial under every deployment configuration.
Critical weaknesses
The evaluation does not yet establish a convincing setting where the method improves serving efficiency while preserving useful answer quality. On the evaluated NQ subset, reordering the same documents reduces F1 from 42.5% to 9.9%, and no tested mitigation reaches the stated joint quality–cache target. This does not imply degradation on all RAG tasks, but leaves the practical benefit unresolved. Missing per-question artifacts also prevent auditing this unusually large drop. The throughput penalty from WPA under isolated two-GPU serving further limits the full system’s benefit.
The additional contribution over existing ordering methods appears modest. ContextPilot and CacheWeaver already study closely related context-ordering mechanisms. The main change here is replacing cache-state information with a historical frequency prior, while the matched held-out advantage is about one percentage point and narrows with longer execution. The paper acknowledges these limitations, but still needs a clearer account of when this incremental benefit is consequential.
Questions for authors
-
Can the quality experiment be rebuilt with preserved per-question artifacts? Does the large drop persist with explicit original-rank annotations comparable to ContextPilot?
-
With matched historical information, cache budgets, and request traces, what sustained latency or throughput advantage remains over a faithful cache-dependent ordering implementation?
-
Is there an evaluated workload showing both serving improvements and acceptable answer quality, including the simpler frequency-plus-round-robin configuration?