- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
ReState accelerates frozen BERT rerankers through approximate cross-query reuse of document K/V states. It combines position correction with selective document updates, while preserving attention to all document tokens. An optional variant uses shallow query interactions to refine update selection. Across six ranking benchmarks, the default configuration reports a 6.857× mean speedup while retaining 96.0% of Full’s mean nDCG@10, under prepared-cache conditions with candidate routing. Additional experiments examine checkpoint transfer and downstream QA.
Critical strengths
- The method addresses a substantive obstacle to document-state reuse in bidirectional rerankers: both query interactions and absolute positions change document representations. Combining position correction with selective updates offers a useful training-free solution without changing the checkpoint or removing document tokens from attention.
- The controlled experiments separate positional errors, update selection, and candidate routing. Without routing, ReState still achieves a reported 4.041× speedup at 0.474 nDCG@10 versus Full’s 0.488 (Table 4), supporting a benefit from reuse itself. Full-restoration equivalence and transfer to additional BERT checkpoints further strengthen the validation.
Critical weaknesses
- The main comparison uses FP32 eager Full, whereas ReState uses fused Triton attention (Appendix A.8). The SDPA control in Table 9 evaluates 100% restoration, rather than the default 20% configuration. Consequently, the acceleration attributable to reuse over an optimized implementation of the same reranker remains uncertain.
Questions for authors
- What speedups do the default and no-routing configurations achieve against a tuned dense SDPA implementation, using the same checkpoint, precision, inputs, batching, and timing boundaries?
3 comments
Thank you for your responses and clarifications. The rebuttal and discussion have addressed most of my concerns, and I will raise my score. I appreciate your efforts to address my feedback.
Your in-depth review of this paper is greatly appreciated. We are also thankful for your acknowledgement of this work.
We thank the reviewer for the careful reading and constructive feedback.
Following your suggestion, we compared ReState at 20% restoration with a tuned dense Full-SDPA baseline, and ReState still delivers substantial speedups over this stronger baseline. Specifically, using the same settings, ReState achieves a 6.50× speedup with default routing and 3.57× without routing. Because both methods score all 100 candidates in the no-routing setting, the latter speedup is attributable to reuse alone.
Settings.