- Rating
- 2
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
LPA-CACHE profiles GPU latency plateaus and uses their boundaries to choose visual KV reuse budgets. It protects tokens using previous-step attention, ranks the remaining tokens by inter-frame embedding changes, and extracts attention from only a few layers to retain FlashAttention compatibility.
Critical strengths
-
The paper provides clear evidence that token reduction and latency reduction are not proportional. As shown in Table 1, reducing the active length from 257 to 256 lowers reported latency from 75.23 to 69.05 ms without changing success. This is a concrete and useful systems observation.
-
Evaluation covers three VLA models and two simulation benchmarks. Reporting observation-to-action latency, including reuse overhead, is more informative than reporting FLOPs or prefill savings alone.
Critical weaknesses
-
The contribution is primarily an engineering refinement of existing KV reuse. Hardware profiling and boundary-aligned budgets are useful, but the paper offers limited new understanding beyond this combination. The resulting observation-to-action speedups are roughly 1.10–1.15× over FlashAttention baselines, and evaluation on one GPU leaves the broader applicability of the central hardware claim uncertain.
-
The experiments do not cleanly isolate plateau alignment. Compared with VLA-Cache, the method changes token ranking, relevance probing, reuse scheduling, and FlashAttention support. The boundary experiment demonstrates the latency phenomenon, but does not establish how much of the overall improvement specifically requires the proposed scheduling strategy.
Questions for authors
-
With identical token selection, probe layers, and attention implementations, how much additional benefit comes from plateau-aware scheduling?
-
Do the gains persist on another GPU, and how sensitive are they to the empirically selected reuse-entry and probe layers?