- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
PACE-VL is a training-free method for compressing the prefill KV cache of vision-language models. It retains exact evidence KV pairs selected using task-field queries, assigns residual tokens to original-key anchors, and stores aggregate values and token counts for count-corrected decoding. The compressed prompt cache has an exact slot budget, while generated tokens are appended separately. Evaluation covers three VLMs and five benchmarks, with component ablations and eager-backend efficiency measurements.
Critical strengths
-
Consistent quality preservation under the evaluated compression protocol. At the nominal 10% budget, Table 1 reports the best score among the compared compressed implementations in all 21 settings, with a median margin of 3.06 points over the strongest compressed competitor. PACE-VL matches or exceeds the corresponding full-cache score in 11 settings and trails by at most 1.87 points otherwise. These results suggest practical potential.
-
A coherent construction with relevant controls. The exact-slot construction and count-corrected representation are clearly specified. Table 7 includes equal-slot controls for residual aggregation, multiplicity, and anchor choices, while Table 4 reports generation latency and decoding throughput. These checks help assess both the mechanism and its practical costs.
Critical weaknesses
-
Insufficient differentiation from prior clustered and merged KV-cache methods. KVMerger already preserves important KV entries while merging the remainder, and CentroidKV uses clustered representations with count-corrected attention. Multipole combines exact attention with count-aware cluster approximations, although it retains the full original cache. PACE-VL’s main incremental choices appear to be task-field selection and original-key anchors, whose significance beyond these precedents is insufficiently established.
-
Unclear comparability of actual cache budgets. Nominal retention ratios can correspond to different realized cache sizes, but actual prompt-KV storage is not reported alongside the main quality comparisons. It therefore remains unclear whether PACE-VL’s advantage over the strongest baselines holds at comparable storage cost.
-
Limited evidence for the general benefit of task-field selection. In Table 7, task-field queries improve the macro-average over Tail32 by only 0.29 points, with the gain driven by GovReport; excluding this dataset, the full method scores 0.48 points lower. This comparison covers only one model and context length, leaving the broader benefit of task-field selection insufficiently established.
Questions for authors
-
Please clarify the main increments over KVMerger and CentroidKV. What does Table 7’s centroid-key control establish about the representative-key choice, and which differences from the prior algorithms does it not isolate? Please also distinguish PACE-VL’s compressed-storage objective from Multipole Attention’s full-cache setting.
-
For Table 1’s Qwen3-VL-8B/K32 comparison at the nominal 10% budget, could you report the realized prompt-KV footprint at the start of decoding, aggregated across layers and heads and including auxiliary state or retained KV buffers? Please clarify whether PACE-VL and HybridKV have comparable storage costs, and how any differences should qualify the interpretation of the reported quality gap.
-
For the GovReport setting in Table 7, please clarify which prompt tokens the task-field and Tail32 probes cover, and whether the query-source change also alters the evidence/anchor allocation. How do these details explain the large score difference, and what scope of benefit is supported by the task-dependent results already reported?