- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes VPCache for repeated-visual LVLM inference. It constructs reusable compressed visual representations and selects among cache views based on question type and answer-preservation criteria, with fallback to a safer base representation when compression is not certified. The method is evaluated on seven visual QA workloads using Qwen2.5-VL-7B-Instruct, showing small accuracy gains over the guarded base while reducing the average injected visual-token count.
Critical strengths
-
The paper identifies a practically relevant problem: preserving answer quality when reusing compressed visual representations across repeated queries.
-
The proposed framework is carefully designed around safe fallback and paired answer preservation, which makes the system more reliable than applying a fixed compression strategy uniformly.
Critical weaknesses
1. Limited methodological novelty
The methodological contribution appears limited relative to the existing literature on visual-token/cache compression and query-aware selection. The proposed APVC largely combines familiar ingredients—pooling/salient-token based initialization, attention-based feature aggregation, residual refinement, and query-conditioned slot selection—while AP-CVS adds a validation-based routing and fallback mechanism over a bank of existing or learned compressed views. Although this combination is reasonable and the answer-preservation criterion is useful, it is unclear that the paper introduces a sufficiently new compression or caching principle beyond composing existing techniques with an empirical routing rule. This substantially weakens the paper's technical contribution.
2. Insufficient experimental evidence for the claimed efficiency and generality
The experimental evaluation does not convincingly establish that VPCache is practically efficient or broadly applicable.
-
Limited baselines: The main comparisons use Fresh/Exact reuse, TokenPool, and KIVI-style/SnapKV-style proxies rather than strong implementations of several directly relevant recent visual-token and VLM cache-compression methods discussed in the paper itself. This makes it difficult to assess the proposed method against the actual state of the art.
-
Limited model generality: All main experiments use only Qwen2.5-VL-7B-Instruct. Since the proposed framework is presented as a general LVLM inference method, evaluation across different LVLM architectures is important to determine whether the observed behavior is model-specific.
-
Limited workload scale and diversity: The evaluation uses only 448 held-out requests (64 per workload), with a request-level rather than image-disjoint split. Moreover, the repeated-visual setting contains only about 2.95 questions per visual input on average. There is no video or large multi-frame benchmark, where visual context and cache size would be substantially larger and the claimed benefits of reusable compression would be more meaningful.
-
Modest effective compression: Although individual compressed paths can reduce the representation substantially, only 44 of 448 held-out requests actually use a shorter visual prefix. Consequently, the workload-level reduction is only 4.90% in injected visual tokens and 2.46% in selected memory slots. These reductions are quite small relative to the paper's efficiency motivation.
-
Missing end-to-end efficiency measurements: The paper does not provide controlled end-to-end latency, throughput, or serving-cost comparisons for the final method. Representation/token reduction alone does not establish actual system-level efficiency, particularly because routing, slot selection, and maintaining multiple candidate cache views can introduce additional overhead.
-
Incomplete memory analysis: The only peak-memory experiment is a small auxiliary diagnostic that predates the final APVC-hifi method. Moreover, its reported on-disk cache footprint provides no storage advantage and is even larger than Exact L1 caching in that diagnostic. Thus, the claimed memory benefit is not demonstrated for the final system.
-
No compression-cost/amortization analysis: APVC requires constructing a compressed memory for each visual input, but the cost of this construction is not quantified. For a repeated-query setting, the key quantity is the break-even point at which the one-time compression cost is amortized by subsequent queries. No FLOPs, construction latency, or efficiency-versus-number-of-queries analysis is provided.
Questions for authors
-
What is the key methodological novelty of VPCache beyond combining existing compression, query-aware selection, and validation-based routing/fallback mechanisms?
-
How does VPCache compare with stronger and more directly related visual-token and VLM cache-compression methods under the same backbone, compression budget, and evaluation setting, using faithful implementations rather than proxy variants?
-
Does VPCache generalize to LVLM backbones other than Qwen2.5-VL-7B-Instruct?
-
Would the conclusions remain consistent when evaluated on larger benchmark subsets, image-disjoint splits, and the official metrics of each benchmark?
-
How does VPCache scale as the visual context becomes larger and the number of repeated queries per visual input increases, including multi-image or video settings?
-
Given that compression is applied to only a limited fraction of requests, how does the quality–compression trade-off change when targeting substantially higher compression coverage or more aggressive compression ratios?
-
What are the end-to-end latency and throughput improvements of the final VPCache system under controlled and identical serving conditions?
-
What are the actual GPU-memory and cache-storage savings of the final APVC-hifi system, including all coexisting cache views and fallback representations?
-
What is the computational cost of constructing the compressed cache, and after how many repeated queries is this one-time cost amortized relative to exact prefix reuse or fresh inference?
2 comments
Part 2 of 2: video contexts, coverage, latency, memory and amortization
Q5. Distinct questions and 32-frame contexts
Reuse reduces repeated visual work on distinct questions. On the archived 65-question / six-image natural-reuse trace, VPCache measures 206.93 ms versus Fresh 310.55 ms, a 33.37% reduction, with 8–16 distinct questions per image.
The expanded video diagnostic evaluates 64 additional clips and 256 questions, sampling 32 distinct frames per case. The image-trained checkpoint and policy are fixed.
| 32-frame method | Correct / 256 | Mean latency, s | Peak host feature tensors, MiB |
|---|---|---|---|
| Fresh | 202 | 2.448 | 0 |
| Exact | 202 | 1.869 | 1047.59 |
| Base | 200 | 1.916 | 1717.38 |
| APVC | 199 | 1.926 | 1717.38 |
APVC reduces total visual tokens by 29.25% at 55.08% request coverage, with an observed 21.35% latency reduction relative to Fresh, although it was trained only on images. APVC−base accuracy is −0.39 points [−1.17, 0.00]; APVC−Exact is −1.17 [−3.52, 1.17], using paired clip-cluster intervals. Fresh and Exact predictions match verbatim.
Q6. Bank capacity and selective compression
Bank capacity and injected prefix are separate operating-point controls. On the archived 141-request stress fixture, hifi APVC scores 49.20, the smaller-bank checkpoint 46.39, and the shorter-injection candidate 37.34. The hifi point is 2.81 macro points above the smaller-bank configuration and 11.86 above the shorter injection. The bank configurations use their respective fixed checkpoints; the injection comparison reuses the hifi checkpoint. Guarded base on that fixture is 61.38 and includes its eligibility/fallback decisions.
Coverage is an explicit policy setting. The frozen Qwen selector compresses 547/3,080 archived requests and 241/1,434 additional requests, reducing total visual tokens by 9.05% and 8.73%, respectively. Qualifying against Exact changes the additional-test operating point to 30 compressed requests. The 0/0.2/0.5/1/2-point mean-calibration-loss margins all select 125 compressed requests, at 8.72% coverage and 4.43% token reduction, so the selection is stable across this range; the grid was fixed before test scores were inspected.
Q7. Complete online latency and throughput
The 6.82% latency and 7.32% throughput improvements over guarded base include the online selection path. The archived measurements use 389 distinct requests, two randomized-order repetitions, and batch-one sequential execution. Preprocessing, routing, construction, cache access, decoding, and fallback are charged. Mean generated lengths are 5.14 tokens for base and 5.08 for VPCache.
Mean latency is 192.99 ms for VPCache, 207.10 ms for guarded base, 187.15 ms for Exact, and 225.02 ms for Fresh; FoLDER's archived mean is 182.54 ms. The paired VPCache−base difference is −14.12 ms [−19.81, −8.41].
Q8. Physical cache, shared features, and deployment assets
Shared L1 removes duplicate visual-encoder work, while a common LRU policy enforces the declared feature-storage capacity. In the complete first repetition, shared VPCache reduces native vision-encoder calls from 173 to 157. The unrestricted cache remains 328.18 MiB because its representations coexist.
| Physical feature-cache policy | Exact, MiB | VPCache, MiB |
|---|---|---|
| Unrestricted | 273.26 | 328.18 |
| Common 160 MiB LRU, final stored cache | 158.81 | 158.00 |
| Common 160 MiB LRU, peak before eviction | 162.14 | 162.11 |
Both methods remain within 160 MiB after each request, and capacity enforcement leaves all predictions unchanged. Under the same capacity, VPCache stores slightly less than Exact.
Archived allocated GPU peaks are 15.585 GiB for base/Exact and 15.678 GiB for VPCache; host memory is 3.208 GiB for base and 3.213 GiB for VPCache. The APVC checkpoint is a separate 98.15 MiB deployment file.
Q9. Construction cost under repeated-question reuse
Reuse distributes construction cost across subsequent questions about the same visual input. Cold APVC construction, including first-use loading, is 186.7–236.9 ms and is charged when the bank is first materialized. The outcome-independent request trajectories show the following measured costs:
| Requests per image | Fresh, ms | Exact, ms | VPCache, ms |
|---|---|---|---|
| R=1 | 281.37 | 277.67 | 274.72 |
| R=8 | 274.88 | 194.52 | 196.44 |
| R=32 | 269.87 | 182.79 | 184.11 |
At R=32, the VPCache and Exact points are 184.11 and 182.79 ms. For the independently calibrated selector, VPCache is already faster than Fresh from the first or second request: crossings against Fresh/Exact occur at R=2/32 on binary visual questions and R=1/1 on the other three standard-resolution categories. The separate 65-question natural-reuse result confirms the saving on distinct questions.
These measurements anchor the contribution in three linked choices: learning reusable visual evidence, qualifying the view for each question, and charging its complete online cost.
Part 1 of 2: contribution, FastV/FoLDER comparison, InternVL and image-disjoint test
Thank you for the questions about technical contribution, cross-architecture behavior, and serving efficiency. VPCache connects a reusable visual bank to question-dependent view selection and QA-based qualification; each experiment below evaluates one part of that design.
Q1. A reusable bank with independently controlled views
VPCache separates reusable capacity, per-question extraction, and eligibility. APVC constructs an image-dependent, question-independent bank; the hifi point retains approximately 0.75N slots and injects approximately 0.5N tokens for an eligible question. Different questions select views from that shared bank. AP-CVS compares paired generated-answer scores on calibration data and freezes a dataset × question-class routing table. Online selection reads the question and table.
This decomposition makes the contribution testable: the fixed-budget control tests representation choice; three seeds test bank learning; independent-versus-transferred routing tests qualification; the serving trace charges construction and fallback. A full-budget interface control additionally reproduces all 147 Exact answers when native spatial grids are retained, confirming that the bank interface leaves uncompressed features unchanged.
Q2. Common-prompt comparison with FastV and FoLDER
Conditional view selection retains document evidence at this operating point: VPCache scores 80.42 DocVQA and 64.94 InfoVQA, compared with FastV's 69.34/44.46 and FoLDER's 68.63/45.87. The differences to FastV are +11.08 and +20.48 points.
All methods use the same 1,434 questions, checkpoint, dataset templates, BF16, greedy decoding, a 262K-pixel input limit, and a 32-token output cap. VPCache compresses 241 requests (16.81% coverage, 8.73% total-token reduction); these FastV/FoLDER settings compress 100% of requests. The table specifies the resulting quality–coverage choices, including Exact's uncompressed reference. Under the conditional VQAv2 rule, the scores are VPCache 78.48, Exact 81.48, guarded base 78.48, guarded APVC 79.11, FastV 79.65, and FoLDER 75.56.
Q3. Qualification transfers across LVLM interfaces
The same qualification interface selects architecture-specific operators on InternVL2.5-2B. With frozen seed-0 weights and 448×448 single-tile preprocessing, our Exact path reproduces the native InternVL inference output on one calibration case from each benchmark. A matched 128-question VQAv2 prompt control yields 1.28 words per answer versus 5.59, with uniformly normalized scores of 68.67 versus 27.50.
The Exact-qualified table selects compressed base in TextVQA/binary and VQAv2/count, and Exact elsewhere. It compresses 42/1,434 requests and reduces total tokens by 1.46%.
On InternVL, qualification selects the base and Exact views that preserve this architecture's answers, so the QA-based qualification mechanism transfers across interfaces. The APVC learning comparison is reported separately at matched candidate budgets.
Q4. A further image-disjoint benchmark test
The additional test contributes 1,434 questions on 521 further images beyond the 3,080-question earlier test. Its images are checked at the pixel level against the fitting, qualification, and test partitions, and all methods are scored on the identical question set.
Whole image groups are selected independently of outcomes. Benchmark-specific scoring and paired image-cluster resampling preserve both the dataset's metric and dependence among questions about one image. A 485-question conservative ChartQA source-exclusion diagnostic is reported as a separate control.