- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes ViFlow-KV, a training-free KV cache eviction method for multimodal reasoning. It combines evolving modality-demand allocation, which dynamically divides each head’s cache budget between visual and textual states, with visual-anchor importance estimation. Experiments cover six multimodal benchmarks, with additional evaluations across model scales and families. At a 2k token budget, the method retains 98% of the FullKV average score. Separate system experiments report up to 3.05x throughput improvement over FullKV.
Critical strengths
The paper addresses a relevant problem with a training-free design. Maintaining access to visual evidence throughout a long reasoning trajectory is important under constrained KV memory. The combination of evolving modality allocation and visually informed importance estimation provides a practical way to address this problem without retraining the model. ViFlow-KV outperforms the selected baselines across all four cache budgets, and the component ablations show that both EMDA and VAIE contribute.
Critical weaknesses
-
The main insight that visual and textual states require different retention policies is already present in prior work. MadaKV (ACL 2025)[1] explicitly allocates modality-specific budgets within each attention head in proportion to attention-derived modality importance, which is closely related to EMDA’s allocation rule. CSP[2] studies separate treatment of intra- and inter-modality attention, and SparseMM[3] allocates budgets according to visual head preferences.
-
The observation that previously unimportant visual evidence may become relevant again also motivates query-aware sparse retrieval that preserves historical KV entries, as in Quest[4], and learned dynamic selection, as in DSA[5]. It would be great to evaluate a non-evicting dynamic-selection setting or establish whether its modality-aware design remains beneficial there.
[1] https://arxiv.org/abs/2506.15724
[2] https://arxiv.org/abs/2412.04652
[3] https://arxiv.org/abs/2506.05344
Questions for authors
-
Beyond using a recent pre-RoPE query centroid and updating the allocation during generation, what distinguishes EMDA from the attention-proportional modality budgeting in MadaKV? Which aspects of the proposed allocation are intended as new contributions?
-
Can the authors compare against a strong modality-aware eviction baseline (e.g., MadaKV, SparseMM) under matched cache budgets and eviction schedules, and evaluate the additional benefit of VAIE on top of that baseline?
-
At matched accuracy, how does ViFlow-KV’s decoding efficiency compare with the eviction baselines, allowing each method to use the KV budget required to reach that accuracy? End-to-end latency or throughput comparisons that include eviction overhead would help establish whether the improved accuracy-budget trade-off translates into practical efficiency gains.
3 comments
Rebuttal by Authors (2/2)
Q2. [Can the authors compare against a strong modality-aware eviction baseline (e.g., MadaKV, SparseMM) under matched cache budgets and eviction schedules, and evaluate the additional benefit of VAIE on top of that baseline?]
A: Thank you. We conduct additional comparisons with MadaKV and SparseMM under matched KV cache budgets and eviction schedules. Furthermore, We integrate VAIE into both baselines to evaluate its additional benefits. We also report the average generated length to examine the impact of different KV retention strategies. The results are presented below:
KV Budget = 512
| Method | MMStar | MathVista | CharXiv RQ | Avg. Accuracy (%) | Avg. Generated Length |
|---|---|---|---|---|---|
| SparseMM | 61.27 | 62.60 | 26.30 | 50.06 | 7916 |
| SparseMM + VAIE | 62.13 | 64.90 | 28.80 | 51.94 | 7397 |
| MadaKV | 53.73 | 54.30 | 24.60 | 44.21 | 5164 |
| MadaKV + VAIE | 59.53 | 55.90 | 29.60 | 48.34 | 4768 |
| ViFlow-KV | 63.93 | 66.60 | 40.20 | 56.91 | 2623 |
KV Budget = 2048
| Method | MMStar | MathVista | CharXiv RQ | Avg. Accuracy (%) | Avg. Generated Length |
|---|---|---|---|---|---|
| SparseMM | 66.93 | 71.70 | 41.30 | 59.98 | 5421 |
| SparseMM + VAIE | 68.13 | 73.40 | 43.10 | 61.54 | 5211 |
| MadaKV | 66.47 | 70.10 | 45.00 | 60.52 | 3440 |
| MadaKV + VAIE | 67.73 | 73.20 | 47.70 | 62.88 | 3085 |
| ViFlow-KV | 71.47 | 78.00 | 53.30 | 67.59 | 2522 |
The results show that ViFlow-KV consistently outperforms MadaKV and SparseMM while generating substantially fewer reasoning tokens. By examining the reasoning trajectories, we find that these baselines allocate larger budgets to original visual tokens, leaving insufficient capacity for visually grounded textual states and leading to redundant visual verification. In contrast, ViFlow-KV better preserves both original visual evidence and its historical textual carriers, facilitating evidence reuse and reducing redundant reasoning. Furthermore, VAIE consistently improves both baselines, confirming the benefits of retaining visually grounded historical states. We will incorporate the above analysis in the revision.
Q3. [At matched accuracy, how does ViFlow-KV’s decoding efficiency compare with the eviction baselines, allowing each method to use the KV budget required to reach that accuracy?]
A: Increasing the KV budget would increase GPU memory consumption and limit batch size. In contrast, ViFlow-KV achieves better accuracy with only marginal additional eviction overhead, offering a more practical trade-off than retaining more KV states at a higher memory cost. To evaluate decoding efficiency at comparable accuracy, we compare ViFlow-KV with R-KV while allowing each method to use its own KV budget. The results are shown below:
| Method | Budget | MMStar | MathVista | CharXiv RQ | Avg. | Throughput (tok/s) |
|---|---|---|---|---|---|---|
| R-KV | 2048 | 70.67 | 76.00 | 50.70 | 65.79 | 1033.5 |
| R-KV | 2560 | 71.47 | 77.20 | 52.30 | 67.00 | 955.1 |
| ViFlow-KV | 2048 | 71.47 | 78.00 | 53.30 | 67.59 | 1007.8 |
The results show that ViFlow-KV achieves higher average accuracy with a smaller KV budget than R-KV. Notably, even when increasing its budget to 2560, R-KV still falls short of ViFlow-KV in average accuracy while incurring lower throughput. Further increasing the KV budget would incur more memory costs and further reduce efficiency. Overall, this better trade-off highlights the advantage of ViFlow-KV's visual-flow-guided retention strategy.
Looking Forward to Your Reply
Thank you once again for taking the time to review our paper and for providing such insightful and constructive feedback.
We have carefully considered each of your comments and have provided detailed responses. We sincerely hope that our efforts adequately address your concerns and contribute positively to your evaluation.
As the author-reviewer discussion period concludes on Nov 18 (AoE), we would greatly appreciate any further feedback you may have. If you have any additional questions or require any clarifications, please do not hesitate to reach out to us.
Rebuttal by Authors (1/2)
Thanks for your insightful comments and questions.
W1. [The main insight that visual and textual states require different retention policies is already present in prior work.]
A: Thank you. Our work provides the first in-depth characterization of KV cache dynamics in long-horizon multimodal reasoning, revealing how visual evidence propagates from original visual states into generated textual representations. This insight guides evolving modality-aware budget allocation and visual-flow-guided importance estimation, enabling ViFlow-KV to identify and preserve important visual and visually grounded textual states as reasoning progresses. Thus, our main contribution goes beyond simply assigning different retention policies/budgets to visual and textual states.
In contrast, MadaKV estimates static head-level modality preferences from the input context, SparseMM offline identifies visually relevant attention heads and allocates larger budgets, while CSP refines importance estimation by distinguishing intra- and inter-modality attention. However, they do not explicitly account for the growing importance of generated textual states that carry visual evidence during subsequent reasoning and reflection. We further provide additional comparisons with both methods in our response to Q2.
W2. [It would be great to evaluate a non-evicting dynamic-selection setting or establish whether its modality-aware design remains beneficial there.]
A: Thank you for the insightful question. Our work focuses on KV cache compression through permanent eviction, whereas Quest and DSA dynamically select KV entries for attention without permanently removing them. Due to their different optimization objectives, ViFlow-KV cannot be directly transferred to non-evicting retrieval settings. Given the orthogonal nature of the two approaches, we explore their combination by integrating ViFlow-KV into Quest, examining the compatibility of this modality-aware design with query-aware retrieval. The results are shown below:
The results demonstrate that ViFlow-KV can be effectively integrated into Quest, substantially reducing the KV retention budget while maintaining comparable performance. Furthermore, these results suggest that the insights from visual flow could extend beyond eviction-based compression, offering a general perspective for multimodal reasoning. We leave further exploration to future work.
Q1. [Beyond using a recent pre-RoPE query centroid and updating the allocation during generation, what distinguishes EMDA from the attention-proportional modality budgeting in MadaKV? Which aspects of the proposed allocation are intended as new contributions?]
A: Thanks. The pre-RoPE query centroid and continuous budget updates are precisely what make EMDA effective and distinctive. Our visual-flow analysis firstly reveals a gradual shift in reliance from original visual states to generated textual states during reasoning. Accordingly, the pre-RoPE query centroid accurately estimates current modality demands, while continuous budget updates dynamically track their evolution during reasoning. We provide fine-grained ablation studies of these design choices of EMDA in our response to Reviewer gEux's W1. In contrast, MadaKV allocates modality budgets based on attention mass from the initial prompt. However, it could not explicitly account for the decreasing reliance on original visual states and increasing reliance on generated textual states as reasoning progresses. We provide additional comparisons with MadaKV in our response to Q2 to further demonstrate the effectiveness of EMDA.