- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes FrameBlend, a training-free method for streaming video understanding that combines shorter attention windows during idle-time prefill with selective KV recomputation after a query arrives. Its QTS strategy selects recomputation targets based on retrieval frequency across layers or layer–head pairs. The selected tokens are jointly re-prefilled, and the recomputed KVs replace their corresponding cached entries. The method is evaluated across ReKV, LiveVLM, and HERMES on two streaming and three offline benchmarks, primarily using LLaVA-OneVision-7B, with additional streaming evaluations using Qwen2.5-VL-7B.
Critical strengths
-
FrameBlend exploits the information obtained after the arrival of the problem to fix the cache representation, thereby reducing the context required for idle phase processing. For retrieval-based frameworks, reusing existing retrieval results directly to select recomputed objects is an attractive solution. Table 5 supports the value of combining the two components: RVS-Ego improves accuracy from 59.3 to 62.4 on ReKV, while either component alone achieves lower accuracy than their combination. Despite the simplicity of the design, this is a useful systematic contribution.
-
The experiments in this paper include retrieval-based and compression-based systems, covering both streaming and offline tasks. For example, HERMES improves accuracy on MLVU from 60.2 to 64.1. Additional Qwen2.5-VL experiments also provide evidence beyond the single backbone model. The paper also reports FLOPs, actual runtime, first token latency, and memory overhead. These results help to evaluate the practical value of the method and understand its benefits and costs simultaneously.
Critical weaknesses
-
The robustness of the reported improvement effects has not been fully verified. Section 5.1 and Appendix B indicate that the pre-filled window was tuned separately for each framework-benchmark combination, but it does not specify which validation set partitioning was adopted, nor does it clarify whether the final evaluation labels or the feedback from the evaluation server affected this choice. Many overall improvement amplitudes are small, such as the increase from 57.3 to 57.5 for HERMES on OVO-Bench, and from 68.4 to 68.7 for ReKV on MLVU, and no uncertainty estimation was provided. Therefore, it is unclear how consistent these improvements will remain under independently selected, fixed window settings.
-
This paper has not fully demonstrated the unique contribution of its proposed selection strategy in terms of methodology. As acknowledged in related studies, selective key-value re-computation has been explored before, so its contribution largely depends on the allocation method of the proposed context and query guidance target selection. Although Table 5 evaluated the individual and combined effects of shorter windows and re-computation, it did not separate the role of QTS itself. Table 4 provides a useful comparison with CacheBlend, but no effective re-computation budget was reported, and there was no comparison with a simple selection strategy under matching conditions. Therefore, these results more clearly support the effectiveness of the overall method than the specific advantages provided by QTS.
Questions for authors
-
Under the condition that the pre-filling window and the recomputation budget remain unchanged, how much additional accuracy can QTS provide compared to simple strategies such as random selection or sorting by aggregated normalized retrieval scores? Please also report the effective recomputation budget used in the CacheBlend comparison as presented in Table 4.
-
In Table 5, adding recomputation alone reduces accuracy on RVS-Ego and RVS-Movie. How do the authors explain these degradations, and under what conditions does recomputation complement a shorter prefill window?
-
Which data partitioning methods and evaluation metrics were used when selecting the pre-filling window? Was the final evaluation label or the results from the evaluation server used in the selection process? Can the authors report the window results based on the non-overlapping validation data selection and attach the paired confidence intervals for the accuracy differences?
-
Figures 3 and 6 indicate that shorter pre-filling windows can improve accuracy, but changing the window length also alters the cache representation, which may affect the set of retrieved tokens. Can the authors isolate the proposed context interference reduction effect by fixing the retrieved and recalculated token sets and only varying the pre-filling window? Does this conclusion still hold under the limited recomputation budget used in the main experiments?
6 comments
Q3/W1. Validation-Set Tuning and Fixed Windows (1/2)
Q3. Which data partitioning methods and evaluation metrics were used when selecting the pre-filling window? Was the final evaluation label or the results from the evaluation server used in the selection process? Can the authors report the window results based on the non-overlapping validation data selection and attach the paired confidence intervals for the accuracy differences?
W1. The robustness of the reported improvement effects has not been fully verified. Section 5.1 and Appendix B indicate that the pre-filled window was tuned separately for each framework-benchmark combination, but it does not specify which validation set partitioning was adopted, nor does it clarify whether the final evaluation labels or the feedback from the evaluation server affected this choice. Many overall improvement amplitudes are small, such as the increase from 57.3 to 57.5 for HERMES on OVO-Bench, and from 68.4 to 68.7 for ReKV on MLVU, and no uncertainty estimation was provided. Therefore, it is unclear how consistent these improvements will remain under independently selected, fixed window settings.
1. Using prefill window size tuned on validation set
We agree with the reviewer's concern that the robustness of the reported improvement effects has not been fully verified. Therefore, We tuned prefill window size with separated validation set, and show that FrameBlend still improves overall accuracy. Since most of the benchmarks used in our evaluation do not provide official validation splits, the results in the original paper were obtained by selecting prefill window sizes within {196, 384 (=128*3), 512, 1024, 2048, 4096, 8192}, based on the final accuracy. We agree that this evaluation protocol may overestimate the accuracy gains of FrameBlend. To directly mitigate this concern, we conduct additional experiments using separate validation sets for window size tuning.
Specifically, we adopt the following evaluation protocol:
- Validation/test split: We randomly split each benchmark into equally sized validation and test sets (50:50), select the prefill window size using only the validation set, and report accuracy exclusively on the held-out test set.
- Repeated evaluation: We repeat this procedure with 100 different random seeds and report the mean test accuracy across all splits.
- Window size search space: We use the same candidate window sizes across all settings: {196, 384 (=128*3), 512, 1024, 2048, 4096, 8192}. So it’s significantly shorter than the original window sizes (15000 for RekV, 12000 for LiveVLM). For HERMES, we exclude 8192 because its original context size is 6000.
- EgoSchema: Since only 500 of the approximately 5,000 samples are publicly available, we exceptionally use these 500 samples for window selection and report accuracy on the full benchmark using the official evaluation server. RVS-Ego/Movie: Since the previously used GPT-3.5-turbo-0125 evaluator has recently been deprecated, we re-evaluate all RVS results using GPT-6-luna, averaging scores over three evaluation runs.
The test-set accuracy tuned with a separated validation set is in the table below. "revised" denotes the results obtained using the validation-set evaluation protocol described above, while "original" denotes results using the window sizes selected in the original paper. And we also reported the standard deviation of average accuracy occurred by test-val set splitting.
(continued in the next comment)
| Method | RVS-Ego | RVS-Movie | OVO-Bench | Video-MME | MLVU | EgoSchema | Average |
|---|---|---|---|---|---|---|---|
| LLaVA-OV-7B | |||||||
| ReKV | 57.60 | 41.15 | 52.31 | 57.92 | 68.41 | 60.03 | 56.24 ± 0.41 |
| ReKV + FrameBlend (original) | 58.46 | 40.91 | 53.76 | 59.48 | 68.74 | 60.27 | 56.94 ± 0.44 (+0.70) |
| ReKV + FrameBlend (revised) | 57.89 | 43.85 | 53.27 | 59.13 | 68.54 | 60.13 | 57.14 ± 0.45 (+0.90) |
| LiveVLM | 59.18 | 44.26 | 53.43 | 57.68 | 66.99 | 60.58 | 57.02 ± 0.46 |
| LiveVLM + FrameBlend (original) | 61.90 | 44.00 | 54.68 | 58.13 | 68.19 | 60.54 | 57.91 ± 0.44 (+0.89) |
| LiveVLM + FrameBlend (revised) | 61.48 | 43.47 | 54.52 | 57.78 | 67.84 | 60.33 | 57.57 ± 0.48 (+0.55) |
| HERMES | 58.05 | 46.74 | 57.27 | 55.79 | 60.04 | 59.67 | 56.26 ± 0.47 |
| HERMES + FrameBlend (original) | 60.43 | 48.29 | 57.42 | 58.78 | 63.98 | 61.28 | 58.36 ± 0.43 (+2.10) |
| HERMES + FrameBlend (revised) | 59.81 | 47.68 | 57.11 | 58.54 | 63.63 | 61.28 | 58.01 ± 0.45 (+1.75) |
Q3/W1. Validation-Set Tuning and Fixed Windows (2/2)
Q3. Which data partitioning methods and evaluation metrics were used when selecting the pre-filling window? Was the final evaluation label or the results from the evaluation server used in the selection process? Can the authors report the window results based on the non-overlapping validation data selection and attach the paired confidence intervals for the accuracy differences?
W1. The robustness of the reported improvement effects has not been fully verified. Section 5.1 and Appendix B indicate that the pre-filled window was tuned separately for each framework-benchmark combination, but it does not specify which validation set partitioning was adopted, nor does it clarify whether the final evaluation labels or the feedback from the evaluation server affected this choice. Many overall improvement amplitudes are small, such as the increase from 57.3 to 57.5 for HERMES on OVO-Bench, and from 68.4 to 68.7 for ReKV on MLVU, and no uncertainty estimation was provided. Therefore, it is unclear how consistent these improvements will remain under independently selected, fixed window settings.
The results above show that FrameBlend continues to improve accuracy over the baselines in most cases, even when window sizes are selected using a separate validation set. Moreover, the average accuracy gains remain comparable to those obtained with the original test-set tuning protocol. These findings indicate that the improvements of FrameBlend are not merely by test-set tuning, but persist under a proper evaluation protocol. Considering that FrameBlend also reduces overall computational cost, we believe these results support the effectiveness of our approach.
We will revise the accuracy results reported in the paper to use the test-set accuracy obtained by validation-set evaluation protocol instead of the original test-set tuning procedure. We appreciate the reviewer for this valuable comment, which prompted us to adopt a more rigorous evaluation protocol and reliably evaluate the effectiveness of our method.
2. Using fixed prefill window size
In addition, we also evaluate FrameBlend using a single fixed window size across all benchmarks for each framework. We show that FrameBlend still improves the average accuracy, even with a single fixed window size.
Since the results reported in the original paper use prefill window sizes tuned for each framework–benchmark pair, we evaluate FrameBlend using a single fixed window size across all benchmarks for each framework. In addition, we further evaluate FrameBlend using fixed window sizes separated for streaming and offline benchmarks within each framework. This setting can be practical, as streaming and offline benchmarks represent different application scenarios and have different temporal context requirements. As shown in the tables below, FrameBlend still improves the average accuracy of all three frameworks in both tables, without benchmark-specific window size tuning.
(ReKV: 2048, LiveVLM: 2048, HERMES: 512)
| Method | RVS-Ego | RVS-Movie | OVO-Bench | Video-MME | MLVU | EgoSchema | Average |
|---|---|---|---|---|---|---|---|
| LLaVA-OV-7B | |||||||
| ReKV | 57.6 | 41.0 | 52.3 | 57.8 | 68.4 | 60.0 | 56.18 |
| ReKV + FrameBlend | 58.0 | 43.3 | 53.8 | 58.9 | 67.8 | 60.0 | 56.97 (+0.79) |
| LiveVLM | 59.3 | 44.1 | 53.5 | 57.7 | 67.0 | 60.6 | 57.02 |
| LiveVLM + FrameBlend | 62.0 | 43.8 | 53.2 | 58.1 | 67.3 | 60.4 | 57.48 (+0.46) |
| HERMES | 57.9 | 46.7 | 57.3 | 55.8 | 60.2 | 59.7 | 56.26 |
| HERMES + FrameBlend | 59.3 | 48.2 | 57.4 | 58.7 | 64.1 | 60.8 | 58.09 (+1.83) |
(ReKV: 384/8192, LiveVLM: 1024/4096, HERMES: 512/512 for streaming/offline benchmarks, respectively)
| Method | RVS-Ego | RVS-Movie | OVO-Bench | Video-MME | MLVU | EgoSchema | Average |
|---|---|---|---|---|---|---|---|
| LLaVA-OV-7B | |||||||
| ReKV | 57.6 | 41.0 | 52.3 | 57.8 | 68.4 | 60.0 | 56.18 |
| ReKV + FrameBlend | 58.8 | 44.3 | 52.4 | 59.4 | 68.7 | 60.2 | 57.30 (+1.11) |
| LiveVLM | 59.3 | 44.1 | 53.5 | 57.7 | 67.0 | 60.6 | 57.02 |
| LiveVLM + FrameBlend | 61.9 | 43.7 | 53.7 | 58.0 | 68.1 | 60.4 | 57.67 (+0.65) |
| HERMES | 57.9 | 46.7 | 57.3 | 55.8 | 60.2 | 59.7 | 56.26 |
| HERMES + FrameBlend | 59.3 | 48.2 | 57.4 | 58.7 | 64.1 | 60.8 | 58.09 (+1.83) |
Q1/W2. QTS vs. Simple Baselines
1. Comparing with simple baselines
We appreciate the reviewer’s concern regarding the role of QTS itself and comparison with simple selection strategies. As shown in the table below, we conducted the suggested experiments and show that QTS consistently surpasses simple baselines. We set a recomputation budget of K=7840 and a fixed prefill window size of 384 (=128*3).
More specifically, ‘score sum/mean’ select tokens based on the sum/mean of their layer-wise normalized retrieval scores, considering only the layers in which each token is retrieved. We also report coverage, defined as the proportion of KVs in the decoding context that are replaced through recomputation (i.e., measuring how much of the decoding context is updated by recomputation). As shown in the table, QTS achieves the best overall accuracy and the highest coverage among the compared strategies. These results support the effectiveness of frequency-based selection for identifying recomputation targets.
2. Recomputation budget of CacheBlend in Table 4
In addition, to address the reviewer’s question, we clarify that CacheBlend's recomputation budget matches FrameBlend's K=7840 in Table 4. Following the official implementation of CacheBlend (link) which selects recomputation targets based on a full recomputation of the first layer, we selected 7,840 tokens for recomputation in subsequent layers. Thus, both methods use the same number of recomputation targets, although CacheBlend basically has additional overhead of calculating full recomputation to the first layer, which is a necessary operation for CacheBlend. We will clarify this implementation detail in the revised manuscript.
Q2. Short Windows and Recomputation
We believe that this is due to the contextual interference (c.f. our response to Q4). Whenever the model fails to retrieve relevant frames—e.g., due to overly long prefill windows during the prefill—recomputing KV with highly irrelevant frames may in fact hurt the model performance by incorporating unnecessary or query-irrelevant information from the other frames. This is in contrast with what happens with correctly retrieved frames (e.g., with shorter prefill window), where the information is enhanced with rich context.
RVS-Ego and RVS-Movie seem to be more prone to such retrieval failures, as these datasets consist of egocentric and movie videos with many blurry frames (due to camera shake) or frequent scene transitions. Such characteristics may introduce substantial redundant or irrelevant information during prefill.