- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Event-ReHead, a training-free framework for streaming video understanding. It combines query-specific attention head selection, question-conditioned event segmentation, event skeleton augmentation, and selective reuse of historical question–answer pairs. This method utilizes representative frames from relevant events to complement token-level KV cache retrieval, aiming to enhance temporal consistency within a limited memory budget. Experiments cover multiple streaming and offline video benchmarks and employ various VLM backbones.
Critical strengths
- The motivation is reasonable: retrieving only visual tokens related to specific content risks losing the temporal context necessary to understand events; incorporating event-level context to complement retrieval effectively addresses a practical limitation in streaming video QA.
- The framework operates without updating backbone network parameters and reuses existing visual representations, offering potential value for adaptation to VLMs.
Critical weaknesses
1. There are inconsistencies between the main text and the tables regarding several key results, complicating the assessment of the actual performance gains. In Table 1, the OVO-Backward score for LLaVA-OV-0.5B is listed as 43.53; however, the corresponding score in Table 5 is 35.24, with 43.53 appearing in the "Overall" column instead. For Qwen2.5-VL-7B, Table 5 reports an "Overall" score of 61.92, yet the scores for its two branches are 72.51 and 50.19; based on the averaging method used in other rows of the table, the result should be 61.35. Furthermore, several improvement figures cited in Section 4.1 cannot be recalculated using the data in Table 1. These discrepancies materially affect the paper's main conclusions. Moreover, the ablation results in Figure 4 are inconsistent with the main tables. In the OVO-Bench panel, judging by bar heights, the result for the full model labeled LLaVA-OV-7B is over than 70, while the result for Qwen2.5-VL-7B is around 66; yet, the corresponding real-time perception scores in Table 1 are 67.02 and 72.51, respectively. These bar values also fail to align with the "Overall" scores in Table 5. Additionally, the claim of "consistent improvement" requires qualification: Table 1 shows that Qwen2.5-VL-32B experienced a 2.33 percentage point drop on VideoMME compared to HERMES.
2. Key implementation details are incomplete, making it difficult to confirm how the method adheres to the fixed cache budget. With a 4000-token cache budget, the KV data for certain event skeleton frames may have already been evicted; however, the paper does not specify how this KV data is retrieved—whether it is stored elsewhere, requires re-computation, or must be selected from the currently retained cache. Equation (14) incorporates event skeleton KV data via set union but fails to define how the total budget constraint is enforced after this addition, nor does it clarify the specific function of the "event ratio = 0.4" parameter. Further explanation is also required regarding the construction of frame representations and question representations in Equation (6). These details directly impact the method's reproducibility and resource overhead.
3. Efficiency experiments are insufficient to validate the actual overhead of the newly introduced components. In Figure 3, the TTFT y-axis is labeled in milliseconds with values below 4, yet the data points for the proposed method fall in the 27–29 ms range; this discrepancy between units and plotted values requires correction. More importantly, the efficiency comparison omits HERMES, the most direct baseline. The paper also fails to clarify whether the TTFT calculation accounts for operations such as attention head selection, question-conditioned event discovery, event skeleton retrieval, and historical QA retrieval. Merely reporting stable GPU memory usage is insufficient to demonstrate the total cost associated with additional historical frame features or KV storage.
Questions for authors
- Specify which set of results in Tables 1 and 5 is the definitive one. Explain the reasons for the score inconsistencies, provide the aggregation rules used for the "Overall" metric, and ensure that the improvement figures cited in Section 4.1 align with the corrected tables.
- How are the KV pairs for event skeleton frames retrieved after cache eviction? Explain the storage or recovery mechanism, the budget execution method following Equation (14), the specific meaning of the "event ratio," and the construction methods for the visual and textual representations in Equation (6).
- Specify the correct units for Figure 3 and the timing range for TTFT. Which operations dependent on the current question are included in the latency calculation? Are there existing measurement results directly comparable to HERMES under the same hardware, backbone, and cache budget?
5 comments
Response to Reviewer hT6Z (Part 2/3)
Response to Question 2: Event skeletons, retrieval budget, and representations
Skeleton construction. For the selected event , we divide its duration into three consecutive stages: beginning, middle, and end. Within each stage, we select the frame with the highest question–frame relevance from Eq. (6):
,
where indexes the three stages. The resulting skeleton is .
The temporal stages determine where to search; question relevance determines which frame represents each stage. This preserves coverage of the event progression without mechanically selecting boundary frames. Candidates must have available KV evidence.
Event ratio. In the clarified budget-allocation specification, limits the share of the retrieval budget assigned to event evidence: , where is the total visual-token retrieval budget and is the maximum number of positions allocated to event evidence.
Thus, allows event evidence to occupy at most 40% of that budget. It neither discards 60% of the skeleton anchors nor permanently reserves 40% of the cache. All three stages participate; the amount of evidence depends on coverage by the base retrieval.
Completion within the budget. Skeleton evidence comes only from KV positions still available in the current cache. We neither recover evicted historical KV nor re-encode the corresponding frames. We first allocate representative evidence to uncovered event stages, then fill the remaining budget with high-scoring base evidence. Overlapping positions are counted only once, and the final set satisfies .
For a 4,000-token budget, event evidence can occupy at most 1,600 positions. If only 300 positions are needed, the remaining budget remains available for fine-grained retrieval. The objective is to improve temporal coverage within the same budget, rather than retain an additional historical KV store.
Frame and question representations. For a frame or temporal bin , we average the projected visual tokens associated with that frame:
.
The visual features are produced by the backbone vision-to-language projection and are retained as compact frame-level features for event analysis. They are not additional per-layer KV pairs.
For a question , we tokenize the text, obtain the corresponding input embeddings, and average them:
.
After L2 normalization, the frame-question relevance is computed as:
.
This similarity is used as a lightweight semantic cue together with visual frame-change scores during question-conditioned event discovery and event selection. It does not require an additional vision-encoder pass. The final KV selection remains based on the contextualized retrieval scores used by the cache-retrieval module. This relevance guides query-conditioned event discovery, allowing the model to locate the relevant event before performing KV retrieval and to reduce temporal fragmentation caused by concentrating retrieval on only a few instants.
Response to Reviewer hT6Z (Part 3/3)
Response to Question 3: Efficiency measurements and timing scope
Figure 3. We will correct the TTFT axis and annotations. If the axis is displayed in seconds, values of 27–29 ms will be plotted as 0.027–0.029 s, with any millisecond annotations explicitly labeled.
HERMES comparison. We supplement the efficiency evaluation using Qwen2.5-VL-7B, with 8 frames per chunk and input lengths of 16, 64, 256, and 512 frames. Both methods use the same input sequences, KV budget, hardware, and decoding configuration. This controlled workload comparison measures GPU memory, first-token latency, and subsequent decoding latency; the frame counts describe the workload rather than an accuracy benchmark.
| Frames | Method | GPU memory (GB) | TTFT (ms) | TPOT (ms/token) |
|---|---|---|---|---|
| 16 | HERMES | 16.54 | 27.01 | 24.43 |
| 16 | Event-ReHead | 16.80 | 27.52 | 24.56 |
| 64 | HERMES | 16.66 | 28.41 | 23.89 |
| 64 | Event-ReHead | 16.83 | 28.71 | 24.31 |
| 256 | HERMES | 16.66 | 28.44 | 24.02 |
| 256 | Event-ReHead | 16.83 | 28.71 | 24.45 |
| 512 | HERMES | 16.66 | 28.41 | 23.98 |
| 512 | Event-ReHead | 16.83 | 28.71 | 24.51 |
At 512 frames, Event-ReHead adds 0.17 GB memory, 0.30 ms TTFT, and 0.53 ms TPOT, approximately 1.0%, 1.1%, and 2.2%, respectively. Our claim is that retrieval quality improves at a modest measured cost, not that Event-ReHead is more efficient than HERMES.
Timing scope. We measure TTFT from the moment the user query is received, before any query-dependent processing begins, until the first answer token is produced. The measured interval includes attention-head selection, question-conditioned event discovery, event-skeleton mapping and retrieval, historical QA retrieval, cache selection, prompt construction, answer prefill, and the computation required to generate the first answer token. Only video processing completed before the query arrives is excluded. This end-to-end query-level timing allows the reported TTFT to capture the additional processing introduced by Event-ReHead when compared with HERMES.
Storage scope. The GPU KV cache stays within a fixed budget, so GPU memory remains stable as the video grows. We also keep one small feature vector per processed frame or temporal segment for event detection. This auxiliary storage grows with video length. Thus, our results show a small GPU-memory and latency overhead in the tested setting.
Declaration of Contribution
We are the first to supplement missing temporal information in streaming videos under a fixed budget by completing event skeletons. Combined with attention-head retrieval and an interaction memory bank, our method improves the performance of base models on mainstream benchmarks with low additional cost. We believe this approach has practical value and hope the reviewers will recognize its contributions and reconsider the paper.
Response to Reviewer hT6Z (Part 1/3)
Response to Question 1: Numerical consistency and ablation corrections
Thank you for carefully checking the text, tables, and figures. We acknowledge that these inconsistencies undermine the verifiability of the results. We will correct the numerical entries, replace the erroneous ablation plot, and revise the associated claims.
Table corrections. For LLaVA-OV-0.5B on OVO-Bench Backward, the entry in Table 1 should be 35.24, not 43.53; the latter was copied from the Overall column of Table 5. For Qwen2.5-VL-7B on OVO-Bench, the Overall score in Table 5 should be 61.35, not 61.92.
We will recalculate the gains in Section 4.1 and the appendix and check the corresponding statements in the abstract, introduction, and conclusion.
Figure 4 correction. In the OVO-Bench panel, the group labeled LLaVA-OV-7B incorrectly used Qwen2.5-VL-7B results on OVO-Bench Real-Time. The group labeled Qwen2.5-VL-7B incorrectly used MVBench ablation data. The intended benchmarks were StreamingBench, OVO-Bench Real-Time, and VideoMME. We will correct the panel, identify the OVO-Bench split explicitly, and revise Section 4.3.
We supplement the corrected ablations using LLaVA-OV-7B and Qwen2.5-VL-7B on OVO-Bench Real-Time. Each variant removes one component while retaining the others; all entries are answer accuracy (%):
Removing memory, event retrieval, and head selection reduces accuracy by 0.64, 2.29, and 1.45 percentage points for LLaVA-OV-7B, and 0.59, 1.72, and 0.73 points for Qwen2.5-VL-7B. All three components contribute positively in this split, with the largest reduction following removal of the event module.
Separate MVBench results. We will report the offline ablations separately in the appendix. We add ablations using LLaVA-OV-7B and Qwen2.5-VL-7B on MVBench to examine whether the components also benefit offline video understanding. Entries are answer accuracy (%):
The full model exceeds each ablation in this table, but the relative contributions vary across backbones and tasks. Through ablation experiments on MVBench, we further demonstrate the roles of our various components.
VideoMME exception. We agree that the claim of consistent improvement was too broad. With Qwen2.5-VL-32B on VideoMME, Event-ReHead scores 63.11, compared with 65.44 for HERMES, a decrease of 2.33 points.
One possible explanation is a trade-off between selective event retrieval and global coverage. Event-ReHead concentrates the cache on question-relevant evidence and supplements temporal continuity within a relevant event. Some offline questions, however, require evidence distributed across multiple events. More concentrated retrieval may reduce coverage elsewhere.
The limitation is specific to the evaluated backbone–dataset combination. On VideoMME, Event-ReHead improves over HERMES by 0.56 points with LLaVA-OV-7B and 1.31 points with Qwen2.5-VL-7B. With Qwen2.5-VL-32B, it improves StreamingBench, OVO-Bench Real-Time, and OVO-Bench Backward by 0.57, 0.50, and 1.14 points, and MVBench and EgoSchema by 1.43 and 3.61 points.
We will therefore state that Event-ReHead improves the reported streaming settings and most offline settings, rather than every backbone–dataset combination. We take responsibility for the presentation errors and will cross-check tables, plots, and textual gains against the original evaluation outputs.