- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes EviStream, a training-free framework for streaming video understanding. ISM incrementally records textual summaries, while MEKS selects keyframes based on temporal novelty, text-gap feedback, and query relevance. After a query arrives, the system retrieves text and associated frames, continues updating memory, and repeatedly decides whether to answer or wait.
Critical strengths
S1: The problem is important, and the framework requires no additional training.
Reducing the token footprint of continuously arriving video while preserving evidence for future questions is an important and practically relevant problem. Organizing complementary textual summaries and keyframes chronologically can reduce redundant visual tokens while retaining event context and visual details. The framework requires no additional training or parameter updates.
Critical weaknesses
W1: The advantages over existing methods are unclear.
The closely related Nar-KFC first selects keyframes and then uses text to supplement the remaining content, whereas this paper first generates textual summaries and then selects keyframes. Does this change in processing order provide a substantive advantage? Moreover, methods such as TimeChat-Online compress similar visual content and retain differences. This is conceptually similar to the proposed “abstraction plus supplementation”, with the main difference being that this work summarizes shared content as text. More concerningly, the experimental comparisons used to demonstrate this design’s advantage are not sufficiently fair: this work uses the stronger Qwen3.5-9B without comparing these methods under the comparable model. Therefore, it is difficult to determine whether the gains come from the proposed memory mechanism.
W2: The retrieval logic at answer time is confusing.
The method uses keyframes to preserve visual information omitted from textual summaries, yet retrieves text first and then accesses keyframes from the corresponding windows. If a summary omits the object or event asked about, the relevant window may not be retrieved. As a result, even if the visual evidence needed to answer the question has been preserved, it may remain inaccessible to the answering model.
Questions for authors
Q1: Can the comparison with Nar-KFC use matched input budgets?
Table 3 lists Nar-KFC with only 8 frames, whereas this method processes video at 1 FPS, giving the two methods access to substantially different amounts of visual information. As Nar-KFC is one of the closest related methods, could the authors compare them using the same environment?
Q2: Query-guided memory writing can limit answers to subsequent questions.
The method mainly focus on a single active query, with a structural limitation for consecutive questions. By using the current query to select future keyframes, the method will drop visual memory that may be needed in subsequent queries. Switching queries or repeating retrieval cannot recover that evidence. This differs from those related works that compress memory independently of the query: even that they are evaluated on the same benchmarks, which test questions separately. In fact, continuous interaction is a basic setting in online video understanding, as studied in StreamBench, EgoServe, and early VideoLLM-online. Could the proposed method avoid this problem?
Q3: Does frame selection mainly benefit from qe rather than nov and gap?
In Table 7, qe alone achieves 63.3 overall, while adding both nov and gap only increases the score to 63.4. This suggests that the benefit mainly comes from qe. Can nov and gap scores actually preserve information needed for future, unknown questions? If not, why include them? Such strong dependence on the current query would also make the consecutive-question concern in Q2 more serious.
Q4: Why does performance decline across all FAR subtasks?
In Table 6, average FAR performance drops from 59.9 to 55.1, with declines on all three subtasks. This suggests that the memory mechanism may impair some streaming video understanding capabilities. The authors attribute this decline to the backbone receiving more complete video information. However, the backbone also directly accesses video in the other tasks. Why does FAR consistently deteriorate, particularly REC, which drops by 11.2 percentage points?
Q5: Can the system sustain polling once per second in practice?
The paper states that the VLM is polled once per second, whereas Table 4 reports a query TTFT of 5.76 seconds. Could this cause an accumulating latency between inference and the video stream? Could the actual delay from evidence arrival to answer delivery become several times larger than the reported TTFT?
References:
[1] Threading Keyframe with Narratives: MLLMs as Strong Long Video Comprehenders (Nar-KFC).
[2] Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledge (StreamChat / StreamBench). ICLR 2025.
[3] Vinci2: Providing Proactive Assistance in Continuous Egocentric Videos (EgoServe). ECCV 2026.
[4] VideoLLM-online: Online Video Large Language Model for Streaming Video. CVPR 2024.
9 comments
Response to Reviewer cChp (Part2)
[W1.3] Comparisons with a matched backbone and frame rate
"More concerningly, the ... this work uses the stronger Qwen3.5-9B without comparing these methods under the comparable model... whether the gains come from the proposed memory mechanism."
We thank the reviewer for raising this concern. We chose Qwen3.5-9B to evaluate whether EviStream can improve an already strong backbone. However, we agree that comparisons across different backbones do not isolate the contribution of the memory framework.
We therefore add comparisons using the same Qwen3-VL-8B backbone and a fixed input frame rate of 1 FPS. Table 12 compares EviStream with the bare backbone, Response-G1, STRIDE, and HERMES under these settings.
Table XII. Comparisons with Qwen3-VL-8B.
| Method | StreamingBench (%) | OVO-Bench RT (%) | OVO-Bench BT (%) | OVO-Bench Avg. (%) |
|---|---|---|---|---|
| Qwen3-VL-8B | 56.4 | 67.2 | 45.7 | 56.5 |
| + Response-G1 | 77.5 | 73.6 | 52.1 | 62.9 |
| + STRIDE | 69.7 | 69.7 | 47.8 | 58.8 |
| + HERMES | 79.0 | 69.8 | 53.7 | 61.8 |
| + EviStream (ours) | 80.2 | 73.7 | 61.1 | 67.4 |
| EviStream achieves the highest scores across the reported metrics under the matched backbone and frame rate. On StreamingBench, it improves over the bare backbone by 23.8 percentage points and exceeds the strongest alternative, HERMES, by 1.2 points. On the reported OVO-Bench average, it exceeds Response-G1 by 4.5 points, with a particularly pronounced advantage on BT. |
These results show that EviStream’s gains are not solely attributable to the stronger Qwen3.5-9B backbone and support the effectiveness of the framework on Qwen3-VL-8B as well. We will incorporate these comparisons into the revised Tables 1 and 2 and document the evaluation settings.
[W2] Recovering visual evidence omitted from textual summaries
"W2: The retrieval logic at answer time is confusing ... it may remain inaccessible to the answering model."
We thank the reviewer for identifying this potential retrieval failure. Textual retrieval provides anchor windows, rather than restricting visual access to the directly retrieved entries. After locating relevant semantic memories and their source windows, EviStream applies Anchor Neighborhood Expansion (ANE) to include adjacent temporal windows and additionally retains the two most recent windows. The associated visual keyframes are recovered through shared window identifiers.
This allows a window to be accessed through the broader event context even when its summary omits a specific visual detail. Neighboring windows and recent observations provide additional routes to the relevant evidence. However, these mechanisms mitigate rather than eliminate the risk: if neither the relevant window nor a suitable neighboring anchor is retrieved, preserved visual evidence may remain inaccessible.
Our additional analysis in response to Q3 measures Recall@128 for visual evidence absent from semantic memory. Recall reaches 73.1% with NOV and GAP, indicating that the evaluated pipeline recovers a substantial proportion of the omitted evidence, while leaving a residual retrieval gap.
We will clarify the anchor expansion and recent-window inclusion mechanisms in the revised manuscript and explicitly discuss failures caused by relevant windows falling outside retrieval coverage.
Response to Reviewer cChp (Part3)
[Q2] Query-guided memory writing and subsequent questions?
"Q2: Query-guided memory writing can limit answers to subsequent questions ... Could the proposed method avoid this problem?"
We thank the reviewer for raising this concern. EviStream maintains a query-independent semantic history rather than constructing its entire memory around the active query. ISM continuously consolidates observed events independently of user queries. MEKS additionally preserves visual evidence using novelty and text-gap feedback before query arrival, with query relevance introduced as an additional selection signal once a query becomes available. Thus, later questions can draw on accumulated semantic history and retained visual evidence beyond the current query’s immediate needs. This mitigates query-specific information loss, although it cannot guarantee preservation of every visual detail needed by future questions.
Evaluation on consecutive, context-dependent questions. We additionally evaluate EviStream on StreamingBench’s Sequential Question Answering (SQA) task. SQA presents sequences of related questions over the same video, with later questions referring to entities or events introduced in earlier interactions. Answering these questions requires resolving cross-turn references and combining conversational context with the available visual evidence. Following the evaluation protocol, previous questions, answer options, and ground-truth answers are included as dialogue history.
Table XIII. Performance on StreamingBench SQA.
| Method | SQA (%) ↑ | Gain (pp) |
|---|---|---|
| Qwen3.5-9B | 42.8 | — |
| STRIDE | 46.4 | +3.6 |
| EviStream (w/o query evidence) | 50.4 | +7.6 |
| EviStream | 58.8 | +16.0 |
Without query evidence, EviStream already exceeds the backbone by 7.6 percentage points, supporting the effectiveness of its query-independent memory configuration. Adding query evidence further improves accuracy by 8.4 points, reaching 58.8%. These results indicate that query-guided selection provides a net benefit under the evaluated sequential-question protocol, while query-independent memory remains useful across related questions.
We will include these results and clarify how query-independent memory and query-guided selection jointly support consecutive questions in the revised manuscript.
Response to Reviewer cChp (Part1)
Thank you for your constructive comments. Below are our point-by-point responses.
[W1.1 / Q1] Design differences from Nar-KFC and comparison under matched input limits
We thank the reviewer for suggesting this clarification and comparison. The different processing order reflects the information available in offline and streaming settings. Nar-KFC addresses offline long-video understanding, where the complete video and the query are available for keyframe selection. It can therefore first select query-relevant and diverse keyframes across the video, then use textual narratives to supplement the temporal content between them.
EviStream instead constructs memory as video frames arrive, before future frames or queries are available. ISM incrementally summarizes the observed content, and MEKS then uses the resulting semantic representation to guide keyframe retention through text-gap feedback and temporal novelty. Generating the summary first allows the system to assess which visual information is insufficiently represented in that summary and prioritize complementary frames.
Comparison with aligned input limits on offline datasets. The eight frames reported for Nar-KFC refer to its answer-time keyframes, rather than all frames considered during selection. Both methods use candidate frames sampled at 1 FPS in our comparison. Following the reviewer’s suggestion, we evaluate EviStream on Video-MME Long without subtitles, reducing its answer-time historical keyframe limit to eight and increasing its textual-entry limit from 30 to 210 to match Nar-KFC’s narrative count.
Table XI. Video-MME Long results with eight keyframes and up to 210 textual entries.
EviStream outperforms Nar-KFC by 4.4 percentage points in this comparison, demonstrating competitive performance under the same frame and textual-entry limits. We will include these results and specify the backbone, inference settings, and input budgets for both methods in the revised manuscript.
[W1.2] Beyond visual redundancy reduction: comparison with TimeChat-Online and our core insight
We acknowledge that EviStream builds on visual redundancy reduction, as explored by TimeChat-Online and cited in Section 3.2. Visual-centric methods primarily reduce redundant visual tokens, while memory-centric methods such as StreamForest organize historical information to extend context coverage. EviStream integrates these directions through the coordinated construction of evolving semantic memory and complementary visual evidence.
Our key insight is that visual retention should depend on what the evolving semantic memory has already captured. ISM incrementally records new semantic information relative to the accumulated history, while MEKS uses text-gap feedback and temporal novelty to preserve visual details insufficiently represented in semantic summaries. This coordination enables semantic memory to guide visual evidence retention, maintaining complementary information across modalities. At query time, relevant semantic history and associated visual evidence are jointly retrieved to support subsequent reasoning.