- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes WRWS, a training-free framework for evidence selection in streaming video understanding. It uses an external encoder and an entropy-based weight to balance semantic relevance with temporal recency, selecting observations before expensive target-model visual encoding.
Critical strengths
- The paper is very easy to follow, it addresses a practical problem: balancing historical evidence and recent context under a limited visual-processing budget.
- The framework requires no training or access to target-model internal states, making it easy to integrate with different Video-LLMs.
- The evaluation covers multiple model families and benchmarks, with additional task-level analyses and qualitative failure cases.
Critical weaknesses
- The design is highly heuristic and essentially relies on a simple weighted combination, providing insufficient novelty for ICLR.
- The parameter experiments in Table 4(a) and Appendix E.1 (Table 14) do not convincingly establish the effectiveness of the weighting mechanism, as the improvements are negligible.
- Table 2 shows no clear performance advantage over related methods.
- The reported reduction to 47.93% of baseline VTAT is not a meaningful efficiency comparison, since it compares against encoding the entire observed history. SimpleStream could simply buffer the latest four raw observations and encode them only when a query arrives.
Questions for authors
- Does the efficiency advantage remain against a Recent-only baseline that buffers the latest four raw observations and encodes them only when a query arrives, under the same visual budget?
- Which processing stages are included in VTAT and inference TTFT? Does WRWS also reduce latency measured from query arrival to the first answer token?
7 comments
Response to Reviewer #7nbS (2/2)
W4 & Q1 — Efficiency–capability trade-off between WRWS and buffered Recent-only
We agree that buffering recent raw observations and encoding them only upon query arrival is a valid and computationally efficient alternative. We clarify that the reported 47.93% VTAT ratio is measured against the evaluated encode-first SimpleStream pipeline.
Importantly, the efficiency of buffered Recent-only also depends on the query workload. In practical streaming scenarios, users may issue multiple queries over the same continuous video (e.g., at seconds). While buffering avoids expensive target-VLM encoding before queries arrive, each query still requires encoding the selected observations. Without cross-query feature caching, overlapping observation windows may lead to repeated encoding of the same frames, making cumulative computation dependent on query frequency and reuse strategies.
More importantly, the key distinction lies in the ability to access historical evidence under the same bounded target-model visual-processing budget. Buffered Recent-only selects only the latest observations, potentially missing relevant earlier evidence. In contrast, WRWS incrementally maintains lightweight representations of the observed history and selectively retrieves query-relevant observations, while encoding at most observations with the target Video-LLM per query.
Thus, WRWS enables history-aware evidence selection with bounded target-model computation, rather than restricting evidence to a fixed recent window. The reported efficiency improvement demonstrates its advantage over the evaluated encode-first pipeline, while the buffered alternative represents a distinct accuracy–efficiency trade-off.
Q2 — Clarifying VTAT&TTFT measurements, and the source of WRWS's efficiency gains
We would like to clarify that VTAT and inference TTFT capture different stages of streaming video understanding. Unlike offline video understanding, streaming systems continuously receive and process incoming video observations before a user query arrives. Accordingly, VTAT measures the elapsed time from the first visual observation to the first answer token, capturing the overall streaming pipeline, whereas inference TTFT starts only when the target Video-LLM begins answer inference, excluding preceding stream processing.
As shown in Appendix C (Table 5), target-VLM visual encoding accounts for 62.36% of the baseline VTAT, motivating our retrieve-first, encode-later design. In contrast, Appendix F (Table 17) shows nearly identical inference TTFT for WRWS and SimpleStream (0.2431 s vs. 0.2429 s), consistent with their identical target-model visual budgets. Thus, WRWS achieves its efficiency gains primarily by reducing cumulative visual-processing costs, rather than accelerating the final answer inference stage. We will clarify this distinction in the revised manuscript.
| Metric | Target-VLM visual encoding | Other pipeline overhead | Inference TTFT | Total VTAT |
|---|---|---|---|---|
| Time (s) | 7.596 | 4.342 | 0.243 | 12.181 |
| Fraction (%) | 62.36 | 35.65 | 1.99 | 100.00 |
| Group | # Samples | WRWS VTAT (s) | SimpleStream VTAT (s) | VTAT Ratio | WRWS Inference TTFT (s) | SimpleStream Inference TTFT (s) | TTFT Ratio |
|---|---|---|---|---|---|---|---|
| Overall | 3035 | 5.84 | 12.18 | 47.93% | 0.2431 | 0.2429 | 100.08% |
[1] Zhijia Liang, Jiaming Li, Weikai Chen, Yanhao Zhang, Haonan Lu, and Guanbin Li. OASIS: On-Demand Hierarchical Event Memory for Streaming Video Reasoning. arXiv preprint arXiv:2604.17052, 2026.
[2] Jitai Hao, Ke Yang, Qiang Huang, and Jun Yu. ShallowStream: Index Shallow Then Answer Deep for Streaming Video Understanding. arXiv preprint arXiv:2609.02780, 2026.
Thank you for the detailed rebuttal and clarifications. I appreciate the authors' efforts in addressing my concerns, particularly the explanations of the efficiency metrics and the practical motivation behind WRWS.
However, my main concerns remain. First, the improvements of adaptive weighting over the best fixed configurations are still marginal (0.25 and 0.34 points), which does not sufficiently demonstrate the effectiveness of the proposed mechanism. Second, the reported efficiency advantage is primarily established against an encode-first baseline, while the requested comparison with a buffered Recent-only baseline remains unanswered. Therefore, the claimed efficiency gains are not yet sufficiently supported by a fair comparison.
Overall, I acknowledge the practical value of this work, but I still find the methodological novelty and empirical advantages insufficiently convincing for ICLR. I will maintain my original score.
Response to Reviewer #7nbS (1/2)
W1 — Methodological contributions beyond heuristic weighted scoring
We appreciate the reviewer's concern and would like to clarify that the contribution of WRWS extends beyond the weighted combination itself, focusing on uncertainty-adaptive evidence allocation and the accuracy–efficiency trade-off in streaming video understanding.
First, WRWS adaptively incorporates historical evidence while preserving a strong recency prior under a bounded visual-processing budget. Unlike fixed evidence-selection policies, WRWS uses query-specific retrieval uncertainty to balance semantic relevance and temporal recency.
Second, WRWS performs evidence selection before expensive target-model encoding. Unlike OASIS [1], which relies on preliminary MLLM reasoning, and ShallowStream [2], which uses target-model internal signals, WRWS operates entirely through lightweight external representations without additional training or access to target-model states.
Importantly, efficiency is central to streaming video understanding, where continuous visual inputs and real-time requirements make computational cost particularly critical. WRWS addresses this practical challenge by limiting expensive visual encoding to selected observations while maintaining competitive accuracy, achieving a 52.1% VTAT reduction against the evaluated encode-first baseline.
W2 — Query-adaptive allocation versus fixed-α policies
The key advantage of WRWS is that it outperforms backbone-specific optimal fixed weightings using a unified uncertainty-adaptive rule, without requiring backbone-specific tuning. As shown in Appendix E.1, Table 14, the best fixed α differs across backbones: α=1.0 for Qwen2.5-VL-7B and α=0.75 for Qwen3-VL-8B. In contrast, WRWS applies the same adaptive rule to both backbones and achieves higher accuracy by 0.25 and 0.34 points, respectively.
W3 — Table 2 shows no clear performance advantage over related methods
We would like to emphasize that WRWS achieves measurable accuracy improvements in several settings, while its primary contribution lies in achieving a favorable accuracy–efficiency trade-off rather than pursuing accuracy gains alone.
First, WRWS achieves competitive or improved accuracy across multiple Video-LLM backbones. As shown in Table 2, WRWS improves over SimpleStream from 80.59 to 81.11 on StreamingBench and from 57.55 to 58.59 on OVO-Bench with Qwen3-VL-8B. Similar improvements are observed on other base model.
Second, WRWS substantially reduces expensive visual-processing costs while maintaining competitive accuracy. As summarized in the table below, WRWS reduces average VTAT by 52.1%, corresponding to a 2.09× speedup over the evaluated encode-first SimpleStream baseline on Qwen2.5-VL-7B, while improving accuracy on both benchmarks.
Together, these results highlight the practical value of WRWS in achieving substantial computational savings while preserving competitive or improved accuracy, which is particularly important for streaming video understanding under real-time constraints.