- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes StreamSieve, a training-free framework for efficient streaming video understanding. It consists of two main components: (i) refined dynamic encoding for spatiotemporal redundancy, which adaptively determines whether to skip, partially encode, or fully encode incoming frames based on temporal similarity, followed by spatiotemporal saliency-guided token refinement; and (ii) query-agnostic diversity-preserving hierarchical memory, which maintains recent visual information in short-term memory and preserves diverse historical tokens in long-term memory for future query answering. Experiments on online and offline video benchmarks demonstrate improved accuracy and efficiency, with reduced visual encoding FLOPs, GPU memory usage, and inference latency.
Critical strengths
- The most interesting module in StreamSieve is the temporal novelty-based dynamic hierarchical encoding (TNDHE), which attempts to reduce computation before visual features are fully extracted, rather than only compressing tokens after encoding. Although the proposed similarity-based dynamic encoding strategy is relatively simple, the direction is practically meaningful for streaming video understanding, where substantial redundancy exists between consecutive frames.
Critical weaknesses
- My main concern is the limited novelty. The proposed StreamSieve is more like an engineering combination of existing techniques. In particular, FluxMem (CVPR 2026) has already explored training-free, query-agnostic spatiotemporal token reduction with hierarchical memory for streaming video understanding. The proposed Spatiotemporal Saliency-guided Token Refinement (SSTR) and Query-agnostic Diversity-preserving Hierarchical Memory (QDPHM) share similar ideas with FluxMem's token compression and hierarchical memory mechanisms. The paper should better clarify its novelty beyond FluxMem, and a direct comparison is missing.
[a] FluxMem: Adaptive Hierarchical Memory for Streaming Video Understanding, CVPR 2026.
Questions for authors
- Clarify the main technical contributions beyond existing methods, particularly FluxMem (CVPR 2026), and explain the differences between their token refinement and hierarchical memory mechanisms.
- The proposed SteamSieve relies on several specified hyperparameters, but their effects are largely unexplored. In particular, the three fixed thresholds used to determine the novelty of each video frame, which directly determines the encoding strategy and computational cost, yet the paper provides no justification or sensitivity analysis for these choices. Other important parameters, such as the shallow encoding depth, token retention ratios, and memory replacement threshold, are also not sufficiently studied. I would like to see whether the reported accuracy-efficiency trade-off remains stable under different settings.
- For Similar frames, the method aggregates shallow representations from consecutive frames before propagating them through the remaining encoder layers. Could the authors clarify when the deeper layers are executed in the streaming setting? Is the accumulated representation updated and fully encoded upon each incoming frame, or only after a group of similar frames has been collected? This distinction is important for understanding the actual computational savings and online latency.
- The efficiency analysis only compares StreamSieve with the baseline, without comparisons against existing efficient streaming video methods. Since efficiency is a major contribution of this paper, the authors should compare with relevant methods, particularly FluxMem, under comparable settings. Moreover, the evaluation should also include comparisons with simpler strategies under the same computational or visual token budget, such as uniform frame sampling, periodic full encoding, or fixed token pruning.
- Could the authors compare the proposed method with existing approaches under the same computational or memory budget?
- OVO-Bench includes Forward Active Responding tasks, but Table 1 only reports Real-Time Visual Perception and Backward Tracing. Could the authors clarify why the Forward Active Responding tasks are excluded from the evaluation?
- Both backbones show some performance drops on Episodic Memory (EPM) and Action Sequence Identification (ASI) in OVO-Bench. Given that preserving historical information is one of the main goals of the proposed hierarchical memory, it would be helpful if the authors could provide some discussion or analysis of these results.