- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes StreamingReTaKe, a training-free method for streaming video QA that extends ReTaKe/AdaReTaKe to the online setting. It has three parts: 1) A per-frame token budget driven by a novelty score, with a “budget bank” that carries unused tokens forward. 2) A recent window of full KVs, plus a long-term memory where tokens are selected jointly at the end of each episode. 3) Retrieval at question time that mixes question relevance with the stored importance scores. The authors claim memory that stays bounded regardless of stream length. They report results on OVO-Bench and StreamingBench with four backbones, along with some offline benchmarks.
Critical strengths
The method is training-free and works on four different backbones. The efficiency part is reasonably analyzed and frame deadlines are met at 2 FPS, peak memory is lower than HERMES, and the TTFT breakdown with pending frames is useful. For a practical system these aspects are important.
Critical weaknesses
-
It is not clear what the contribution is. The setup (a recent window, plus a compact long-term memory built before the question is known, plus question-time retrieval over both) is already common in streaming video work, for example [R1, R5, R6]. Most of the individual components also look close to existing methods: query-agnostic attention scoring for retention is close to HERMES, KV novelty and norm-based retention is close to [R4], dropping frames or tokens based on change from the previous frame is close to [R8] and FluxMem, KV retrieval with position re-indexing is close to ReKV, LiveVLM and StreamKV. The contribution list in the intro does not say which of these is the new idea, and I could not tell after reading it. Several relevant works are also not discussed (list below).
-
The results do not clearly support the claims: Compared with HERMES, the gains are small and not consistent. The method is +0.5 on StreamingBench with Qwen3-VL-8B. It is worse on OVO Backward and Avg with LLaVA-OV, worse on StreamingBench with Qwen2.5-VL, and worse on Video-MME.
-
The hyperparameters (B, K, W, M_KV) are different for every backbone. The LLaVA row in Table 1 uses K=64, W=6, which is not the stated reference setting. The method is also quite sensitive: W=64 loses 12.6 points and M_KV=2048 loses 11.8 points. It is not clear if the settings were tuned on the test benchmarks. There are no runs with different seeds or error bars, while many margins are under 1 point.
-
The TTFT advantage over HERMES mostly comes from HERMES having pending frames at question time. After pending frames are processed, HERMES is faster (72 ms vs 148 ms).
-
The paper also has no baseline that uses only the recent window. Real-time subsets are known to depend mostly on recent frames, so such a baseline is needed to show the long-term memory is actually helping.
Some missing references:
[R1] Shehreen Azad, Vibhav Vineet, Yogesh Singh Rawat. StreamReady: Learning What to Answer and When in Long Streaming Videos. CVPR 2026 (arXiv:2603.08620).
[R2] Vaggelis Dorovatas, Soroush Seifi, Gunshi Gupta, Rahaf Aljundi. Recurrent Attention-based Token Selection for Efficient Streaming Video-LLMs. NeurIPS 2025.
[R3] Azad et. al. HierarQ: Task-Aware Hierarchical Q-Former for Enhanced Video Understanding. CVPR 2025 (arXiv:2503.08585).
[R4] Minsoo Kim, Kyuhong Shim, Jungwook Choi, Simyung Chang. InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video Understanding. NeurIPS 2025 (arXiv:2506.15745).
[R5] Haoji Zhang, Yiqin Wang, Yansong Tang, et al. Flash-VStream: Efficient Real-Time Understanding for Long Video Streams. ICCV 2025.
[R6] Zhenpeng Huang, Xinhao Li, Jiaqi Li, et al. Online Video Understanding: OVBench and VideoChat-Online. CVPR 2025.
[R7] Ruyi Xu, Guangxuan Xiao, Yukang Chen, et al. StreamingVLM: Real-Time Understanding for Infinite Video Streams. ICLR 2026 (arXiv:2510.09608).
[R8] Linli Yao, Yicheng Li, Yuancheng Wei, et al. TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videos. ACM MM 2025.
[R9] Haibo Wang, Bo Feng, Zhengfeng Lai, et al. StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistant. NeurIPS 2025.
Questions for authors
-
It is not clear what is new compared with AdaReTaKe combined with HERMES style retention and ReKV-style retrieval? Which component would be the main contribution, and how much does it add on its own?
-
How were B, K, W and M_KV chosen for each backbone? Why does the LLaVA result in Table 1 use K=64, W=6? Results with one shared config, or with settings tuned on a held-out split, would help a lot.
-
Why does accuracy drop so much at W=64? What does “author-reported” mean for the M_KV=2048 row?
-
It will be good to see a recent-window-only baseline, and a comparison of question-only retrieval against ReKV.
-
HERMES reports much lower TTFT in its own paper. It will be good to compare under matched conditions, i.e. no pending frames and the same KV budget.
-
It will be good to see mean and std over a few runs, and results on hour-long streams. The duration-independence claim is only really tested up to 1,024 frames.
-
The MVBench number (56.95) is the same as HERMES with 6K. It is not clear how this score was obtained.
Thank you for the author's explanation, which has resolved some of my doubts. I will raise my score, but I may not be confident.