- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
StreamBank combines speculative decoding and event memory for streaming video understanding. It uses parallel multi-token prediction and reuses previous responses to propose draft tokens. A frozen target model verifies these tokens. Feedback from verification guides the retention and eviction of historical events in a fixed-capacity memory. Experiments report a 2.01× decoding-throughput speedup and improved average scores on three streaming video benchmarks.
Critical strengths
- StreamBank uses verification feedback for both generation acceleration and event memory management. Table 2 reports a 2.01× decoding-throughput speedup with improved overall pAUC. Table 3 shows better event-recall accuracy than FIFO and visual similarity under the same memory budget. These results support the practical value of the combined framework.
Critical weaknesses
- The value of verification feedback for memory selection is not sufficiently established. Table 3 compares retention policies but does not isolate the gain and correction signals. It remains unclear whether these signals select useful events better than simpler task-relevance scores. This weakens the evidence for the paper’s main contribution.
- The efficiency evaluation is incomplete. The reported 2.01× speedup measures decoding throughput. The paper does not report end-to-end latency that includes video processing, memory updates, and event-caption generation. The actual benefit for real-time interaction therefore remains unclear.
Questions for authors
- Do you have ablations showing the separate contributions of the gain and correction signals? How do they compare with a simpler task-relevance score under the same memory budget?
- What is the end-to-end speedup when video processing, memory updates, and event-caption generation are included?
5 comments
Thanks for running these extra experiments. The ablation clears up my main concern about whether verifier feedback helps memory selection. I have one follow-up on the timing results. Could you share which stages account for most of the 2.85× speedup across the full workload? That would help me connect the overall speedup to the per-response latency numbers.
Answer to Reviewer hVoJ
Thank you for your follow-up. The overall speedup primarily comes from the response generation stage. On the one hand, StreamBank compresses historical context through event memory management, reducing the number of context tokens that need to be processed and thereby lowering prefill overhead. On the other hand, MTP and response reuse further accelerate autoregressive generation. Consequently, the response generation time decreases from 28.8 s/clip with AR to 10.3 s/clip with MTP, and further to 9.5 s/clip with MTP + Reuse, making it the primary contributor to the end-to-end speedup.
Answer to Reviewer hVoJ
Thank you for this suggestion. We agree that the contributions of the signals obtained from target-model verification should be assessed separately. We therefore ablate Verifier Gain and Correction Evidence on ProactiveVideoQA while keeping memory capacity and decoding settings unchanged. The baseline without either verification-derived signal still retains Novelty and Direct-response Support.
Removing both signals reduces overall pAUC from 31.0 to 29.5; adding Verifier Gain or Correction Evidence alone increases it to 30.7 and 30.8, respectively, with the best result obtained by combining both. This shows that verification-derived feedback provides additional improvements beyond novelty and direct-response support, offering direct evidence for the effectiveness of generation–memory coupling.
Thank you for raising this question. To assess overall system efficiency, we conduct additional end-to-end experiments comparing AR, MTP, and MTP + Reuse under the same evaluation workload.
E2E Speedup is computed as the ratio of total processing time under the same workload, including video processing, memory updates, event-caption generation, and response generation. TTFT and response latency measure per-response performance and are reported as P50/P95.
MTP achieves a 2.71× end-to-end speedup over MMDuet2, which further increases to 2.85× with response reuse. These results demonstrate that our proposed mechanisms improve overall inference efficiency, with benefits extending beyond token-level decoding acceleration.
Thank you again for your insightful comments. We hope our additional results and implementation details have addressed your concerns, and we would greatly appreciate your reconsideration of our work.