- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper treats continuous VideoLLM inference as a pipeline over visual preprocessing, visual encoding, token dropping, and LLM prefilling and decoding. It combines chunked execution and CUDA-stream overlap with temporal token dropping, local visual attention, and query-side retrieval, and fine-tunes Qwen2.5-VL-3B/7B with supervised instruction tuning under an adaptive visual-scope mask. Component profiling and evaluations on six streaming-oriented benchmarks report a calculated maximum of 134 FPS, query TTFT below 50 ms, and accuracy near full-history streaming under moderate compression.
Critical strengths
-
Chunking amortizes visual-encoder calls, local attention bounds the LLM work for each incoming chunk, and retrieval restores selected older evidence. These parts address different bottlenecks in one pipeline, while Equation (3) adapts training to the same restricted attention scopes used at inference.
-
One 3B configuration in Table 3 drops 57.6% of the visual tokens while improving StreamingBench by 0.12 points, StreamBench by 1.35 points, and RVS-Ego by 0.63 points. Its losses on OVO-Bench and RVS-Movie remain below two points, showing that moderate compression can provide a useful trade-off on this backbone.
Critical weaknesses
-
The technical novelty is limited. Section 4 combines chunk-wise encoding, CUDA-stream preprocessing, similarity-based token dropping, local attention, query retrieval, and a matching training mask, but none of these components introduces a substantial new algorithm. The paper therefore needs strong end-to-end evidence that coordinating them produces a system-level benefit beyond a straightforward combination of existing techniques.
-
It is not fully clear whether the reported 134 FPS represents sustained wall-clock throughput of the complete pipeline or an estimate based on stage-wise latency. In the , 30%-retention row of Table 2, the number is consistent with ms, where 119 ms is the slowest measured stage. Clarifying the measurement procedure and reporting a sustained concurrent run or pipeline trace would make this efficiency result easier to assess, especially because the GPU stages share one A100.
-
The current component studies make it difficult to isolate the benefit of coordination. Table 5(b) focuses on preprocessing, while Table 5(c) changes ,
acn, the dropping rule, and the first-frame anchor without matching the actual retained token count or prefill latency. A serial-versus-pipelined end-to-end control and an ablation of the adaptive training mask would help clarify how much each part contributes. -
It is difficult to connect the headline efficiency and accuracy results at exactly the same operating point. Table 2 reports 134 FPS for and 30% token retention, while Tables 3 and 4 vary
acn,ur, and without reporting the corresponding , FPS, or TTFT. The rows closest to 30% retention show noticeable drops on several 7B benchmarks, although they may not be the exact configuration used for the 134-FPS result. Reporting accuracy and efficiency together for that configuration would make the central trade-off clearer.
Questions for authors
-
A sustained wall-clock run of the complete pipeline under the , 30%-retention setting would be helpful. Reporting total processed frames, elapsed time, and a serial or pipeline-disabled control would show whether the derived 134 FPS is achieved in practice.
-
Adding chunk size, streaming FPS, and query TTFT to Tables 3 and 4 would allow accuracy and efficiency to be compared at the same operating point. Table 4 would also benefit from the measured token retention already reported for Table 3, especially for the exact configuration used in the 134-FPS result.
-
Table 5(c) would be more informative if the frame-level and chunk-level rows matched the retained token count, visible temporal span, and LLM prefill latency. This would separate the effect of chunk-level dropping from the additional context or tokens retained by each configuration.
5 comments
W3: The current component studies make it difficult to isolate the benefit of coordination. ... A serial-versus-pipelined end-to-end control and an ablation of the adaptive training mask would help clarify how much each part contributes.
We thank the reviewer for these three requests.
For serial-versus-pipelined control: As provided in W1: the same modules run 1.59–2.05× faster under four-stage coordination than under serial execution, with preprocessing-only overlap accounting for an intermediate gain.
For Table 5(c) budget matching: The frame-level rows are calibrated to retain about 40% and 25% of visual tokens at s=0.25 and s=0.10 against the measured 42.4% and 27.1% for the chunk-level rows. In this study, we achieve the frame-level token dropping across adjacent frames rather than within a chunk so that the retained-token ratio differs by at most 2.6% between the two groups. Under this budget, chunk-level dropping leads at both thresholds: 70.68 against 68.80 at s=0.25 and 68.28 against 66.48 at s=0.10.
| Input setting | Frames per chunk | acn | s | Retained tokens | StreamingBench |
|---|---|---|---|---|---|
| Frame-level | 1 | 16 | 0.25 | 39.8% | 68.80 |
| Frame-level | 1 | 16 | 0.10 | 25.1% | 66.48 |
| Chunk-level | 8 | 4 | 0.25 | 42.4% | 70.68 |
| Chunk-level | 8 | 4 | 0.10 | 27.1% | 68.28 |
| Chunk-level | 8 | 8 | 0.25 | 42.7% | 70.48 |
| Chunk-level | 8 | 8 | 0.10 | 27.1% | 67.68 |
Note that Qwen2.5-VL uses a default temporal_merge_size of 2, so the frame-level and chunk-level settings cover the same temporal span despite using different acn values. This ensures a fair comparison in terms of temporal coverage.
For adaptive training mask: The training mask is determined by acn and ur, which control how much historical context the model can access during training and inference. Actually, we train separate checkpoints for different acn/ur configurations, each using its corresponding attention mask. Varying acn and ur therefore constitutes an ablation of the training-mask configurations, rather than an inference-only attention modification. Moreover, applying an acn/ur setting at inference that differs from the one used in training would produce a train-eval mismatch, so in all experiments the mask is kept identical between the two. These configurations allow us to evaluate the effects of different attention scopes, as shown in the following table.
| Backbone | Dropping threshold | Full-attention configuration | Bounded-attention configuration | Accuracy change |
|---|---|---|---|---|
| Qwen2.5-VL-3B | 0.25 | 69.96 | 70.44 | +0.48 |
| Qwen2.5-VL-3B | 0.10 | 67.40 | 68.32 | +0.92 |
| Qwen2.5-VL-7B | 0.25 | 73.28 | 71.64 | −1.64 |
| Qwen2.5-VL-7B | 0.10 | 71.36 | 69.71 | −1.65 |
According to the results, bounded attention improves accuracy by 0.48–0.92 points on 3B, while reducing it by 1.64–1.65 points on 7B. These models are trained with adaptive attention masks corresponding to their acn and ur settings, rather than applying bounded attention only at inference. Therefore, these results characterize the accuracy trade-offs of different training-mask configurations with matched inference attention, showing that the effect depends on the backbone.
W4: It is difficult to connect the headline efficiency and accuracy results at exactly the same operating point. ... Reporting accuracy and efficiency together for that configuration would make the central trade-off clearer.
We thank the reviewer for this suggestion. We now report both quantities for the exact configuration behind the headline number: c=16, 30% retention, acn=4, ur=64.
| Pixel budget | Measured FPS | StreamingBench | OVO-Bench |
|---|---|---|---|
| 90,000 | 159 | 66.52 | 51.29 |
| 200,704 | 94 | 71.20 | 52.13 |
The trade-off is now readable from a single row pair: the 90,000-pixel setting runs 65 FPS faster and gives up 4.68 points on StreamingBench, while the 200,704-pixel setting is the one behind the 134-FPS estimate.
Q1: A sustained wall-clock run of the complete pipeline under the (c=16), 30%-retention setting would be helpful. ... a serial or pipeline-disabled control would show whether the derived 134 FPS is achieved in practice.
We thank the reviewer for this request. We report the sustained run, the frame count, the elapsed time, and a serial control for the requested setting.
| Configuration | Frames | Elapsed (s) | Serial FPS | VP-overlap FPS | Four-stage FPS | Speedup |
|---|---|---|---|---|---|---|
Base 3B, 200,704 px, c=16, 30% | 2,560 | 27.4 | 52.38 | 69.09 | 93.35 | 1.78× |
Fine-tuned 3B, 200,704 px, c=16, 30% | 2,560 | 27.2 | — | — | 94.20 | — |
Fine-tuned 3B, 90,000 px, c=16, 30% | 2,560 | 16.1 | 77.80 | 114.13 | 159.30 | 2.05× |
Fine-tuned 3B, 90,000 px, c=8, s=0.25 | 1,280 | 13.8 | 58.44 | 80.67 | 92.79 | 1.59× |
For the configuration behind the 134-FPS estimate (c=16, 30% retention, 200,704 pixels), the base-3B variant completes 2,560 frames in 27.4 s at 93.35 FPS, against 52.38 FPS with the same modules in serial execution. The fine-tuned variant reaches 94.20 FPS. At 90,000 pixels the same pipeline completes 2,560 frames in 16.1 s at 159.30 FPS, against 77.80 FPS serially. The derived 134 FPS is not reached at the original budget, where the measured value is 93–94 FPS, but the coordinated pipeline does exceed 100 FPS at the lower visual budget.
Q2: Adding chunk size, streaming FPS, and query TTFT to Tables 3 and 4 ... Table 4 would also benefit from the measured token retention already reported for Table 3 ...
We thank the reviewer for this suggestion. Tables 3 and 4 use 8-frame chunks, while the headline configuration in Table 2 uses 16-frame chunks with fixed 30% token retention. We will clarify these settings in the revised captions.
Table 4 uses acn=4 and ur=32. Since ur does not affect token retention, the measured retention ratios from Table 3 also apply to Table 4:
| Backbone | s_drop | Tokens dropped | Tokens retained |
|---|---|---|---|
| 3B | 0.25 | 57.6% | 42.4% |
| 3B | 0.10 | 72.9% | 27.1% |
| 7B | 0.25 | 48.0% | 52.0% |
| 7B | 0.10 | 70.6% | 29.4% |
For the headline 16-frame configuration, measured complete TTFT is 121 ms at 90,000 pixels and 107 ms at 200,704 pixels, with 30-FPS input and queries every 10 seconds. The corresponding throughput and accuracy results are reported in W4.
Q3: Table 5(c) would be more informative if the frame-level and chunk-level rows matched the retained token count, visible temporal span, and LLM prefill latency. ... separate the effect of chunk-level dropping from the additional context or tokens retained by each configuration.
We thank the reviewer for this suggestion. As detailed in W3, we compare frame-level and chunk-level dropping under similar token budgets and the same temporal coverage. Their retention ratios differ by at most 2.6 percentage points. Chunk-level dropping improves StreamingBench accuracy by 1.88 and 1.80 points at the two thresholds, respectively. It also requires fewer prefill operations (4 versus 16), although prefill latency is not matched between the settings.
References
[1] Yao, et al. Timechat-online: 80% visual tokens are naturally redundant in streaming videos. ACM MM (2025).
[2] Zhang, et al. Flash-vstream: Memory-based real-time understanding for long video streams. arXiv:2406.08085 (2024).
[3] Chen, et al. Streamingtom: Streaming token compression for efficient video understanding. arXiv:2510.18269 (2025).
We thank the reviewer for recognizing the complementary roles of chunking, local attention, retrieval, and training-time visibility. We agree that the systems contribution requires direct execution controls and careful ablations.
We thank the reviewer for stating this concern directly. We position the contribution as the stage-wise formulation and analysis of continuous VideoLLM inference, including how reducing LLM-side work exposes front-end bottlenecks. The individual uses of CUDA streams, token dropping, and local attention are not claimed as algorithmic novelty and coordinating them changes throughput. To test that claim in isolation, we run the same modules under three execution modes (serial, preprocessing-overlapped, and four-stage) in the following study.
The table reports wall-clock throughput on one A100-SXM4-80GB and is averaged over two runs per setting.
Timing covers startup/drain, preprocessing, transfers, encoding, dropping, and LLM execution including the output projection, on 720p CPU-resident frames.
According to the results, coordination improves end-to-end throughput by 1.59–2.05× over serial execution of the same modules, showing that the gains come from stage scheduling rather than changes to the model or compression policy. Although the individual components are not algorithmically novel, ViCoStream integrates these acceleration techniques into a unified system-level framework that makes high-throughput streaming video understanding practical. More importantly, it reveals how inference bottlenecks shift across processing stages as different parts of the pipeline are accelerated.
We thank the reviewer for this question. We confirm that the 134-FPS entry is an estimate derived from stage-wise latency,
1000 × 16 / max(113, 119, 44, 42). To clarify more about ViCoStream's efficiency, we run the same 16-frame, 30%-retention configuration as a sustained wall-clock workload in the study.According to the results, the original stage-based estimate is 1.42–1.44× higher than the measured throughput, consistent with shared-GPU contention and execution overhead that the idealized estimate does not capture. Nevertheless, ViCoStream achieves 93–94 FPS at the original 200,704-pixel budget and 159 FPS at 90,000 pixels. Furthermore, the serial controls in W1 show that coordinated execution improves throughput by 1.78–2.05× using the same model and compression settings. These results confirm substantial end-to-end acceleration of ViCoStream, despite the gap from the theoretical stage-based estimate.