- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes CountRoute, a training-free agent for streaming video counting that chooses between pattern-based readout and evidence-grounded counting. It maintains query-agnostic textual and visual memory, selectively inspects relevant blocks, and switches to record-level counting when sufficient evidence accumulates. On SVCBench, CountRoute achieves 38.4 overall GPA, substantially outperforming existing streaming systems and most offline baselines.
Critical strengths
-
The paper identifies that different counting questions require different computation strategies, motivating pattern-based versus evidence-grounded readout.
-
The component study shows consistent gains from block selection, record reading, transition counting, and periodic counting, with overall GPA increasing from 20.2 to 38.4.
-
CountRoute achieves 38.4 overall GPA, compared with 32.0 for the strongest offline baseline and 18.7 for the strongest listed streaming baseline.
Critical weaknesses
-
Most components are based on existing techniques; the main contribution is their routing and composition. The paper should better justify why this constitutes a substantial methodological advance.
-
Key decisions in the routing rely on fixed thresholds. Although sensitivity studies are provided, it remains unclear how well this routing principle generalizes beyond the benchmark.
-
Although query-time inspection is reduced, preprocessing is substantial: about 3.55s per 4s window, with roughly 2.3× the image inputs of direct 64-frame answering.
Questions for authors
-
How much of the final improvement comes specifically from the routing policy, compared with simply combining the available readers?
-
Can you report full end-to-end latency, throughput, memory, and model-call costs?
-
How does the method perform on natural repetition videos rather than highly structured periodic data?
3 comments
Response to Reviewer Zcsq (Part 2 / 2)
Q3. Natural repetition beyond structured periodic data
We evaluate CountRoute on all 227 public PushupBench test clips. The periodicity gate routes 212 clips to recurrence counting, 14 to visual inspection, and one to caption-based reading. As in Table 7 of the submitted paper, MAE excludes clips with an absolute error greater than 50.
| Method | Exact accuracy | Off-by-one accuracy | MAE |
|---|---|---|---|
| Direct-64 | 10.13% | 23.79% | 6.84 |
| CountRoute (Periodic only) | 29.07% | 52.86% | 4.03 |
| CountRoute | 28.19% | 51.10% | 4.66 |
Routing some clips away from recurrence slightly lowers overall accuracy relative to forced recurrence, while the full system still outperforms Direct-64.
I thank the authors for a detailed rebuttal. Most of my concerns are resolved, except the contribution is primarily a heuristic combination of existing components. I therefore keep my rating at 3.
Response to Reviewer Zcsq (Part 1 / 2)
Thank you for the positive assessment of the problem decomposition and empirical gains. We address the routing contribution, threshold transfer, computational accounting, and natural-repetition evaluation below.
Q1/W1. Contribution and comparison with simple reader combinations
Counting a target event in a video stream depends on finding evidence of its occurrences. A common baseline uses the same procedure across queries, asking an MLLM to infer the count from frames sampled from the observed video prefix. The evidence needed to establish an occurrence, however, is event-specific. It may require extensive context or be apparent in a brief observation. A salient visual pattern can even give the count directly when its instances correspond one-to-one with event occurrences.
CountRoute uses these differences to decide how to count (Figure 1 of the revised paper). It first checks whether a dedicated pattern readout applies to the queried event. When counting requires context, CountRoute selects relevant blocks and chooses how to integrate the evidence based on how many blocks it selects. When few blocks are selected, it inspects their frames to preserve visual detail. For larger selections, it reads the continuous textual record to reduce double counting across block boundaries.
The controlled ablations in our response to Reviewer 3eQY, Q1/W1 separately quantify the contributions of specialized readouts and adaptive evidence reading. We compare two ways of averaging the same branch predictions: taking the mean of Cuts, Periodic, Inspect, and Read at every query point, or keeping the original pattern readouts and averaging Read and Inspect only on Evidence queries.
W2. Switching-threshold sensitivity and transfer
To assess switching-threshold sensitivity and transfer to new data, we built an event-counting evaluation set from EgoLife, a dataset of long-form egocentric recordings of daily life. Our set covers long events and short actions and contains 36 videos, 48 counting questions, and 384 query points. We used the official EgoLife captions to draft the event annotations, then manually verified them. All 48 questions route to Evidence under the submitted router. We vary only N on both datasets and evaluate the submitted N=3 on EgoLife without retuning.
Q2/W3. Latency and resource costs
Latency. Memory construction and query answering can run concurrently; we evaluate their latency separately. For memory construction, we use two H100 GPUs per stream: one for frame embeddings and captioning, and one for transition verification. In the full 28.0-hour SVCBench replay, memory construction keeps pace with 1× playback. From window availability to saving the memory and transition index, processing averages 2.58 s and takes at most 3.79 s per four-second window. Query answering averages 53.80 s per query point.
Model calls. Memory construction averages 2.24 visual calls and 0 text-only calls per four-second window. Query answering averages 0.84 visual calls and 1.49 text-only calls per query point.
Runtime memory and retained storage. During memory construction over the full 28.0-hour SVCBench replay, the measured peaks were 6.08 GiB of host memory (combined process PSS, including vLLM) and 64.98 GiB of GPU memory. The saved stream memory occupies approximately 2.96 MB per minute of video on disk.
Comparison with baselines using more visual input. We extend the Direct-64 baseline in Table 2 by sampling the available video prefix more densely, with frame caps of 192 and 320. All three settings use Qwen3-VL-8B and the original Table 2 prompts. We report estimated mean VLM visual input tokens per query point, counting memory construction once per video and amortizing it over the evaluated query points, with standalone frame embedding excluded.