- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents DyQCA, a training-free, model-agnostic keyframe selection framework for long-video understanding that replaces uniform temporal segmentation with dynamic, content-driven segmentation. It detects boundaries via adjacent-frame semantic difference combined with change-acceleration and change-rate signals, then allocates the frame budget across segments by query relevance and content deviation and performs anchor-centric selection within each segment. On four long-video benchmarks with three Video-LLM backbones under a 64-frame budget, it improves over uniform sampling across the board and surpasses much larger models. However, its improvement over the strongest prior baseline is small and inconsistent, and only the segmentation stage is new relative to prior work.
Critical strengths
S1. A simple, practical, and genuinely plug-and-play framework with clear gains over the ubiquitous uniform-sampling default. DyQCA is training-free, model-agnostic, and requires no architectural change; it improves over uniform sampling consistently across all four benchmarks and all three backbones, and with a small frame budget it surpasses much larger proprietary and open models on several benchmarks. For the widely used "select-then-answer" pipeline, this is an immediately usable improvement, which is the main basis for a positive view of the paper.
S2. The motivation is well articulated and the breakdown analyses support the claimed mechanism. The paper explains why uniform segmentation misallocates frame budgets (splitting events, merging unrelated content), and the evidence aligns with this story: the largest gains appear on the long-video subsets and on temporal-ordering tasks, i.e., exactly where semantic non-uniformity should hurt uniform segmentation most. The authors also report where the method does not help, which adds credibility.
Critical weaknesses
W1. The contribution is incremental over prior work. The budget-allocation stage (query relevance + content deviation) and the anchor-centric keyframe selection stage are taken directly from the prior QCA framework; the sole new component is threshold-based dynamic segmentation using simple difference/acceleration/rate signals. The idea that segment boundaries should align with semantic transitions is intuitively appealing, but as realized here it is a straightforward, heuristic substitution within an existing pipeline rather than a fundamentally new mechanism, which limits the paper's novelty.
W2. The new component does not deliver a consistent or clearly evidenced advantage over the prior pipeline it modifies. Against QCA, DyQCA is worse on most benchmarks with LLaVA-Video-7B, trails on LVBench with Qwen3-VL-8B, and wins by only small margins where it does win; the paper's own ablation further shows that the proposed boundary criterion is not the best choice on every dataset. With differences of this size, the absence of repeated-run statistics or confidence intervals makes it difficult to separate a real gain from noise, and the abstract's claim of "consistently outperforming recent keyframe selection baselines" is not supported by its own main table.
Questions for authors
- Comparison with QCA. On LLaVA-Video-7B, DyQCA is below QCA on most benchmarks, and also trails it on LVBench with Qwen3-VL-8B (Table 1). Under what conditions does dynamic segmentation hurt, and should the headline claim be qualified accordingly? What distinguishes the benchmarks where it helps from those where it does not?
- Hyperparameter protocol. How are the boundary thresholds and the segment count chosen for each benchmark? Do all reported results use a single frozen setting across datasets? Please report sensitivity (e.g., performance spread across reasonable threshold ranges) so that the gains can be separated from per-dataset tuning.
- Statistical reliability. Are the small differences over QCA stable across seeds/runs? Please provide confidence intervals or repeated-run statistics for the main comparisons.
- Efficiency. What is the end-to-end selection latency (including 1-FPS feature extraction and segmentation) per video, compared with uniform sampling and QCA? This matters for the claimed practical deployment value.
- Claim-evidence alignment. The abstract promises consistent improvement over recent selection baselines, while Table 1 shows mixed outcomes against QCA. Could you align the claims with the table?
- Reproducibility. No code or data release is mentioned. Given that all components are training-free, do the authors plan to release the segmentation/allocation code?
- Presentation. The budget-allocation and keyframe-selection sections largely restate prior work. Could you mark more explicitly which parts are new, to help readers locate the contribution?
Thank you for your detailed reviews and insightful questions. Please see our point-by-point responses.
[W1, Q7] As described in our paper, the budget allocation and the anchor-centric keyframe selection are taken from QCA (Peng et al., 2026). The new component is the training-free, threshold-based dynamic segmentation using Dt, At, and Rt, together with its interface to the existing pipeline. Our contribution is a simple, training-free segmentation substitution plus an empirical characterization of when it helps.
[W2, Q1, Q5] We will revise the abstract’s ``consistently outperforms’’ to state that DyQCA is competitive across all benchmarks and backbones, yields clear gains on MLVU, and is comparable or slightly lower elsewhere. As demonstrated in Fig.3, gains concentrate on queries whose answer depends on a short, localized event, e.g., a flag, a biscuit, an egg, where dynamic boundaries let the budget concentrate on the segment containing that evidence. Declines concentrate where evidence is distributed over long ranges or events are already close to uniformly spaced, so uniform segmentation provides a reasonable temporal prior and occasional over-segmentation of stable intervals offsets the benefit.
[Q2] S is fixed in all experiments and set identically to QCA’s configuration. The threshold rule in Eq.6 does not fix the number of segments by itself. For each video, we compute the three signals Dt, At and Rt for all frames and sort each signal independently. The three thresholds are determined as follows:
λ3 is set to the 85th percentile of Rt for that video, obtained via np.percentile. λ1 and λ2 are determined by a joint grid search. We sweep both thresholds over the percentile range from 50% to 98% in steps of 2, i.e., over np.arange(50,100,2) or each signal, and evaluate every (λ1,λ2) combination. For each combination, frames satisfying Eq. (6) and yielding exactly the target S−1 boundaries are taken as candidate boundaries, and a boundary score is computed. The combination with the highest score is selected as the final (λ1,λ2) for that video. γ and η are fixed at 0.5, inherited unchanged from QCA.
We will state this protocol explicitly in revision.
[Q3] With a fixed random seed and unchanged frame features, the segmentation result for a given video is identical across runs. Consequently, seed-based repeated runs would report zero variance. The relevant source of variability is instead the evaluation set, not the method.
[Q4] On Video-MME, we measured per-stage throughput: BLIP feature extraction 29 FPS, dynamic segmentation 4056 FPS, QCA allocation/keyframe selection 11430 FPS, and MLLM inference 5 FPS. Dynamic segmentation is about two orders of magnitude cheaper than feature extraction and three orders cheaper than MLLM inference, so its added cost over uniform segmentation is negligible relative to the end-to-end pipeline.
[Q6] All components are training-free, and we will release the code for dynamic segmentation, budget allocation, and keyframe selection, together with the S setting and hyperparameter search procedure described above.