- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
Authors propose a training-free query-dependent frame selection framework for long video understanding. They first feed uniformly sampled input frames to a pre-trained VLM and select good frames based on predicted answer alignment against image patch embeddings. This is calculated with normalization against model context prior to predicted answer tokens. Each following round adjusts the frame selection based on the parts of the video that result being most informative in the previous round.
This process is applied iteratively, and stopped when KL divergence between successive distributions is below a threshold. A final VLM call answers the question using the selected frames. Authors evaluate on five video QA benchmarks and demonstrate consistent improvements and strong results.
Critical strengths
- Interesting approach for frame selection without explicit key-frame supervision.
- Extensive evaluations across benchmarks to establish SOTA performance.
- Thorough ablations and comparisons against alternate variants of method.
Critical weaknesses
-
Compare against highly similar prior work ReQuest: Rethinking-based Question-Aware Frame Selection for Long-Form Video QA from ECCV 2026. They build a similar query-dependent frame selection approach using model outputs requiring no key-frame supervision. Clearly discuss novelty of "activation-based frame scoring" contribution over such prior work.
-
Applicability in domains where base model performance is weak remains unclear. Consider discussing significance of base model accuracy for this method to work. Particularly, how important is base model accuracy for iterative refinement to provide meaningful improvements?
-
Generality of interpolation for videos with significant scene changes could be separately evaluated to further establish methods boundaries of operation.
-
Presentation could be much improved. Figures are too complex and very hard to read/understand.
-
Minor: Please fix citations to use \citep when appropriate.
Questions for authors
Interesting paper with clear contributions and strong results. Please clarify novelty against ReQuest.
We thank the reviewer for the constructive follow-up suggestions.
Regarding the definitions, which was commented by Reviewer TLc7 (rating of 3: weak accept), we have responded to that reviewer and also have uploaded a revised manuscript clarifying the duplicate-frame handling in Lines 215–216 and explicitly defining rm and w(am) in Line 257.
Regarding the generalization of KL-based τ stopping, we would like to make two important notes about QSTAR configuration:
First, KL divergence measures sampling-distribution stability rather than directly guaranteeing answer accuracy. Below we provide additional round-wise ablations on the three benchmarks, following the request by this reviewer zqXa:
The results for these three benchmarks are consistent with the two other benchmarks provided in our initial rebuttal. All five benchmark results consistently demonstrate that this KL τ stability criterion provides an effective early-stopping signal, achieving accuracy close to the best fixed-round settings with fewer rounds on average.
Another reason that accuracy does not necessarily increase monotonically with additional rounds, as observed by Reviewer zqXa, is due to the fact that newly sampled frames may change the model's prediction outcome, which is expected. Hence, the use of τ KL divergence to measure and control the sampling-distribution stability is an important convergence optimization.
Secondly, it is also important to note that QSTAR sets the same τ=10−2 across all five benchmarks without dataset-specific tuning. The threshold was selected to balance refinement quality and computational cost. As shown in Table 1 (page 8), even with this uniform KL τ setting, QSTAR consistently improves backbone accuracy by 3.3 ~ 9.8 points, and Table 4 further demonstrates improvements across different VLM backbones using the same τ threshold.
We sincerely hope that the these additional experimental results address the reviewer's concern about τ-KL generalization.
We once again thank Reviewer zqXa for the constructive comments.
The authors