- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors introduce KEYSTONE, a training-free frame selector for long-video QA. It scores temporal units using the KL divergence between the leave-one-out answer distributions and an averaged centroid, and uses cached visual features to reduce computation. Experiments conducted on different models and benchmarks show reasonable gains over existing approaches, supported by a set of ablations and runtime analysis.
Critical strengths
-
The method is intuitive and simple. It does not require additional training or an external judge. Rather, it directly uses the model’s distribution to provide frame selection signals.
-
The paper includes extensive ablations, alternative selection strategies, and runtime measurements.
Critical weaknesses
-
Limited novelty. While the experiments show promising results, the proposed method is an incremental application of removal-based feature attribution.
-
Unclear practical advantage. The results in Table 8 leave me uncertain about the practical value of KEYSTONE. Specifically, on Qwen2.5-VL, directly using all 128 frames achieves essentially the same accuracy at lower latency. It is unclear when the additional selection step is worthwhile.
-
The proposed mechanisms need stronger support. Specifically, the centroid reference has a reasonable motivation, but the evidence linking it to reduced reinforcement of incorrect predictions remains limited. This leaves an important part of the method’s rationale insufficiently validated.
Questions for authors
-
Could the authors provide more insight and evidence on why the centroid reference is preferred? Particularly how it helps avoid reinforcing incorrect predictions?
-
Section 3.4 mentioned that cached masking approximates actual removal. Does this approximation affect the final accuracy compared with actually removing the frames?
-
Table 8 shows comparable accuracy and lower latency for full-resolution dense answering on Qwen2.5. Under what practical scenarios would KEYSTONE be preferred?
-
Although evaluation on the latest models is not strictly required, the evaluated models are relatively outdated. Have the authors tested more recent models to see whether the improvement persists?