- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes E2Q, a training-free method for selecting frames from long videos to help models answer questions under a fixed frame number constraint. The method segments the video based on the relevance of each frame to the question. It ensures that the sum of relevance scores within each segment is roughly equal. Then it selects the most relevant frame from each segment. To prevent a small number of high score frames from only a few segments, the method adjusts the weights used for the partitioning process. The paper also provides a theoretical analysis, evaluates performance across four video qa benchmarks and three models, and assesses the quality of the selected frames. The authors show E2Q improvements on four benchmarks, three downstream MLLMs, and a frame-selection benchmark.
Critical strengths
-
The algorithm description is clear and reasonable. Especially, Proposition 2 demonstrates that under the stated conditions, the upper-bound constraint ensures the sequence of nearest cumulative boundaries is strictly increasing and minimizes the boundary error in Eq. (4). And Corollary 1 shows that each interval contains at most one candidate frame that reaches the upper bound. These explain how projection and partition work together.
-
This paper not only shows the performance on final qa accuray on benchmarks, but also gives the scores on KFS-Bench for frame selection, it is important since it more directly reflects the quality of frame selection, while final qa accuracy also depends on the backbone model. And Table 3 compares different selection rules with the same candidate frames, relevance scores, backbone model and visual preprocessing. These experiments help to show the effects of partition and representative selection separately.
Critical weaknesses
-
The authors highlight one of their important contributions is "separates where to allocate the budget from which observation to retain" (L17-20, 85-91, 99-101), but there is already a close related work SLICE [1] with the same core design for long video qa. SLICE also divides the video into intervals with approximately equal relevance mass, then selects the frame with the highest raw relevance score in each interval. One main difference between E2Q and SLICE is how to adjust the frame allocation weights before segmentation. E2Q also provides more detailed partition rules and theoretical analysis, but the capped KL projection itself follows previous work (authors acknowledged it in L194-196). Without a discussion and comparison with SLICE, it is unclear what additional value these differences bring. Based on this missing related work, I am concerned about the contribution and novelty of this work.
-
This paper does not include a latency analysis. The paper only shows the algorithm complexity after scoring (which is O(KN)), but does not have any end-to-end latency analysis that includes scoring the candidate frames. The efficiency-accuracy balance is pretty important for a frame selection paper because a small performance gain may not justify a large increase in latency. Existing frame selection methods such as BOLT, MDP3, FrameOracle [2] all report the latency. Without this analysis or comparison with existing frame selection methods, I do not think the authors provide enough evidence to judge whether the performance gain is worth the inference cost.
-
Although Table 1 shows that E2Q has the highest average performance on all existing methods across all three backbone models, the improvements over the second best models are small (1.2, 0.7, and 0.3 for Qwen2.5-VL-7B, InternVL3-8B, and LLaVA-OneVision-7B). In Table 7, the authors present a statistical comparison between E2Q and WFS-SB using the Qwen2.5-VL-7B. The results for the Video-MME benchmark show no statistically significant difference (the confidence interval includes zero). Thus, I am concerned that similar non-significant results might also the same for other benchmarks when using the other two backbones (cause the average performance gains are even smaller than Qwen2.5-VL-7B). Weaknesses 2 and 3 reduce my confidence in the significance of this paper.
[1] Han et al. SLICE: An Efficient and Tuning-Free Keyframe Sampling Framework for Long-Form Video Understanding. IEEE Access 2026.
[2] Li et al. FrameOracle: Learning What to See and How Much to See in Videos. ICML 2026.
Questions for authors
-
Could you clarify what are the main contributions of E2Q over SLICE, and could you provide a direct comparison with SLICE under the same settings (like some new rows in Table 1)?
-
Could you report the end-to-end latency, including candidate scoring, with uniform sampling and strong competing selectors under the same settings? If possible, also please report the peak memory and FLOPs.
-
Could you provide the paired confidence intervals as Table 7 for E2Q vs. RIDGE under InternVL3-8B and vs. WFS-SB under LLaVA-OneVision-7B?
-
How are videos with fewer than candidate frames handled? And is the same fallback rule applied to all selectors?
2 comments
Dear Authors,
Thank you for your detailed responses and the additional experiments. My concerns about inference latency and my question about short videos have been addressed.
Regarding my concern about statistical significance, 1) I understand Appendix D.3 already showed the 95% CI on three benchmarks, but my concern was that these experiments use Qwen2.5-VL-7B as the backbone, and E$^2$Q performed best on this backbone. That is why I asked for the results for the other two backbones. And I appreciate that you provided the results in the rebuttal. 2) I agree that for different benchmarks the strongest baseline is different, but I do not agree with the claim of "comparison with an oracle". I am not picking the best baseline on a benchmark, but on the backbone, and the users can choose the best selector for the backbone they use. That is why these model comparisons are important. For the worst case results, it is an interesting analysis from Table 1 but not relevant to this concern. 3) The CI results on InternVL3 and LLaVA-OV confirmed my concern, so it seems the statistical significance is only partly shown under Qwen2.5-VL.
Regarding my concern about comparison with SLICE, I appreciate that you provided detailed explanations and experiments, and revised the paper accordingly. As I understand, the main theoretical difference from SLICE is the cap and the guarantees it provides for the discrete partition. This makes the function of cap particularly important. The 2.1 point over SLICE on cap-active questions compares two methods that also differ in smoothing and boundary selection. So I think the comparison between full E$^2$Q and its no cap variant can isolate the effect of the cap. I found the 5.8 point on the cap-active subset of LongVideoBench (95% CI [2.1, 9.6]) helpful. Could you report the same accuracy difference on the cap-active subsets of Video-MME, MLVU, and LVBench (from Table 8)? Maybe also report a paired 95% confidence interval for each benchmark. It does not require a new experiment.
Looking forward to your feedback
Dear Reviewer m9EC,
Since the discussion period will close soon, we would very much value your feedback on our responses, which address each of the points you raised. (1) On the contribution over SLICE, we clarified how E²Q differs in handling concentrated relevance and in its partition guarantees, added SLICE to the paper, and ran a direct comparison under identical conditions. E²Q outperforms SLICE in every model–benchmark setting, and the gains come mainly from questions where the two methods differ by design. (2) On efficiency, new end-to-end measurements show that E²Q adds no extra latency compared with selectors that use the same scorer, and we also report memory and FLOPs. (3) On significance, we provided the requested paired confidence intervals and a pooled test over all settings, which shows that the gains over the strongest baselines are statistically significant. (4) On videos with fewer than K candidates, we clarified that the same fallback rule is applied to all selectors.
We have revised the paper to include all of these analyses, and we believe they address your concerns and make clear what E²Q contributes beyond SLICE. Should any questions remain, we would be more than happy to address them. We sincerely appreciate your time.