- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes VEI, which first goes through a spare preview, lists the evidence needed for the question, and selects timestamps for each target. The lists are merged and filled or reduced to K frames, which are reloaded at higher resolution. During training, a frozen teacher model sees the correct answer and a 24-token-per-frame preview. Reverse KL trains the student's indexing trace. The trained model is used for both selection and answering. Experiments on VideoMME, LVBench, and MLVU demonstrate the effectiveness.
Critical strengths
-
The method improves on uniform sampling with the same visual input limit. At 24 tokens per preview frame, VEI achieves 66.9 average accuracy, versus 59.4 for uniform sampling. This supports using a cheap preview to decide which frames need a closer look.
-
It addresses missing frame-level labels by deriving supervision for VEI decisions from an answer-aware teacher. Privileged self-distillation transfers this guidance across preview resolutions.
Critical weaknesses
- Some questions might have small text or a small object change, which can disappear in resizing. If the preview gives no clue about that moment, the model may never select it. The paper acknowledges this limit, but does not show how much of this gap comes from missing preview information, wrong selection, or wrong answering.
- The proposed method is quite similar to zoom-in methods which first have a global view of a video and the predict a temporal span. Recent recent adaptive selectors also have similar idea. The novelty is quite incremental compared with those methods.
- There is no comparison with zoom-in methods (VideoZoomer, VideoChat-R1.5) about whether selecting segment or select frames is better.
- Missing comparisons with recent adaptive selectors. FOCUS uses uncertainty to guide sampling under a frame budget, which is cited but has no quantitative comparison. FFS also learns variable frame counts without frame annotations, though its model and training setup differ.
- The experiments only show the final accuracy, but there is no quantitative analysis on the frame selection correctness.
Questions for authors
- For the one-token VideoMME Long result, what causes the drop below uniform sampling? Is the needed information missing from the preview, visible but indexed incorrectly, or selected correctly but answered incorrectly?
- Can you provide results from other frame selections methods or zoom-in methods [1-5].
- After merging and subsampling, how often does a target lose all its requested frames? Can you report target coverage on the final evidence set?
- CG-Bench is a challenging long video understanding benchmark which contains the temporal spans for the question. The paper should also check the selection result on this benchmark instead of just reporting accuracy.
[1] Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning, CVPR 2026 [2] Flexible Frame Selection for Efficient Video Reasoning, CVPR 2025 [3] FOCUS: Efficient Keyframe Selection for Long Video Understanding, ICLR 2026 [4] VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoning, ICLR 2026 [5] VideoChat-R1.5: Visual Test-Time Scaling to Reinforce Multimodal Reasoning by Iterative Perception, NeurIPS 2025
5 comments
Response to Reviewer xot9 Part II
Q3. After merging and subsampling to the final evidence budget, how often does an observation target lose all its requested frames? Please report target coverage.
Only 0.27% of analyzable traces lose all requested frames for at least one target; overall target retention exceeds 99.9% in our offline replay. We replayed the 32-frame cap once per analyzable trace across all three datasets at 50K / 24 preview tokens per frame, using seed 42:
| Method | Questions losing at least one target | Target retention |
|---|---|---|
| Training-Free VEI | 14 / 4,712 (0.30%) | 99.91% |
| Self-Distilled VEI | 11 / 4,681 (0.23%) | 99.92% |
Among the 1,343 traces requiring truncation, 25 (1.86%) lost at least one target. Losing every requested frame for a target is uncommon, although not explicitly prevented. The per-dataset breakdown is provided in Appendix I.3, Table 22.
W5. There is no quantitative analysis of frame-selection correctness. Q4. CG-Bench provides temporal evidence spans for long-video questions. Please evaluate selection correctness on this benchmark rather than reporting only answer accuracy.
We would like to kindly clarify that the submitted Section 4.4, Figure 4, and Table 5 already quantitatively evaluate frame-selection correctness using LVBench's temporal annotations. They report evidence-window hit rate, recall, precision, and the relationship between localization and answer correctness.
The added evaluation on all 3,000 CG-Bench-mini questions also shows improved evidence recall, precision, and answer accuracy after distillation. Both methods use final 32-frame Evidence Sets at 50K / 24 preview tokens per frame:
| Method | Answer accuracy (%) | Window hit | Evidence recall | Evidence precision |
|---|---|---|---|---|
| Training-Free VEI | 50.9 | 57.1 | 34.9 | 27.4 |
| Self-Distilled VEI | 53.9 | 62.6 | 41.9 | 33.2 |
Values are percentages. Scoring uses five-second tolerance and the union of annotated clue intervals, preserving gaps. With the same evidence-frame budget, distillation improves evidence coverage and precision alongside QA. The full protocol and conditional QA results are provided in Appendix I.4, Table 23.
Please let us know if you have any further questions.
Friendly reminder from authors
Dear Reviewer xot9,
As the rebuttal deadline approaches, we would like to take this opportunity to ask whether our response has adequately addressed your concerns and whether there is any additional information or clarification we could provide.
Thank you for taking the time to review our paper and for your valuable feedback.
Best regards, The Authors
Response to Reviewer xot9 Part I
Thank you for recognizing the matched-budget gains and privileged guidance for learning evidence acquisition. We clarify the existing analyses and provide additional comparisons, localization results, and target-retention measurements below.
Our supplementary one-token LVBench analysis points to evidence acquisition as a bottleneck: Training-Free VEI hits fewer annotated windows but answers comparably well on hit cases. With very limited visual detail, it may formulate an incorrect target and repeatedly sample around it, yielding less useful evidence coverage than uniform sampling. In one inspected trace, the question asks about a phone's previous generation, but the targets focus on the phone itself. This illustrates how an early interpretation error can misdirect subsequent selection.
Because VideoMME does not provide ground-truth evidence windows, we use LVBench's temporal annotations to support this analysis, examining the two one-token settings already reported in Table 1:
Training-Free VEI answers comparably well when an evidence window is hit (57.8% vs. 55.6%), but reaches these windows much less often. Its lower overall accuracy is therefore associated with fewer evidence hits and poorer performance on non-hit cases (29.0% vs. 38.2%), both of which improve after distillation.
Our contribution is the VEI framework, which learns to turn a dense, compressed Video Preview into explicit observation targets and target-wise evidence acquisition through privileged self-distillation. Coarse global preview cues guide where to inspect, while evidence localization and budget planning jointly determine how to allocate limited high-resolution observations across complementary evidence needs. An answer-aware teacher supplies guidance for these intermediate decisions without frame-level annotations, teaching the policy what evidence to seek and how to distribute its observations.
FOCUS uses uncertainty-guided search, while FFS learns flexible frame selection. VEI makes the evidence needs explicit and adopts privileged self-distillation to train target formulation and timestamp allocation without frame-level labels. Section 4.3 tests this coupling: target-wise budget planning adds 2.4/2.6 accuracy points for Training-Free/Self-Distilled VEI beyond target formulation with one global selection list. One policy trained on MLVU's training split supports all four tested preview resolutions.
We would like to clarify that Figure 2 and Table 2(c) already compare representative evidence-acquisition approaches: predictive compression (AdaCodec), multi-agent aggregation (MACF), encoder-based selection (AKS and AdaQ), and RL-trained video-tool acquisition (ParaVT).
We now include four of the specifically requested methods; Self-Distilled VEI achieves the highest accuracy on all three datasets in this expanded comparison. Additional evaluations and reported results are summarized below; VEI uses the 50K / 24-token setting.
For FFS, we could not locate a publicly released implementation or checkpoint; its methodological distinction is discussed above. Further details are provided in Appendix I.2, Table 21.