- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces DES, a training-free frame selector for multiple-choice long-video question answering. It combines question relevance, answer-option matching, visual cues, and temporal information, then selects frames using redundancy control and temporal spacing. Experiments show improvements over uniform sampling and BOLT in most tested settings.
Critical strengths
-
DES works with frozen video-language models without additional training. The evaluation covers four benchmarks and six downstream models overall, providing useful evidence of its applicability.
-
The paper includes meaningful ablations. Table 4 shows that multi-cue scoring helps when combined with diversity-aware selection, although it does not improve direct Top-K selection. The authors also report settings where DES underperforms the baselines.
Critical weaknesses
-
The advantage over closely related methods remains unclear. Comparisons mainly use Uniform and BOLT, while methods such as MDP³ already combine relevance and diversity. The ablations support the usefulness of DES’s components, but do not establish whether its discrimination score improves over simpler option-aware scoring with the rest of the pipeline fixed.
-
The computational cost and some reported results need clarification. DES improves accuracy at fixed downstream frame budgets, but also processes candidate frames at 1 FPS using CLIP. Without end-to-end timing, its overall accuracy–cost trade-off is unclear. The MLVU results also differ between Table 1 (67.2/68.8/68.3 at 8/16/32 frames) and Table 8 (67.8/68.6/68.6). This may be a reporting issue, but it needs to be reconciled to establish the exact gains.
Questions for authors
-
Can DES be compared with MDP³ using the same candidate frames, downstream models, and frame budgets? With the other cues and diversity-aware selection fixed, how does the margin-and-concentration cue compare with maximum similarity over question–option pairs?
-
What is the end-to-end runtime relative to Uniform and BOLT, including frame decoding, CLIP feature extraction, selection, and downstream inference? Are video features reused across questions, and how does this affect the cost per question?
-
Which MLVU results are correct, those in Table 1 or Table 8? Do the differences reflect reporting errors or different experimental settings?
3 comments
New MDP³ results on MLVU at 16 and 32 frames
CW1 / Q1: Comparison with MDP³
Thank you for suggesting this comparison. We evaluated MDP³ and DES as frame selectors on MLVU at two frame budgets, 16 and 32 frames per question. At each budget, each method selected the specified number of frames from the video associated with the question. LLaVA-OneVision-7B then answered the original question using the frames selected by that method. The table reports downstream question-answering accuracy.
| Frames selected per question | MDP³ accuracy (%) | Source of MDP³ result | DES accuracy (%, our experiments) | DES − MDP³ (percentage points) |
|---|---|---|---|---|
| 8 | 69.8 | Original paper | 67.8 | −2.0 |
| 16 | 67.3 | Our run | 68.6 | +1.3 |
| 32 | 67.9 | Our run | 68.6 | +0.7 |
Using frames selected by DES, LLaVA-OneVision-7B achieved higher accuracy than it did with frames selected by MDP³. The difference was 1.3 percentage points at 16 frames and 0.7 percentage points at 32 frames.
Each method retained its own frame-selection protocol. MDP³ used SigLIP to score 128 uniformly sampled candidate frames, while DES used CLIP-L/14 to score candidate frames sampled at one frame per second. We will report these results and experimental settings in the revised manuscript and identify the source of each result.
Thanks for the authors' response and the additional experiments/results , my concerns have been partially addressed, but the controlled comparison with MDP3 is yet not fully completed and also I would like to verify the promised changes will be reflected in the actual paper. I will maintain my score. Again, thanks for the authors effort!!
Response to CW1–2 and Q1–3: MDP³ comparison, discrimination-cue ablation, end-to-end runtime, and MLVU corrections.
Thank you for your comments on comparisons with related methods, computational cost, and result consistency.
CW1 / Q1: Comparison with MDP³
MDP³ reports higher accuracy than DES. We have not yet completed the requested controlled comparison using identical candidate frames, downstream models, and frame budgets. Reported accuracy (%) with LLaVA-OneVision-7B and 8 input frames is shown below.
MDP³ uses 128 uniformly sampled candidates and SigLIP; DES uses candidate frames sampled at 1 FPS and CLIP-L/14. These cross-paper results cannot replace a controlled comparison or establish why the performance gap occurs. We will add the reported results and protocol differences to the manuscript.
CW1 / Q1: Incremental value of the discrimination cue
Replacing our cue with maximum similarity over question–option pairs reduces accuracy by 1.3 percentage points (pp); removing it reduces accuracy by 1.1 pp. On Video-MME with LLaVA-OneVision-7B and 8 frames, all other scoring cues and the DAS selection procedure are held fixed.
Maximum similarity measures the strongest option match, whereas our cue also captures relative matching differences across options. This ablation supports its incremental value within DES under this setting; the cue remains a proxy for discriminative utility, not a guarantee of answer correctness. We will add this controlled ablation to the manuscript.
CW2 / Q2: End-to-end runtime and feature reuse
Across all 2,700 Video-MME questions, DES improves accuracy by 4.2 pp over Uniform and 2.0 pp over BOLT, while reducing total runtime by 18.3% and 13.0%, respectively. All methods use LLaVA-OneVision-7B, 8 input frames, the same hardware, and the same timing protocol.
Timing starts with data loading and ends at completion of downstream inference, covering the stages actually executed by each method: data loading/frame decoding, CLIP image/text feature extraction, scoring/frame selection, and downstream inference. Candidate-frame CLIP image features are not reused across questions. Mean processing time is total evaluation time divided by 2,700. These runtime results apply to the measured configuration. We will add this table and timing protocol to the manuscript.
CW2 / Q3: MLVU result consistency
The correct MLVU results are those in Table 8, including the corresponding per-task scores. The inconsistency arose from mixing results from different runs with the same experimental configuration and evaluation protocol. We apologize for this reporting oversight. M-Avg is the unweighted mean accuracy across seven tasks.
Under these settings, DES achieves higher accuracy than Uniform and BOLT at all three frame budgets. We will make the following corrections:
We appreciate your review and look forward to your response!