- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes BEPA for evidence selection with a frozen video answerer. It evaluates two independent settings: CORE refines BOLT’s frame selection through option-centered matching scores and a coverage objective, while an agent-guided approach uses prediction disagreement to generate retrieval queries for one additional observation. Experiments are conducted on VUDG using VideoLLaMA3-7B.
Critical strengths
- The matched CORE–BOLT comparison fixes the answering model, prompt, and frame budget, and reports a positive paired confidence interval alongside repairs and harms.
- The answer-preserving call-omission condition is mathematically justified.
Critical weaknesses
-
Insufficient independent validation. The evaluation is explicitly development-exposed, and Appendix A states that the existing data splits are not independent validation sets. The reported bootstrap intervals therefore do not address possible selection bias during development. Evidence is also limited to one benchmark and one video answerer.
-
The distinctive mechanism is insufficiently validated. There are no controlled ablations separating option centering, peak normalization, the coverage objective, anchor protection, and the swap budget. Consequently, the modest improvement over BOLT supports the complete configuration but does not establish which proposed design provides the benefit. The advantage over WFS-CLIP remains inconclusive, with a confidence interval crossing zero.
-
Essential reproduction details are missing. Appendix A explicitly states that the C1 environment, public planner/CLIP snapshot mappings, and complete processor settings remain undocumented. The full planner prompt and local WFS-CLIP configuration are also insufficiently specified, preventing independent reproduction of the reported results.
-
The agent contribution is not adequately isolated. Global-coverage and stem-retrieval controls are useful, but the paper does not establish whether the gain requires prediction-disagreement information rather than generic question-and-option rewriting. Table 3 lacks paired uncertainty estimates for the principal agent comparisons, and the reported base-call counts require reconciliation before the execution controls can be considered fully understood.
Questions for authors
-
Which configurations and decisions were selected using the evaluation data, and what evidence from data not used during development confirms the gains?
-
Under identical candidate pools and frame budgets, what are the individual contributions of centering, normalization, and option coverage compared with uncentered option-aware retrieval?
-
Does the agent still improve performance when prediction summaries are removed, while preserving the same trigger population and additional-observation budget?
Generally, the paper's quality cannot meet the requirements of this conference, and many questions are left unexplained. Therefore, I recommend a reject to this paper.