- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper introduces FoveaBench, a benchmark targeting small visual evidence in long videos, and FoveaSeek, an agent combining temporal search with spatial localization and tracked close-up inspection. FoveaBench contains 1,044 questions over 454 videos, selected through model-based gates intended to identify cases where low-resolution whole-frame observations are insufficient but magnified evidence enables answering. An oracle study compares different evidence renderings. FoveaSeek predicts a temporal interval and target description, emits a bounding box, and uses SAM 2 to obtain tracked crops with contextual views. Supervised fine-tuning achieves 25.2% accuracy versus 18.2% for the backbone, while an exploratory single-seed RL run reaches 31.1%.
Critical strengths
- Important evaluation setting. Small evidence in long videos is a meaningful challenge that temporal resampling alone may not resolve. Releasing target boxes and annotated instants makes the benchmark potentially useful for studying both temporal search and spatial grounding.
- Informative oracle study. Comparing whole videos, evidence windows, individual frames, tight crops, and context-preserving tracked views demonstrates that locating evidence and presenting it effectively are different problems. The finding that a tight crop can underperform its source frame highlights the importance of context.
- Meaningful empirical gains. The supervised model improves over its own backbone by 7.0 percentage points, providing evidence that the setting is addressable. The fixed-weight observation ablations further demonstrate that performance depends on access to the tool’s returned evidence.
- Transparent reporting. The paper carefully discloses unmatched budgets, benchmark-selection effects, unavailable spatial baselines, and the preliminary status of RL. Paired statistical tests, cross-family judge agreement, a human audit, and documented training–evaluation separation strengthen the experimental presentation.
Critical weaknesses
- Benchmark validity. The admission gates do not consistently isolate spatial legibility from temporal evidence availability. For 16.5% of egocentric items, the whole-frame gate excludes the annotated evidence frame. Failure in this condition can therefore reflect missing evidence rather than insufficient resolution. Moreover, the whole-frame and magnified conditions differ in resolution, spatial guidance, and presentation, so passing the gates establishes sensitivity to these particular renderings rather than the necessity of spatial magnification alone. The human audit is useful, but covers only 100 construction candidates, uses one reviewer, and assesses legibility at a different resolution. Re-running admission with guaranteed evidence-frame coverage and matched contextual information would strengthen the benchmark’s central premise.
- Incomplete controls. No spatially aware or query-adaptive resolution baseline is evaluated. The lack of public weights for the closest agents is understandable, but simpler controls remain important: high-resolution keyframe selection, static dilated crops with context, or query-aware allocation of the same visual-token budget. The oracle rendering simultaneously changes dilation, context, frame count, tracking, and budget. Similarly, the temporal-only ablation evaluates a tube-trained policy under different observations, which measures dependence on its trained interface rather than the best achievable temporal-only alternative. The authors acknowledge these limitations, but they leave the necessity and distinctive benefit of the proposed procedure insufficiently established.
- Missing grounding evidence. Although policy-emitted boxes are central to the method, spatial localization is not evaluated quantitatively. Answer accuracy alone cannot establish whether training improves box selection, temporal search, or answer generation. This is particularly important for RL, whose explicit process reward supervises time rather than space. There is also ambiguity in the reward reference: Appendix F distinguishes generator-produced anchoring intervals from certified evidence instants, with the annotated instant inside the shipped interval for only 56.9% of egocentric items. This does not directly establish the quality of RL training windows, which needs separate validation. Aggregate localization metrics, reward ablations, and repeated RL runs would clarify what is learned.
- Limited coverage. Of the 1,044 questions, 939 concern colour, substantially narrowing the benchmark’s coverage of fine-grained perception. Both domains also come from the Ego4D/Ego-Exo4D ecosystem. The question-only construction filter does not establish that language priors are negligible: it explicitly permits abstention, and the gate model returns UNKNOWN on 982 retained items. A forced-answer text-only baseline on the final benchmark is needed to measure residual predictability. Furthermore, general-video results omit the actual Qwen3-VL-8B backbone and use unmatched configurations, so they cannot quantify the specialization cost relative to the model’s starting point.
- Presentation. The manuscript is overly narrative and repetitive, repeatedly restating the motivation and caveats while scattering essential implementation and evaluation details across multiple appendices. A concise algorithm and a clearer separation of benchmark construction, method design, and experimental findings would substantially improve readability. Despite the manuscript’s length, the decisive empirical evidence remains limited: the main results rely on one custom benchmark and one backbone, the central tool ablations use a single exploratory RL checkpoint, and several general-benchmark comparisons quote external results. Important spatial baselines, quantitative grounding evaluations, and controlled rendering ablations are missing. Consequently, the work feels preliminary, with the breadth of the narrative exceeding the completeness of the experimental validation. Explicitly acknowledging these gaps does not substitute for addressing them.
Questions for authors
- Validity: Can you re-run the whole-frame gate with the annotated evidence frame guaranteed to be included, and report results separately for the currently affected 16.5% of egocentric items?
- Controls: How does FoveaSeek compare with high-resolution keyframe selection or static multi-frame crops with context under matched total visual-token budgets? Can you separate the contributions of dilation, contextual views, tracking, and additional frames?
- Grounding: Can you report temporal coverage of certified instants, box IoU or target-hit rates, and tracking success before and after RL? How are the reward’s reference intervals validated, and how does answer-only RL compare with temporal shaping?
- Generality: What accuracy do forced-answer text-only models achieve on the released questions? Please also provide non-colour results and evaluate Qwen3-VL-8B on the general-video benchmarks under the same configuration as FoveaSeek.
Summary for the Discussion Period
We thank the AC and the reviewers for their time. This comment summarizes the discussion, and our responses under each review give the details.
What the reviews agree on. All three reviews value the problem and the benchmark. Reviewer UBnt notes "valuable, counterintuitive findings" and results that "significantly outperform the strongest baselines". Reviewer QZKp calls the benchmark "genuinely hard and interesting". Reviewer 8uL5 names an "important evaluation setting", an "informative oracle study" and "meaningful empirical gains".
Six new results. The weaknesses ask for controls and measurements. The results below answer each weakness, and they support the claims of the paper.
The revised paper (8uL5 W5). Section 5 gives the agent loop as Algorithm 1. The Limitations paragraph reports the main measurements, and the new last appendix (Appendix P, Tables 14 and 15) gives all of them. Every change is in blue. No number of the submission changes, and the section, table and figure numbers stay the same.
Discussion status. Reviewer QZKp has no further question and keeps the positive score. Reviewers UBnt and 8uL5 did not reply yet, and their reviews came before these results. Reviewer 8uL5 names three missing items: "spatial baselines, quantitative grounding evaluations, and controlled rendering ablations". Points 3, 4 and 5 give all three. We would be glad to answer any further question.