- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
VideoSeeker introduces a multi-turn agentic framework for cross-video understanding. It uses three tools to replay relevant video segments, compare evidence across videos, and record findings in persistent memory. The authors construct 4,000 verified multi-turn trajectories across nine task types for supervised fine-tuning. They further introduce note-level information gain rewards based on changes in ground-truth answer likelihood. A visual grounding gate restricts positive rewards to notes that reference previously observed frames. Experiments on CrossVid, CVBench, and MVU-Eval show consistent improvements over the corresponding base models.
Critical strengths
-
Effective cross-video reasoning framework. VideoSeeker combines targeted frame retrieval, explicit cross-video comparison, and persistent evidence memory.
-
Careful training data construction. The authors construct 4,000 multi-turn trajectories using independent ground-truth generation, answer-leakage checks, and trajectory verification. These procedures improve the reliability of agentic training supervision.
-
Fine-grained RL design. The proposed information gain reward assigns credit to individual evidence notes, while the visual grounding gate discourages unsupported notes.
Critical weaknesses
-
Insufficient related works. The paper lacks direct comparisons with recent agentic video reasoning methods, such as Video-o3, Video-MTR, VideoSEAL, Open-o3-Video and VideoTemp-o3 (ICML 2026). This makes it difficult to establish the advantages of VideoSeeker over existing evidence-seeking and grounding approaches.
-
Unclear necessity of joint cross-video reasoning. The paper lacks video concatenation and independent per-video reasoning baselines. Several tasks may be solved by analyzing each video separately and aggregating the results, leaving the benefit of explicit cross-video comparison unclear.
-
Insufficient evaluation of long multi-video scalability. The paper motivates its approach through the high cost of processing multiple long videos, but lacks performance analysis across different video durations and video counts.
-
Limited gains from the proposed RL rewards. The full reward design improves over outcome-only GRPO by only 0.5–1.0 percentage points across the three benchmarks. The paper lacks statistical significance tests, reward sensitivity analysis, and direct evidence that the grounding gate improves note correctness.
Questions for authors
-
Training data reliability. What fraction of teacher-authored training samples have only partial teacher agreement? Can the authors provide a human annotation auditt?
-
Role of persistent memory. How does performance change when correct evidence notes are removed, replaced, or reordered? Can the authors quantify whether persistent memory improves reasoning beyond simply retaining the full interaction history?
6 comments
Response Part2
W3: On the Scalability of Long Multi-Video Understanding
We agree that evaluating long multi-video scenarios is important for validating the scalability of VideoSeeker. To directly address this concern, we conducted additional experiments across three benchmarks, adopting a strict criterion that every input video must be at least 7 minutes long. As shown in Table C, this yields 2,003 questions.
| Benchmark | Questions |
|---|---|
| CVBench | 17 |
| MVU-Eval | 0 |
| CrossVid | 1,986 |
| Total | 2,003 |
Table C: Distribution of long multi-video questions (each video ≥7 minutes).
We further evaluated four settings, including a Frame-Matched Direct baseline, with results aggregated equally across all 2,003 questions.
| Method | Score (%) |
|---|---|
| Official Direct | 32.48 |
| Base + Tools | 31.62 |
| Frame-Matched Direct | 32.47 |
| VideoSeeker (Ours) | 47.33 |
Table D: Performance comparison on long multi-video inputs (each video ≥7 minutes).
As shown in Table D, VideoSeeker achieves 47.33%, outperforming Official Direct, Base + Tools, and Frame-Matched Direct by 14.85, 15.71, and 14.86 percentage points, respectively. These substantial improvements demonstrate the effectiveness of our active evidence-seeking framework on long multi-video inputs, where targeted replay, explicit comparison, and persistent evidence memory enable more effective evidence accumulation than fixed-frame observations.
While further evaluations across varying video counts and computational budgets would provide a more comprehensive scalability analysis, these additional results strengthen the empirical validation of VideoSeeker on long multi-video understanding tasks. We will incorporate these findings into the revised manuscript.
W4: On the Effectiveness and Reliability of the Proposed RL Rewards
To further validate the effectiveness of our reward design, we conducted additional experiments with three independent training seeds, comparing outcome-only GRPO, note-level credit assignment without the grounding gate, and our full method.
| Method | CVBench | MVU-Eval | CrossVid |
|---|---|---|---|
| GRPO | 72.84 ± 0.26 | 65.62 ± 0.67 | 49.41 ± 0.40 |
| CA (Note) w/o Gate | 73.34 ± 0.70 | 64.85 ± 0.76 | 48.79 ± 0.59 |
| VideoSeeker-9B (Ours) | 74.61 ± 0.56 | 66.96 ± 0.49 | 50.82 ± 0.49 |
| Δ vs. GRPO | +1.77 | +1.34 | +1.41 |
Table E: Performance across three independent training seeds (mean ± standard deviation, %).
As shown in Table E, our method achieves consistent improvements over GRPO across three benchmarks, with average gains of 1.77, 1.34, and 1.41 percentage points, respectively. Furthermore, removing the grounding gate reduces performance by 1.27, 2.11, and 2.03 points, demonstrating its contribution to effective note-level credit assignment.
Importantly, multi-turn video reasoning involves progressively acquiring and refining visual evidence, as explored in VideoSearcher [5] and FrameThinker [6]. Our grounding gate is therefore designed to constrain positive information-gain rewards through verifiable visual provenance, encouraging evidence-grounded note generation during iterative reasoning.
To further assess note reliability, we employed Kimi K3 to audit all 2,060 notes generated by VideoSeeker-9B across three benchmarks, evaluating their consistency with referenced visual evidence and contribution to final answers.
| Audit Category | Count | Percentage |
|---|---|---|
| Fully supported by visual evidence | 628 | 30.49% |
| Partially supported by visual evidence | 1,350 | 65.53% |
| Contradicted by visual evidence | 82 | 3.98% |
| Notes supporting final answers | 1,717 | 83.35% |
Table F: Visual consistency and final-answer support analysis of 2,060 generated notes.
As shown in Table F, 96.02% of notes were judged to be fully or partially supported by visual evidence, with only 3.98% classified as contradicted. Moreover, 83.35% were judged to support final answers. Together with the multi-seed experiments and grounding-gate ablation, these findings provide complementary empirical evidence for the effectiveness and reliability of our grounded note-level reward design. We will incorporate these additional analyses into the revised manuscript.
Response Part3
Q1: On the Reliability of Training Data
We agree that ensuring training data reliability is essential. To directly address this concern, we analyzed teacher agreement across all 4,000 SFT training samples and conducted an additional human validation study. Specifically, our pipeline employs two teacher models, Kimi K3 and Kimi K2.6, for independent verification. Samples are classified as Full Consensus when both teachers pass verification and Half Consensus when only one succeeds.
| GT Source | Total | Half Consensus | Ratio |
|---|---|---|---|
| Constructed | 1,226 | 74 | 6.04% |
| Label-anchored | 1,399 | 133 | 9.51% |
| Teacher-authored | 1,375 | 96 | 6.98% |
| Total | 4,000 | 303 | 7.58% |
Table G: Teacher agreement statistics across different GT sources.
As shown in Table G, only 6.98% of teacher-authored samples exhibit partial agreement, while 92.43% of the entire training set achieves full consensus. To further assess annotation reliability, we conducted a human audit of all 303 Half Consensus samples, verifying their ground-truth answers against the corresponding video evidence.
| Audited Samples | Correct | Incorrect | Accuracy |
|---|---|---|---|
| 303 | 294 | 9 | 97.03% |
Table H: Human validation results on Half Consensus samples.
As shown in Table H, 294 out of 303 samples (97.03%) were confirmed correct through human validation, with only 9 problematic samples identified. These results demonstrate that partial teacher agreement does not necessarily indicate incorrect annotations and provide additional evidence for the reliability of our multi-stage verification pipeline. We will incorporate these statistics and human validation results into the revised manuscript.
Q2: On the Role of Persistent Evidence Memory
We agree that evaluating whether persistent evidence memory provides benefits beyond retaining the full interaction history is important. To directly address this concern, we conducted an additional intervention study on all 872 questions from CrossVid Free-form QA (FF/CCQA), which require integrating evidence across multiple videos.
Specifically, we evaluated four settings by retaining, removing, randomly replacing, or shuffling the evidence notes, while keeping the remaining interaction history unchanged. The results are summarized in Table I.
| Setting | Original Notes | Removed | Replaced | Shuffled |
|---|---|---|---|---|
| Accuracy (%) | 33.64 | 27.87 | 28.21 | 28.81 |
Table I: Performance under different note interventions on CrossVid FF (872 questions).
As shown in Table I, removing evidence notes reduces performance from 33.64% to 27.87%, a drop of 5.77 percentage points. Randomly replacing or shuffling notes also decreases performance by 5.43 and 4.83 points, respectively.
Importantly, these performance drops occur even when the remaining interaction history is preserved, demonstrating that evidence notes provide additional benefits beyond simply retaining previous observations. These results highlight the importance of structured evidence memory in preserving and integrating relevant information throughout multi-turn cross-video reasoning. We will incorporate these additional findings into the revised manuscript.
References
[1] Zeng X, Zhang Z, Zhu Y, et al. Video-o3: Native Interleaved Clue Seeking for Long Video Multi-Hop Reasoning[C]//Proceedings of the International Conference on Machine Learning. 2026.
[2] Xie Y, Chen T, Ge Z, et al. Video-MTR: Reinforced Multi-Turn Reasoning for Long Video Understanding[C]//Proceedings of the International Conference on Machine Learning. 2026.
[3] Meng J, Li X, Wang H, et al. Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidence[C]//Proceedings of the International Conference on Machine Learning. 2026.
[4] Liu W, Wang Y, Ma S, et al. VideoTemp-o3: Harmonizing Temporal Grounding and Video Understanding in Agentic Thinking-with-Videos[C]//Proceedings of the International Conference on Machine Learning. 2026.
[5] Gao Z, Bao Y, Peng J, et al. VideoSearcher: Empowering Video Deep Research with Multi-Tool Agentic Reasoning via Reinforcement Learning[C]//Proceedings of EMNLP. 2026.
[6] He Z, Qu X, Li Y, et al. FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlighting[C]//International Conference on Learning Representations. 2026.
We deeply value your detailed feedback and the opportunity to substantiate our methodological choices with further analyses. These suggestions have been instrumental in improving the clarity and rigor of our revised manuscript.
Best regards, The Authors of VideoSeeker
Response Part1
Dear Reviewer AzCm,
We greatly appreciate the care and attention you devoted to reviewing our work. Your constructive comments prompted us to conduct additional comparisons and controlled evaluations, which help clarify the contributions of VideoSeeker. We address your concerns point by point below.
W1: On Comparisons with Recent Agentic Video Reasoning Methods
We agree that comparisons with recent agentic video reasoning methods are important for validating our approach. To directly address this concern, we conducted additional experiments with four representative methods, including Video-o3 [1], Video-MTR [2], Open-o3-Video [3], and VideoTemp-o3 [4], across three cross-video understanding benchmarks.
Table A: Comparison with recent agentic video reasoning methods (%).
As shown in Table A, VideoSeeker consistently outperforms all four methods across the three benchmarks. Notably, VideoSeeker-9B surpasses the strongest competing results by 21.29, 14.58, and 11.70 percentage points, respectively, achieving an overall average of 63.71%, compared with 47.78% for the strongest competing method.
We attribute these improvements to our dedicated training for active evidence seeking and cross-video evidence integration. Unlike methods primarily designed for single-video reasoning, VideoSeeker jointly learns targeted replay, explicit cross-video comparison, and persistent evidence recording. Furthermore, as shown in Table 1 of the manuscript, simply equipping Qwen3.5-9B with the same tools yields only a marginal improvement from 50.33% to 51.14%, whereas our agentic training achieves 63.71%. These results highlight the importance of learning effective tool-use strategies rather than merely providing tool access. We will incorporate these additional comparisons into the revised manuscript.
W2: On the Necessity of Joint Cross-Video Reasoning
We agree that distinguishing joint cross-video reasoning from independent per-video analysis is important for validating our approach. We first clarify that our Official Direct baseline already processes frames from all input videos jointly within a single context. To further address this concern, we conducted an additional experiment using Independent Per-video Reasoning + Aggregation, where each video is analyzed separately before aggregating the results.
Table B: Comparison of joint-input reasoning, independent per-video reasoning, and VideoSeeker.
The independent baseline covers 9,006 of 9,015 CrossVid samples.
As shown in Table B, independent per-video reasoning underperforms joint-input direct reasoning across all three benchmarks. More importantly, VideoSeeker surpasses the independent baseline by 13.98, 14.51, and 18.60 percentage points, respectively.
Furthermore, our existing tool ablation (Table 4 of the manuscript) provides complementary evidence for explicit cross-video comparison. Adding the Compare tool alongside Choose Frames and Note improves performance by 0.80, 1.45, and 0.88 points across the three benchmarks, respectively. These results demonstrate that our improvements stem not merely from combining multiple videos, but from actively acquiring, comparing, and integrating cross-video evidence through multi-turn reasoning. We will incorporate these additional results and clarifications into the revised manuscript.