acceptodds
Under review as a conference paper at ICLR 2027

VideoResearchAgent: Training Agents for Open-Web Video Research via Sim-to-Real Reinforcement Learning

Abstract

Existing deep research agents are designed primarily for text- and image-based web sources, while video reasoning systems typically assume that relevant videos are provided in advance. We study open-web video research, where an agent must autonomously discover relevant videos, navigate their temporal content, and ground answers in visual evidence. Training such agents at scale is challenging as live video interaction is slow and unreliable, whereas fixed local simulation can induce retrieval-specific shortcuts that fail to transfer to the open web. We introduce VideoResearchAgent, a scalable training framework to address these challenges. First, we introduce controllable task synthesis pipeline to synthesize multi-hop research tasks from timestamped visual evidence while filtering text-only shortcuts. Second, we build a field-aligned local video simulator that preserves deployment-facing search and watch interactions while accelerating video search by 34.5–64.6×. Third, we introduce Retrieval-Domain-Randomized GRPO (RDR-GRPO), which diversifies candidate rankings, distractors, metadata, and result structure during training to reduce overfitting to simulated retrieval. On Video-BrowseComp, the VideoResearchAgent trained using Qwen3.5-4B achieves accuracy, comparable to Gemini-3-Flash-Preview, while reducing cumulative API-token consumption by relative to the untrained model. Together, these results establish an accurate and efficient training recipe for open-web video research.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multimodal Agents, Open-Web Video Research

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_HTACOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the training of agents for open-web video research, where an agent must discover relevant videos, navigate their temporal content, and answer questions using grounded visual evidence. The paper identifies an important practical bottleneck: training directly with live video-search and video-access services is slow, expensive, and unreliable, whereas a conventional local simulator is stable but may fail to reproduce the variability and failure modes of the deployed environment. Policies trained in such a simplified simulator may therefore exploit simulator-specific shortcuts and transfer poorly to the real web.

The proposed VideoResearchAgent framework contains three main components. First, it synthesizes multi-hop video-research tasks from visual evidence and filters questions that can be answered through text-only shortcuts. Second, it constructs a field-aligned local video simulator that preserves the policy-facing search and video-access interfaces of the live environment. Third, it introduces Retrieval-Domain-Randomized GRPO (RDR-GRPO), which perturbs result rankings, injects hard negatives, corrupts displayed metadata, varies result counts, and occasionally returns empty results. These perturbations simulate not only the functionality of the online tools but also their uncertainty, errors, and structural variability.

The method is evaluated on Video-BrowseComp. For Qwen3.5-4B, supervised fine-tuning improves accuracy from 30.00% to 36.67%, and RDR-GRPO further improves it to 40.48%. The trained policy also uses fewer tool calls and 74.9% fewer cumulative API tokens than the untrained model. Additional analyses suggest that retrieval-domain randomization improves transfer from the local simulator to the live web and reduces repetitive or non-convergent interactions.

Critical strengths
  1. The paper identifies an important and practically consequential training bottleneck. Live video search, download, and inspection are slow and unreliable, but a clean and fixed local environment does not adequately represent deployment conditions. This tension is central to scalable training of open-web video agents and is not resolved by simply reproducing the functional API of a search service.

  2. The simulator models deployment-time uncertainty rather than only interface functionality. The most valuable aspect of the work is that it explicitly simulates ranking changes, hard distractors, metadata corruption, variable result counts, missing gold results, and empty responses. These perturbations encourage the policy to reformulate queries, inspect alternative candidates, and recover from failures. The resulting improvements in live-web accuracy, interaction convergence, tool-use efficiency, and API-token consumption indicate meaningful practical value even though the underlying algorithmic ideas are relatively incremental.

Critical weaknesses
  1. The algorithmic novelty is limited. High-fidelity simulation, domain randomization, synthetic task generation, supervised warmup, and GRPO are established ideas. The primary contribution is their careful adaptation and integration for open-web video research rather than a fundamentally new learning algorithm. I consider this acceptable given the practical importance of the problem, but the paper should characterize the contribution as a video-specific training framework and empirical recipe rather than a broadly novel reinforcement-learning principle.

  2. The evidence for sim-to-real generalization could be stronger. Video-BrowseComp contains only 210 tasks, and the key fixed-retrieval versus RDR-GRPO transfer analysis is conducted on a randomly sampled 30% subset of 63 tasks. The comparison emphasizes fitted accuracy trends over training checkpoints, but repeated evaluation on the same small subset may not provide a sufficiently independent estimate of generalization. The main trained result also focuses on one policy backbone. These limitations reduce confidence in how broadly the proposed randomization strategy transfers across models, web conditions, and video domains, although they do not invalidate the central result.

Questions for authors
  1. Which retrieval perturbations are most important? Please provide an ablation separating rank intervention, hard-negative injection, metadata corruption, and structural noise/empty results. This would clarify whether the gain comes from general exposure to noise or from a particular deployment failure mode.

  2. How much of the final improvement comes from the synthesized task data, SFT trajectory quality, and RDR-GRPO respectively? The reported Base → SFT → RDR-GRPO progression is useful, but an additional control using the same SFT initialization followed by fixed-retrieval GRPO on the full evaluation set would better isolate the contribution of retrieval randomization.

1 comment

We thank Reviewer HTAC for the insightful review. We are grateful for the precise characterization of our core contribution: identifying the critical training bottleneck for open-web video agents and addressing it via a simulator that explicitly models deployment-time uncertainty rather than mere functional APIs. The constructive suggestions on conceptual positioning and experimental controls are invaluable. Below, we address each point in detail.


W1: Clarification on Algorithmic Novelty and Paper Positioning

We agree with your perspective. Our core contribution lies in designing a video-specific training framework and empirical sim-to-real recipe for open-web video research—a critical, high-impact domain that previously lacked a viable, scalable training methodology due to the severe latency and fragility of live web environments. As you noted, merely reproducing the functional API of a search backend is insufficient because agents easily exploit static retrieval shortcuts. Our key contribution is the disciplined adaptation and end-to-end integration of grounded task synthesis, accelerated simulation, and retrieval domain randomization, enabling a compact 4B model to match the accuracy of Gemini-3-Flash-Preview while slashing cumulative API token consumption by 74.9%. We will reposition the paper's claims around this practical training framework and empirical recipe.


W2 & Q2: Disentangling Gains Across Synthetic Tasks, SFT Quality, and RDR-GRPO on the Full Benchmark (210 Tasks)

We evaluated the entire progression across all 210 tasks on Video-BrowseComp under the same 85-step budget:

Training Progression StageFull Benchmark Accuracy (210 tasks, 3 runs)Incremental Delta
1. Untrained Base (Qwen3.5-4B)30.00%—
2. Synthetic Tasks + SFT Warmup36.67%+6.67 pp
3. SFT + Fixed-Retrieval GRPO (85 steps)35.60 ± 0.99%-1.07 pp (vs. SFT)
4. SFT + RDR-GRPO (Ours) (85 steps)40.63 ± 0.99%+5.03 pp (vs. Fixed GRPO)+3.96 pp (vs. SFT)

This matched-budget control provides a clean decomposition of our framework's gains:

  1. Synthetic Tasks & SFT (+6.67 pp): Establish foundational tool orchestration and query execution.
  2. RDR-GRPO (+5.03 pp over Fixed GRPO): Acts as the decisive sim-to-real bridge. Without RDR, reinforcement learning in a static simulator actually regresses below SFT due to shortcut memorization.

Q1: Dissecting the Impact of Individual Retrieval Perturbations

To evaluate which retrieval perturbations drive policy robustness, we conducted single-factor and leave-one-out replays across 500 training tasks under identical local corpus settings:

Perturbation ConditionGold Visibility Rate in Top-5Gold Rank-1 RateMRRPerturbation Nature & Diagnostic Role
Full RDR0.8910.4860.648Combined stochastic deployment condition
Rank Intervention Only0.8960.4870.658Directly reshapes candidate ranks; pushes gold down
Hard Negatives Only0.9360.6510.793Injects distractors into 24% of display slots
Metadata Corruption Only0.9360.6510.793Corrupts 23% of candidate titles
Structural Noise Only0.9300.6470.788Varies result count kk and injects empty results (0.7%)
Full RDR w/o Rank Intervention0.9300.6470.788Removes candidate rank shuffling
Full RDR w/o Hard Negatives0.8910.4860.648Removes adversarial distractors

Key Findings:

  1. Rank intervention pushes target videos down. It is the only perturbation that moves the target video's rank (rank-1 rate drops from 0.651 to 0.487). This teaches the agent not to simply click the first result, but to scroll down and try new search terms when top results are unhelpful.
  2. Hard negatives and messy titles force real video checking. In isolation, distractors and title noise do not move the target video's rank, but they surround it with misleading look-alikes. This stops the agent from guessing based on title words alone, forcing it to actually open and inspect video frames.
  3. How they work together. Changing ranks forces the agent to search deeper, while confusing distractors force it to check the actual video content. Together, they prevent the agent from taking easy shortcuts in the simulator.
Reviewer_ZLVTOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

VideoResearchAgent studies open-web video research, where an agent must discover relevant videos, locate informative moments, and answer questions using the resulting evidence. The proposed training framework combines synthetic video-grounded research tasks, supervised fine-tuning on teacher trajectories, and reinforcement learning in a local video environment. The local simulator preserves the deployed search and viewing interfaces while reducing the cost of video interaction. Retrieval-Domain Randomization perturbs search rankings, distractors, metadata, and result structure during GRPO training to improve transfer to live retrieval. On Video-BrowseComp, the trained Qwen3.5-4B model improves from 30.00% to 40.48% accuracy, with a reported 74.9% reduction in cumulative API-token consumption.

Critical strengths

I like that the paper tackles finding the right video as well as reasoning within it. The simulator makes this expensive interaction practical for training, and the same-backbone results show a useful improvement. The behavioral analysis is also informative: the model removes many repeated calls rather than merely answering sooner. Improving accuracy while cutting token use by 74.9% is a worthwhile result.

Critical weaknesses
  1. The contribution of retrieval randomization is not yet clear. The main table compares SFT against SFT plus RDR-GRPO; the direct comparison with ordinary GRPO uses only 63 questions. Given the checkpoint variation in Figure 4, I would like to see the final fixed-retrieval baseline on the full benchmark before attributing the transfer gains to RDR.

  2. The paper shows better search control more clearly than better visual grounding. Since the tools also return transcripts and other text, more video calls do not necessarily mean the answers depend on frames. The training-task filters help, but a visual-input ablation at evaluation would make this claim more convincing.

Questions for authors
  1. What does fixed-retrieval GRPO achieve on all 210 questions at the same training step as the final RDR-GRPO model? How stable is the difference across runs or repeated live-web evaluations?
  2. How much accuracy is lost if frames and visual-grounding outputs are removed while search metadata and transcripts remain available? Does this dependence on visual input increase after training?
3 comments

We thank Reviewer ZLVT for the encouraging assessment and insightful feedback. We are delighted that the reviewer appreciated our dual focus on finding relevant videos and reasoning within them, the training efficiency brought by our local simulator, the 74.9% reduction in API token consumption, and our behavioral analyses. Below, we provide the two targeted experiments addressing the reviewer's concerns: (1) the full-benchmark comparison between fixed-retrieval GRPO and RDR-GRPO, and (2) an evaluation-time visual-input ablation assessing pre- vs. post-training visual dependence.


W1 & Q1: Fixed-Retrieval GRPO vs. RDR-GRPO on All 210 Questions at Matched Training Budget

To conclusively isolate the contribution of retrieval randomization on the entire evaluation suite, we evaluated both Fixed-Retrieval GRPO and RDR-GRPO at the same final budget of 85 training steps across all 210 tasks of Video-BrowseComp:

Model / ConfigurationTraining BudgetFull Benchmark Accuracy (210 tasks, 3 runs)
Untrained Qwen3.5-4B Base—30.00%
SFT Warmup Baseline—36.67%
Fixed-Retrieval GRPO85 steps35.60 ± 0.99%
RDR-GRPO (Ours)85 steps40.63 ± 0.99%
  • Performance Delta: At step 85, Fixed-Retrieval GRPO achieves only 35.60%, which is actually slightly lower than the SFT initialization (36.67%). In contrast, RDR-GRPO achieves 40.63%, representing a +5.03 percentage point gain over Fixed-Retrieval GRPO and +3.96 pp over SFT.
  • Stability and Generalization: In a static local simulator, fixed retrieval rankings allow the RL policy to memorize deterministic search shortcuts that completely break when deployed on live web engines. RDR-GRPO explicitly exposes the agent to ranking perturbations, missing targets, and hard distractors, driving robust sim-to-real transfer. Paired task-level evaluation shows RDR-GRPO achieves 23 positive flips vs. only 13 regressions over Fixed-Retrieval GRPO (net gain: +10 tasks).

W2 & Q2: Visual-Input Ablation at Evaluation Time & Visual Dependence Shift

To rigorously test whether our agent's improvements stem from genuine visual grounding rather than mere text/transcript search control, we conducted an evaluation-time visual ablation across all 210 tasks.

Ablation Setup: When frames are withheld, watch_video proceeds through normal video downloading, decoding, and transcript extraction, but visual image tokens are replaced with a [frames_withheld: N] placeholder. visual_grounding returns an informative fallback message. The action space and schema remain completely unchanged.

1. Evaluation Accuracy With vs. Without Visual Frames:

Evaluation ConfigurationFull Benchmark Accuracy (All 210 tasks)Accuracy Drop from Withholding Frames
Qwen3.5-4B Base (with frames)30.00%—
Qwen3.5-4B Base (frames withheld)18.57%-11.43 pp
RDR-GRPO RL Agent (with frames)40.48%—
RDR-GRPO RL Agent (frames withheld)26.67%-13.81 pp

2. Tool-Invocation Composition (Mean Calls per Task and Proportions):

Model Conditionweb_searchsearch_youtubewatch_videovisual_groundingTotal Tool Calls
Qwen3.5-4B Base42.45 (96.1%)0.26 (0.6%)1.29 (2.9%)0.15 (0.3%)44.15
RL Agent (frames withheld)6.21 (59.4%)2.37 (22.7%)1.39 (13.2%)0.50 (4.7%)10.47
RL Agent (with frames)3.81 (60.3%)1.75 (27.7%)0.73 (11.6%)0.03 (0.4%)6.32

Key takeaways directly answering your questions:

  1. Accuracy is heavily dependent on visual frames. Removing visual inputs causes the RL agent's performance to plummet from 40.48% to 26.67% (a massive 13.81 pp drop). Because tool metadata and transcripts remain fully accessible, this substantial drop proves that answers strictly depend on frame pixels rather than text shortcuts.
  2. Dependence on visual input increases after training. The performance penalty from removing visual frames increases from 11.43 pp on the base model to 13.81 pp on the trained RL agent.
  3. Behavioral shift towards active visual evidence seeking. As shown in the tool-use table, the untrained base model passively executed 42.45 web searches per query (96.1% of all actions) while almost ignoring videos. Our post-trained agent proactively navigates to video content (search_youtube and watch_video account for nearly 40% of actions) and efficiently extracts answers from visual frames with minimal token overhead.

I maintain my original score. The rebuttal substantially addresses my main concerns by providing controlled comparisons for multi-span retrieval and reflection supervision, as well as multi-seed and multi-checkpoint evaluations for SEPO. The additional process-level analysis also provides useful evidence that SEPO improves evidence acquisition and error recovery. However, the gains from SEPO remain relatively modest, and multi-span retrieval does not consistently improve performance across all benchmarks (e.g., WorldSense). The process-level evaluation is also still somewhat limited in scope.

Dear Reviewer,

Thank you for taking the time to review our rebuttal.

We noticed that some points in your latest response do not seem to align with the content of our paper or our rebuttal. We wonder if there might have been an accidental mix-up with another submission. Could you kindly double-check whether your response was intended for our paper?

Thank you again for your time and consideration.

Reviewer_4yPqOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose VideoResearchAgent, a framework for training agents to answer questions using videos from the web. They utilize the Qwen 3.5-4B model as the foundation for their agent. The agent first finds relevant videos, then locates the necessary visual evidence, and finally answers the question based on that evidence. To train this agent, the authors generated multi-hop research questions from videos and collected teacher trajectories from the Seed-2.0-Pro teacher for the supervised fine-tuning stage. They filtered out questions that could be answered without inspecting the video frames. They then used a local video environment for the GRPO training stage to avoid slow live video interactions while keeping general web search live.To make the agent less dependent on their local retrieval setup, they introduced Retrieval-Domain Randomization, which modifies search results during training to help the agent handle live searches. These modifications include reordering results, adding distracting videos, modifying video titles, and sometimes removing relevant results. They evaluated their agent on the Video-BrowseComp benchmark and reported on the performance and transfer of results from the local environment to live web search. Their agent achieved comparable performance to Gemini-3-Flash-Preview while using fewer tool calls and tokens than the base Qwen model.

Critical strengths

-The overall training framework is a useful contribution. The authors combine synthetic video-grounded tasks and a local simulator to train agents for open-web video research more efficiently. In addition, they maintain the same search and watch interfaces used during inference.

-The synthetic task-generation pipeline is also a well motivated component of the framework. The questions are grounded in timestamped visual evidence, and the authors filter out those that can be answered without inspecting the visual content. This encourages the training process to rely more on visual evidence.

-The proposed RDR is a good adaptation of domain randomization to the retrieval during GRPO. It directly addresses a difficulty in transferring training from local simulators to live searches. The same idea can be useful for other agents that are trained with simulated search, like text-based research or shopping agents.

Critical weaknesses

-While Figure 4 compares the full setup with fixed retrieval, it doesn’t disentangle what drives RDR's gains. The authors should ablate candidate-pool size and gold-source placement separately. Reporting the recall and rank distribution of gold sources would also clarify whether RDR simplifies the retrieval task or not.

-Figure 4 compares RDR-GRPO with fixed-retrieval GRPO on just 63 tasks and reports results only up to step 70. However, the final RDR model is trained for 85 steps. A matched-budget comparison of the two final models on all 210 tasks is missing

-The margin over SFT is thin and lacks error bars across training seeds. Figure 4’s confidence intervals show checkpoint progression rather than the reliability of the final delta over SFT. Reporting multi seed variance and a paired task-level comparison would better establish that these gains aren't just noise.

-The efficiency comparison would be stronger if it also included a base model with simple guardrails against repetitive calls. You could evaluate this baseline with loop detection, retry limits, or query-reformulation prompts to clarify how much of the accuracy–efficiency gain can be achieved without training.

-Appendix G tracks access to relevant videos, but opening a video does not mean the sampled frames contain the answer. This would require checking a sample of training examples and rewarded answers against the actual frames to assess how reliable the supervision is and how often unsupported answers are actually receiving credit.

Questions for authors

1.The filtering judge only sees text, not the actual frames. How do you know the retained SFT trajectories are actually grounded? Was there any manual verification on a random sample?

2.Are full-benchmark results available for fixed-retrieval GRPO and RDR-GRPO at the same final training budget?

3.For the same training queries, how did RDR change how often gold-labelled videos appeared in the displayed results and where they ranked, compared with retrieval without RDR?

4.What data and criteria were used to choose the RDR probabilities and the training stopping point?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 4yPq (1/2)

We thank Reviewer 4yPq for the constructive and detailed review. We are glad you recognized the value of our training framework, the grounded task synthesis pipeline, and the motivation behind RDR. Below, we address each weakness and question with comprehensive experimental analyses.


W1 & Q3: Deconstructing RDR Gains — Disentangling Candidate Pool Size, Gold Placement, and Ranking Difficulty

To determine whether RDR simplifies or complicates retrieval, we replayed retrieval steps across 500 training tasks under identical corpus settings, analyzing candidate recall, ranking distribution, and perturbation mechanisms.

  • Recall vs. Ranking Difficulty:
    • Without RDR (Fixed Retrieval): R@1 is 0.531, R@5 is 0.779, R@10 is 0.857, R@20 is 0.905, and R@50 is 0.936.
    • With RDR: Target video visibility in the top-5 display window rises from 0.779 to 0.891 (and reaches ~0.95 across iterative queries because the keep action pulls relevant candidates into view).
    • Ranking and candidate discrimination become significantly harder: The rank-1 rate drops from 0.531 to 0.486 overall, and plummets to 0.173 in the hard perturbation tier where 67% of displayed results are distractors. The average rank slips from 1.57 to 1.86.

Conclusion: RDR slightly relaxes recall to prevent the agent from getting trapped in empty search dead-ends. However, it substantially increases the difficulty of ranking and candidate discrimination. The policy cannot simply click the top candidate and is forced to carefully discriminate between the target video and deceptively similar look-alikes.

  • Disentangling Candidate Pool Size from Placement:
    • Candidate pool size: Testing candidate pool sizes k in {5, 10, 20, 30, 50} yields target visibility of 0.754 -> 0.815 -> 0.861 -> 0.877 -> 0.891, and MRR of 0.546 -> 0.590 -> 0.624 -> 0.636 -> 0.648. This confirms that candidate pool size primarily governs the recall upper bound.
    • Isolating placement: To isolate position displacement from recall expansion, we tested a condition where placement is only allowed to push target videos down (never pull them up). Under this setting, visibility drops to 0.741 (below the 0.779 of fixed retrieval), confirming that placement intervention strictly increases task difficulty.
    • Role of distractors and corrupted metadata: Only rank intervention directly shifts where the target video lands (MRR 0.658). Adding hard distractors (24% of display slots) and noisy titles (23%) does not alter rank metrics because it modifies the surrounding context. Their role is semantic: they surround the target with confusing look-alikes, preventing the agent from relying on superficial title keyword matching.

Conclusion: Candidate pool size sets the recall ceiling, while placement strictly increases difficulty. Semantic distractors and title noise prevent superficial keyword matching and compel the agent to verify actual video frames.


W2 & Q2: Matched-Budget Comparison on Full Benchmark (210 Tasks) at Step 85

We evaluated both Fixed-Retrieval GRPO and RDR-GRPO at the exact same final budget of 85 training steps across all 210 tasks of Video-BrowseComp:

Model / ConfigurationTraining BudgetVideo-BrowseComp Accuracy (Full 210 tasks, 3 runs)
Untrained Qwen3.5-4B Base—30.00%
SFT Warmup Baseline—36.67%
Fixed-Retrieval GRPO85 steps35.60 ± 0.99%
RDR-GRPO (Ours)85 steps40.63 ± 0.99%

Conclusion: At 85 steps, Fixed-Retrieval GRPO drops below SFT. In contrast, RDR-GRPO achieves 40.63%, proving that retrieval randomization is what drives robust sim-to-real transfer.


W3: Multi-Seed Variance and Task-Level Paired Analysis

We launched two additional training runs with different random seeds and evaluated them on the full 210 benchmark. Due to the time constraints of the rebuttal phase, we only trained for 50 steps. We also conducted some paired analysis experiments. Here are the results:

  • Multi-Seed Training Consistency (Full 210 Tasks):
    • Seed 1 (85 steps, paper run): 40.48% (+3.81 pp over SFT)
    • Seed 2 (50 steps): 37.14% (+0.47 pp over SFT)
    • Seed 3 (50 steps): 38.10% (+1.43 pp over SFT)
    • Cross-seed mean: 38.57 ± 1.72% (+1.87 pp over SFT)
  • Paired Task-Level McNemar Test (210 Tasks):
    • SFT -> RDR-GRPO: 22 tasks flip from incorrect to correct vs. 14 from correct to incorrect (net gain: +8 tasks, +3.81 pp, p=0.24).
    • Fixed GRPO -> RDR-GRPO: 23 tasks flip from incorrect to correct vs. 13 from correct to incorrect (net gain: +10 tasks, +4.76 pp, p=0.13).

All three seeds consistently beat the SFT baseline even at an intermediate checkpoint, confirms that the gain is a reproducible training effect rather than an artifact of a lucky seed. Besides, The transitions are consistently net positive, demonstrating that policy learning genuinely acquires new problem-solving capabilities rather than oscillating at random.

AuthorsOpenReview ↗

Response to Reviewer 4yPq (2/2)

W4: Heuristic Efficiency Guardrails Baseline Without Training

To explore the benefits of optimizing only the prompt and framework for solving this type of problem, we evaluated rule-based guardrails on the untrained base model across all 210 tasks under the same 50-turn cap and environment. To optimize the prompt, we have incorporated additional guidance—building upon the existing prompt—designed to prevent the agent from making errors when performing tasks of this nature. The results are as follows:

  1. Base + Query Reformulation Prompt: Accuracy is 28.6%, average tool calls is 47.03, and 94.3% of tasks hit the 50-turn cap (722K tokens/task).
  2. Base + Anti-Repetition Guardrail (blocking identical tool+arguments executed > 2 times): Intercepts 39.6% of attempted calls (affecting 169/210 tasks), but 81.0% still hit the cap, and accuracy is only 30.4% (flat with raw base 30.0%, far below our agent's 40.48%).
  3. Control on Qwen3.8-Max: Accuracy remains unchanged (52.86% vs. 52.38%), while tool calls drop by 23% and tokens drop by 32% (cap rate 4.3% -> 1.0%).

These experiments show that rule-based guardrails can reduce token overhead on strong models, but cannot teach the base model how to reformulate queries or inspect video evidence. When blocked, the untrained base simply switches to other unhelpful queries until hitting the turn limit, confirming that policy training is essential.


W5 & Q1: Verification of Visual Grounding in SFT Trajectories and Reliability of Supervision

During the data synthesis phase, we conducted manual spot checks on the synthesized data, confirming that it indeed relies on information from video frames. Here we conducted additional multiple complementary analyses to verify visual grounding and rule out text shortcuts:

  1. Live web text-agent filtering audit: An external search agent (GPT-6.1-Terra + Serper, up to 10 turns) solves 0 / 35 (0.0%) of final retained tasks (down from 3 / 50 on unfiltered candidates), and closed-book memory solvability drops from 11.1% to 0.0%.
  2. Context-withholding test on 100 solved SFT trajectories (GPT-6.1-Terra probe):
Input Condition to GPT-6.1-TerraAccuracy on 100 Verified Tasks
Question only (Q)0.0%
Question + Tool text / transcripts (frames omitted) (T)2.0%
Question + Video frames only (text omitted) (V)52.0%
Question + Tool text + Video frames (T + V)46.0%
Original Agent Trajectory Accuracy100.0%
  1. 200-trajectory frame audit: Providing visual frames yields a +15.2% accuracy boost (p = 0.0002); 70.5% of answers directly require visual details, while only 1.5% appear in transcripts. Unsupported answers receiving accidental credit through text shortcuts is < 2%.

These results confirm that final tasks cannot be bypassed by text shortcuts or memory leakage, and that the supervisory signal reliably rewards genuine visual grounding.


Q4: Hyperparameter Selection for RDR and Training Stopping Point

  1. RDR probability calibration: In early runs, we found that purely hard perturbations dropped rank 1 to 17.3% with 67% distractors, causing exploration collapse. Our calibrated mixture retains sufficient basic signal while introducing realistic search noise (rank shuffling, 24% hard negatives, 23% title noise), maintaining a stable learning gradient.
  2. Training stopping point: By steps 76–85, both the mean training Critic score and policy KL divergence entered an extended flat plateau:
Step WindowMean Training ScoreMean KL Loss
1–100.6400.016
11–200.6900.045
21–300.7100.064
31–400.7600.079
41–500.7550.092
51–600.7470.102
61–700.7720.106
71–800.7600.102
76–850.7560.101

Both task score and KL drift reached stability by steps 76–85, so we stopped training.

Thanks for the additional experiments and clarifications. These largely address my concerns, and I’m raising my score to 3. Please also double check the reported McNemar p value, as it seems inconsistent with the task level counts.

Dear Reviewer 4yPq,

Thank you for pointing this out. The previous p-value was reported incorrectly, and we have now corrected it and characterized the gains as being consistent in direction.

We would like to thank you once again for taking the time from your busy schedule to review our rebuttal.

Reviewer_YRrKOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents VideoSearchAgent, a scalable training framework for finding relevant videos on the open web. To address the high cost and instability associated with reinforcement learning through direct interaction with web sources, the framework employs a local simulator. Its task-generation pipeline begins by collecting a diverse set of videos and using an LLM to generate seed queries that identify anchor videos. It then expands an evidence graph to collect visual evidence, which an LLM uses to formulate grounded questions. Finally, a rigorous filtering pipeline ensures that the resulting tasks require examining the visual content of video frames. Experimental results demonstrate the effectiveness and efficiency of the proposed method.

Critical strengths
  • To the best of my knowledge, this is the first work to combine video research, open-web video search, and policy learning, making a valuable contribution to the research community.

  • The paper is well written and easy to follow.

  • The experimental results demonstrate substantial performance improvements over the base model.

Critical weaknesses
  • The ablation studies are insufficient to establish the effectiveness of the four modules: diverse video discovery, recursive evidence graph construction, grounded task generation, and task filtering and refinement. More comprehensive ablations are needed to isolate the contribution of each module.

  • To what extent does the retrieved visual evidence improve reasoning? More detailed experiments are needed to clarify its contribution and identify what additional information it provides to the base model without further training.

Questions for authors
  • Please provide additional ablation studies to demonstrate the effectiveness of each module.

  • Please conduct further experiments to clarify which types of retrieved visual evidence contribute to improved reasoning.

3 comments
AuthorsOpenReview ↗

Response to Reviewer YRrK (1/2)

We thank Reviewer YRrK for the constructive review and for recognizing our work as the first to combine video research, web search, and policy learning. Below, we provide clear module ablations and detailed experiments showing how visual evidence helps reasoning.


W1 & Q1: Effectiveness and Module Ablation of the Task Synthesis Pipeline

To test the four modules in our task synthesis pipeline (diverse video discovery, recursive evidence graph, grounded task generation, and task filtering and refinement), we ran controlled replacement tests and checked the generated dataset distribution:

1. Controlled Module Replacement Ablation (30 tasks per variant under identical seeds and generator):

Pipeline Variant (30 tasks each)Answerable via Text Alone (%)Cross-Video Tasks (%)Filtered as "Answerable by Model Knowledge Alone" (%)
Full Pipeline0.0%66.7%9.1%
w/o Grounded Generation (text-only prompt from titles/attributes, no frame evidence required)24.1%73.3%19.4%
w/o Recursive Graph (restricted to single-hop video evidence)0.0%0.0%21.2%
  • Grounded generation forces the model to watch the video: In our full pipeline, 0% of tasks can be answered from text metadata (titles or tags) alone. Without frame grounding, 24.1% of tasks can be solved by text search shortcuts without opening any video.
  • Recursive evidence graph creates multi-hop questions: Restricting the graph to a single hop drops cross-video tasks from 66.7% to 0%. Also, single-hop tasks are much easier to guess from memory (questions rejected due to prior knowledge jump from 9.1% to 21.2%).

2. Diverse Video Discovery Ablation & Corpus Audit:

To isolate the effect of our taxonomy-guided discovery from LLM pre-training priors, we tested three seed-generation variants under identical generator settings (Qwen3.8-Max, temp 0.7, same deduplication) and retrieved videos on YouTube:

Seed Generation VariantRegions CoveredRegional EntropyEuro-American % (EN)Median ViewsLong-Tail % (< 100k views)
Taxonomy-guided (Ours)92.84 bits22.5%32k67.9%
Naive Prompting (w/o Taxonomy)51.78 bits90.0%97k50.0%
Popular Search Terms41.36 bits—634k30.0%

Experiments show that removing the system causes the seeds to collapse toward the model’s own internal priors: in the English-language seed set, 90.0% of items are concentrated on entities from Europe and North America (compared to only 22.5% in our study), and the number of covered regions drops sharply from 9 to 5 (with entropy decreasing from 2.84 to 1.78 bits). Furthermore, retrieved videos become heavily skewed toward high-frequency, mainstream content, with the share of long-tail videos (those with fewer than 100,000 views) falling from 67.9% to 50.0%. The taxonomy systematically forces discovery toward long-tail, globally diverse content that has minimal textual web presence, ensuring tasks depend on watching video frames.

3. Task Filtering and Refinement Ablation (Auditing Text Shortcuts & Parametric Memory Leakage):

We tested tasks before and after filtering using an external search agent (GPT-6.1-Terra with live Serper web search, up to 10 turns, allowed to say "I don't know"), a closed-book memory probe, and DeepSeek-V4.1-Flash as judge:

Pipeline StageSolved by Live Web Text Agent (Serper)Solved by Closed-Book Probe (Parametric Memory)Mean Text-Agent Turns
Unfiltered Candidates (50 tasks)3 / 50 (6.0%)6 / 50 (11.1%)9.00 turns
Final Retained Tasks (35 tasks)0 / 35 (0.0%)0 / 35 (0.0%)9.15 turns

The conclusion is that filtering and refinement block both web-text shortcuts and memory leaks—web-search-solvable candidates drop from 6.0% (3/50) to 0.0% (0/35) even with >9 search turns on average, and memory-answerable questions drop from 11.1% (6/50) to 0.0% (0/35)—so all final tasks strictly require looking at video frames.

AuthorsOpenReview ↗

Response to Reviewer YRrK (2/2)

W2 & Q2: Contribution and Types of Retrieved Visual Evidence in Improving Reasoning

1. Visual Evidence Contribution on the Base Model (Without Training):

On all 210 Video-BrowseComp tasks, we tested the untrained base model (Qwen3.5-4B Base) with and without video frames (withholding frames leaves video titles and transcripts intact):

Evaluation Configuration (Full 210 tasks)AccuracyAccuracy Drop from Withholding Frames
Qwen3.5-4B Base (with visual frames)30.00%—
Qwen3.5-4B Base (frames withheld; text/transcripts kept)18.57%-11.43 pp

Experiment shows that even without training, hiding video frames causes accuracy to drop by over 11 points, proving that video frames contain essential facts not found in search text or audio transcripts.

2. To what extent does the retrieved visual evidence improve reasoning; Which Types of Visual Evidence Drive Reasoning:

Next, using the SFT training data we constructed, we will further illustrate why the additional information provided by visual frames is irreplaceable compared to text-only data during model inference. We will also examine which types of visual evidence are useful, thereby providing a direct answer to your concern.

We evaluated 100 correctly answered SFT trajectories on GPT-6.1-Terra under four input conditions (instructed to use only given info and decline if unsure):

Input Condition to GPT-6.1-TerraAccuracy on 100 Verified Tasks
Question only (Q)0.0%
Question + Tool text / transcripts (frames omitted) (T)2.0%
Question + Retrieved visual frames only (text omitted) (V)52.0%
Question + Full tool text + Visual frames (T + V)46.0%
Original Agent Trajectory Accuracy100.0%

Breakdown of visual evidence types where frames made the decisive difference:

Visual Evidence CategoryShare (%)Representative Examples
Action counting and frequency61.8%Physical hits, exercise reps, screen motions unmentioned in audio
On-screen text and OCR22.2%Scoreboards, recipe ingredient amounts, background signage, diagrams
Visual appearance and physical state13.8%Liquid reaction color, vehicle models, appliance plug-in status
Spatial layout and position2.2%Relative positioning and multi-view orientation of subjects

Based on these results, the vast majority of tasks here cannot be "bypassed" via textual shortcuts or information leakage from the model's memory; incorporating visual frame data into the decision-making process significantly improves the model's reasoning performance. When relying solely on tool-related text and audio transcripts, the accuracy is a mere 2.0%; however, once visual frames are introduced, accuracy reaches 52.0%. This confirms that visual frames serve as the most direct source of factual information for addressing questions that hinge on specific visual details.

The authors’ response addressed my concerns, so I’ve raised my rating to Weak Accept.

Comments on the paper

2 comments

Dear reviewers, the authors have posted their responses to your reviews. Please have a look and discuss ASAP as the author-reviewer discussion period ends at November 18 at 11:59pm AoE

Best, AC

Dear Area Chair,

We thank the reviewers for their thoughtful feedback and recognition of the value and practical impact of our work.

Strengths of our paper. Reviewers consistently highlighted several key contributions:

  • First work to combine open-web video research, live web search, and policy learning, tackling a consequential training bottleneck where live web interactions are prohibitively slow, expensive, and fragile (Reviewers YRrK, 4yPq, HTAC).
  • A local simulator modeling deployment-time uncertainty (ranking perturbations, hard distractors, metadata corruption) to enable robust sim-to-real transfer via Retrieval-Domain Randomization (RDR) (Reviewers HTAC, 4yPq).
  • A well-motivated synthetic task pipeline that grounds multi-hop questions in timestamped visual evidence and eliminates text-only shortcuts (Reviewers 4yPq, YRrK).
  • Substantial empirical and behavioral gains on the same backbone, eliminating non-convergent repetitive calls, outperforming the base model, and cutting cumulative API token consumption by 74.9% (Reviewers ZLVT, HTAC, YRrK).

Key concerns and our empirical resolutions:

  • Sim-to-real transfer and RDR gains (Reviewers 4yPq, ZLVT, HTAC): We evaluated fixed-retrieval GRPO versus RDR-GRPO under matched training budgets across all 210 tasks, complemented by multi-seed runs and perturbation replays on 500 tasks. The results confirm that RDR successfully prevents simulator shortcut memorization and is essential for robust sim-to-real transfer to the live web.
  • Visual information necessity to solve the task. (Reviewers YRrK, 4yPq, ZLVT): We conducted an evaluation-time visual frame withholding ablation across the full benchmark and audited 100 solved trajectories under text-only versus visual-input conditions. This demonstrates that our agent strictly depends on inspecting visual video frames rather than exploiting textual shortcuts or audio transcripts.
  • Task synthesis pipeline ablations (Reviewer YRrK): We performed controlled module replacement tests across all four pipeline stages (video discovery, evidence graph, grounded generation, and task filtering). These ablations verify that every component is necessary for generating diverse, multi-hop tasks and eliminating text-search shortcuts or memory leakage.
  • Rule-based guardrails control (Reviewer 4yPq): We evaluated query-reformulation prompts and loop-detection guardrails on the untrained base model. The findings show that heuristic rules cannot teach agents how to formulate helpful queries or inspect visual content, confirming that policy reinforcement learning is indispensable.
  • Framework positioning (Reviewer HTAC): We clarified and repositioned the paper around our core contribution as a practical training framework and empirical sim-to-real recipe for open-web video research.

Following our rebuttal, all four reviewers support acceptance with unanimous positive consensus. Reviewer YRrK officially raised their score from 2 to 3 (Weak Accept), and Reviewer 4yPq confirmed in their latest comment that their concerns were largely resolved (pending numeric update in the form). Reviewers ZLVT (3) and HTAC (3) both maintained their positive recommendations.

We believe our additional results and clarifications substantially strengthen the paper and address the reviewers' central concerns. We are also committed to incorporating the reviewers' constructive suggestions and the promised revisions into the final version. We hope you will consider these developments in your final assessment.

Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.