acceptodds
Under review as a conference paper at ICLR 2027

AdaVDR: Adaptive Tool use and reflection for video deep research

Abstract

Video deep research aims to answer questions by jointly understanding video content and retrieving external knowledge from the open Web. However, these tasks often require diverse and multi-step tool-use trajectories, which may lead to unnecessary video tool calls, incurring additional costs and failure risk. In this paper, we propose AdaVDR, an adaptive video deep research agent with adaptive tool use and reflection, which selects tools based on task and model capabilities while backtracking when needed. To this end, we develop a data construction pipeline that builds adaptive tool-use trajectories according to task types and model capabilities. Specifically, we construct QA pairs that require both video and external information, and organize the information acquisition processes into task-specific tool-use trajectories. We then apply model-conditioned tool necessity filtering to remove tools that can be bypassed using video understanding and internal knowledge of the model. Based on this pipeline, we construct a training dataset with adaptive trajectories and perform SFT on them to initialize these capabilities of AdaVDR. To further improve adaptive capabilities, we propose NASA-GRPO, a necessity-aware segment-guided credit assignment method, which evaluates segment-guided evidence contribution and tool necessity to construct normalized tool-level rewards. We also construct VDR-EE, a benchmark built using the proposed pipeline. Experiments show that AdaVDR outperforms the baselines by up to 15% and 29% on VDR-EE and the existing VideoDR, respectively, demonstrating its superiority.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video Deep Research, Multimodal Agent, Agentic Reasoning, Video Understanding, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_5LRjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces AdaVDR, a video agent that selects tools and reflects on unreliable intermediate results. The approach constructs model-conditioned tool-use trajectories for SFT and proposes NASA-GRPO, which assigns tool-level rewards based on evidence contribution and tool necessity. The authors further present VDR-EE, a new benchmark for video deep research.

Critical strengths

Important problem: The paper tackles unnecessary tool invocation and error propagation in video deep research—practically relevant challenges for multimodal agents.

Comprehensive experiments: The paper includes ample ablations, tool-use analyses, and evaluations on additional image and video benchmarks, all of which support the effectiveness of the overall framework.

Critical weaknesses

Limited novelty and missing related work: The core idea of model-conditioned tool necessity has been explored in prior works—like Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use—yet this work is not discussed.

Questionable tool necessity criterion: A single correct prediction does not guarantee that the tool is unnecessary. This may introduces unreliable labels into training.

Insufficient justification for task-specific training: The paper lacks comparisons with carefully prompt-optimized agents that use the same tools. In addition, stronger general-purpose models already surpass AdaVDR on VDR-EE.

Questions for authors
  • See weakness 1.

  • Can AdaVDR perform on unseen tools?

  • See weakness 3.

6 comments
AuthorsOpenReview ↗

Response(1/4): W1 & Q1 (How AdaVDR’s Tool Necessity Criterion Differs from Prior Work)

W1 & Q1: Limited novelty and missing related work: The core idea of model-conditioned tool necessity has already been explored in prior works—most notably Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use (Cheng et al., 2026)—yet this work is not discussed. The paper should more clearly distinguish its contributions from existing adaptive tool-use and efficiency-aware RL methods. How does the proposed model-conditioned necessity criterion differ from prior model-adaptive tool necessity research?

Thank you for pointing out the missing related work. We compare AdaVDR with Model-Adaptive Tool Necessity [1] and other adaptive tool-use and efficiency-aware RL methods below. Prior methods use prediction probabilities or answer correctness to assess tool utility at the task or call level. As discussed below, task-level feedback is sparse, and tool-level assessments can miss how several calls work together to recover evidence in video deep research. AdaVDR’s novelty lies in using verifiable evidence acquisition to provide segment-guided supervision, capturing the joint contribution of multiple tool calls, and introducing necessity-aware credit assignment to weight individual tool calls according to their necessity for the current model. In this way, AdaVDR provides more accurate and appropriate supervision for learning tool-use necessity.

Existing adaptive tool-use and efficiency-aware RL methods differ in how they select and reward tool calls. Model-Adaptive Tool Necessity [1] uses ten tool-free trials to assess whether the whole question requires tools and uses the resulting labels to train hidden-state probes that examine the gap between perceived necessity and actual tool use, rather than training the model itself to determine when it needs to use tools. Toolformer [2] retains calls that reduce prediction loss, even if the model can already recover the same information. Search-R1 [3] rewards final-answer correctness, EAPO [4] adjusts rewards by question difficulty and tool-call count, and CoBRA [5] compares answer rewards with and without tools. These trajectory-level signals give limited guidance on which calls can be skipped. TRACE [6] measures each call’s immediate contribution to the answer and may undervalue preparatory actions. Spurious Tool Use [7] uses an external judge to identify unnecessary calls and penalizes them in the trajectory reward. Its judgments leave open whether the current model can recover the same intermediate evidence after skipping a call.

However, video deep research often requires multiple pieces of evidence. The model may already know some of them and still need tools to obtain the rest. Task-level necessity judgments cannot distinguish these cases, final-answer rewards provide sparse feedback, and immediate call-level rewards can undervalue preparatory actions whose contribution becomes clear only after a later retrieval succeeds. AdaVDR therefore uses each piece of evidence as a verifiable intermediate target. We test whether the model can recover the same evidence after bypassing a call and use this judgment to filter redundant calls during SFT. During RL, segment-guided credit assignment groups calls that pursue the same evidence and gives them base credit when the segment recovers it, including preparatory calls. Necessity-aware credit assignment then adjusts that credit for each call. Necessary calls retain full credit, while bypassable calls retain half, reducing their contribution to the process advantage.

We have added these comparisons to the Related Work in Section 2.3 of the revised manuscript.

[1] Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use — NeurIPS 2026

[2] Toolformer: Language Models Can Teach Themselves to Use Tools — NeurIPS 2023

[3] Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning — COLM 2025

[4] Learning When Not to Act: Mitigating Tool Abuse in Agentic Reinforcement Learning — arXiv 2026

[5] CoBRA: Learning Tool-Use Boundaries via Counterfactual Margins — EMNLP 2026

[6] TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents — arXiv 2026

[7] Spurious Tool Use: When RL Agents Learn the Wrong Reason to Act — arXiv 2026

AuthorsOpenReview ↗

Response(2/4): W2 & Q1 (Tool Necessity Label Stability and Counterfactual Tool Removal)

W2 & Q1: Questionable tool necessity criterion: The proposed method considers a tool unnecessary if the model correctly predicts its target information without invoking it. However, a single correct prediction does not guarantee that the tool is unnecessary (e.g., a model may answer correctly with 60% probability without the tool but achieve 95% accuracy with it). This potentially introduces unreliable necessity labels into both SFT and RL training. How reliable are its labels under repeated predictions or counterfactual tool removal?

R: Thank you for raising this question. In both SFT and RL, we label a call as bypassable only if all eight sampled responses pass. Our experiments show that the resulting labels are stable and reliable and that removing calls labeled as bypassable does not reduce final-answer accuracy in our evaluation. Together, these results support the rationale for our method and show that necessity-aware training can make tool use more efficient while improving final-answer accuracy.

1. Stability of necessity labels. We compared the 8/8, 16/16, and 32/32 rules on the same 200 bypass checks, requiring all sampled responses to pass in each setting. The 16/16 and 32/32 rules agreed with the 8/8 rule on 98.0% (196/200) and 96.5% (193/200) of labels, respectively. This high agreement indicates that eight samples are sufficient to obtain stable labels in our setting. During RL training, we sample the eight responses in parallel to limit the additional wall-clock latency of each bypass check.

ComparisonLabel agreement
8/8 vs. 16/1698.0% (196/200)
8/8 vs. 32/3296.5% (193/200)

To limit the impact of potential labeling errors during RL, NASA-GRPO assigns bypassable calls a necessity weight of 0.5. A bypassable call that contributes useful evidence therefore retains half of its evidence reward.

2. Effect of necessity filtering and weighting on final-answer accuracy. We evaluate how removing calls classified as unnecessary from SFT trajectories affects final-answer accuracy. We also compare the full training pipeline with a variant that omits both SFT necessity filtering and RL necessity weighting.

SettingVideoDRVDR-EE
SFT without necessity filtering54.046.4
SFT with necessity filtering52.048.0
SFT without necessity filtering + RL without necessity weighting62.053.2
SFT with necessity filtering + RL with necessity weighting66.055.2

We evaluate necessity filtering at the SFT stage and jointly remove necessity filtering and necessity weighting in the end-to-end training ablation. Filtering alone has mixed effects across the two benchmarks. After RL, the full method improves accuracy by 4.0 percentage points on VideoDR and 2.0 percentage points on VDR-EE over the variant with both components removed. This complements the ablation analysis in Section 6.4 (Table 5), where adding necessity weighting improves accuracy while reducing the average number of interaction turns. These findings are consistent with Theorem 1 in Section 3.2, which shows that misleading tool outputs can increase prediction risk when the model already has sufficient information.

3. Effect of counterfactual tool removal on final-answer accuracy. To test whether calls labeled bypassable can be skipped at inference time, we evaluated Qwen3.5-9B on 85 questions associated with 200 evidence checks. Both conditions used the same checkpoint and pre-call contexts, and all other tools remained available. The first condition retained the original tool results. The second skipped calls that passed the 8/8 bypass check and withheld their results.

Inference settingRun 1Run 2Run 3Mean accuracy
Retain the original tool results30.628.231.830.2
Skip calls labeled bypassable32.930.628.230.6

Mean final-answer accuracy remained similar after skipping bypassable calls (30.6 vs. 30.2). This provides direct evidence that the calls flagged by the 8/8 rule can be skipped with little change in aggregate accuracy in this setting. The base model still made these calls in its original trajectories, which motivates training it to skip them on its own.

We have clarified the 8/8 bypass rule and necessity weighting in Sections 4.2 and 5.2. We added the label agreement and counterfactual tool removal results to Appendix F.2 (Tables 19–20) and the training ablations to Appendix H.1 (Table 26). All changes are marked in blue.

AuthorsOpenReview ↗

Response(3/4): W4 & Q2 (Generalization to Unseen Tools) and W3 (Gains over a Prompt-Optimized Baseline)

W4 & Q2: Limited tool-Set generalization: Both trajectory construction and NASA-GRPO depend on predefined tool types and evidence-acquisition patterns. The paper does not evaluate whether the trained agent can adapt to newly introduced or upgraded tools without additional data construction or training, which limits the demonstrated extensibility. Can AdaVDR generalize to previously unseen tools without additional training? And how does its performance change when the available tool set is modified?

Thank you for raising this question. Without additional training, we replace the original video tools with unseen tools and outperform Qwen3.5-9B Base by 25.0 on VideoDR and 12.8 on VDR-EE. We also transfer AdaVDR-9B to WebShop and SlideVQA with their task-specific tools, outperforming the base model by 7.0 pp and 7.9 pp, respectively. These results show that AdaVDR can adapt to new tools on video tasks and transfer to new tasks with different tools, outperforming the base model in both settings.

1. Tool replacement on video tasks. On VideoDR and VDR-EE, we combine existing operations into three tools that were not used to train AdaVDR and use them to replace the corresponding original tools at inference time. AdaVDR-9B still achieves strong results with the replacement tools.

  • search_and_visit searches the web and opens result pages in a single call.
  • crop_search_and_visit crops a region of a video frame, uses it for image search, and opens result pages in a single call.
  • inspect_video provides a single tool for selecting frames, locating time segments, and cropping regions within frames.
BenchmarkQwen3.5-9B BaseAdaVDR-9B (ours)
VideoDR39.064.0 (↑ +25.0)
VDR-EE41.254.0 (↑ +12.8)

2. Transfer to new tasks with task-specific tools. We further evaluate AdaVDR on WebShop and SlideVQA. Both models receive the same tools on each task. AdaVDR outperforms the base model on both benchmarks, demonstrating that it can transfer to new tasks with different tools without additional training.

  • WebShop provides search_products to find products by keyword and click_element to click a product, option, or button on the current page.
  • SlideVQA provides search to retrieve page images from a text query and bbox to crop a region of a retrieved page using specified coordinates.
BenchmarkQwen3.5-9B BaseAdaVDR-9B (ours)
WebShop31.238.2 (↑ +7.0)
SlideVQA31.239.1 (↑ +7.9)

We added the tool replacement and cross-task transfer results to Appendix E.7 (Tables 15 and 16) of the revised manuscript.

W3: Insufficient justification for task-specific training: Although AdaVDR outperforms the untrained Qwen3.5-9B agentic baseline, the paper lacks comparisons with carefully prompt-optimized agents that use the same tools.

All models evaluated in the agentic setting in Table 1 use the same prompts tuned for video deep research and have access to the same tools. We selected the shared prompt by comparing candidate prompts on a held-out development set with Qwen3.5-9B before evaluating on VideoDR and VDR-EE. Qwen3.5-9B and AdaVDR-9B also share the same backbone. Under this matched setup, task-specific training improves accuracy from 40.0% to 55.2% on VDR-EE and from 37.0% to 66.0% on VideoDR. We have clarifyed this evaluation setup in the revised manuscript.

Reviewer_GPk6OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AdaVDR, an adaptive agent for video deep research that integrates video understanding and external knowledge retrieval. The main motivation is that unnecessary or inappropriate tool calls may increase computational cost and introduce unreliable evidence into multi-step reasoning. To address this, the authors introduce a data construction pipeline that generates task-specific tool-use trajectories and filters unnecessary operations according to the target model's capabilities. The resulting trajectories are used for supervised fine-tuning (SFT), followed by a proposed reinforcement learning method, NASA-GRPO, which combines segment-guided evidence rewards, model-conditioned tool necessity weights, and tool-type reward normalization. The paper also introduces VDR-EE, a manually verified benchmark containing 250 questions that require both video understanding and external information. Experiments on VDR-EE and VideoDR show that AdaVDR-9B improves accuracy over its Qwen3.5-9B baseline by 15.2 and 29.0 percentage points, respectively.

Critical strengths

The paper addresses an important challenge in multimodal agents: deciding not only which tools to invoke, but whether individual tool calls are necessary given the task and the model's existing capabilities. The model-conditioned tool necessity filtering is an interesting practical mechanism for constructing adaptive training trajectories, while the incorporation of failed attempts and corrective actions provides useful supervision for evidence recovery. This moves beyond fixed tool-use workflows and is relevant to developing more reliable multimodal research agents.

AdaVDR demonstrates substantial improvements over its base model on two video deep-research benchmarks. The experiments include SFT data-composition studies, RL reward ablations, tool-use analysis, and additional evaluations beyond the main VDR tasks. In particular, the ablations in Table 3 suggest that the model-conditioned necessity component provides an important contribution to the final performance. The introduction of VDR-EE, with annotations covering different question types and video durations, is also a useful contribution to the evaluation of video-grounded external research.

Critical weaknesses
  1. The reliability of model-conditioned tool necessity and its relationship to existing approaches need stronger justification.

A central assumption of the proposed method is that the necessity of a tool can be estimated by bypassing it and checking whether the target model can recover the expected information (Sections 4.2 and 5.2). However, successfully recovering a target answer in an individual bypass evaluation does not necessarily imply that the tool is unnecessary for reliable prediction. The model may occasionally recover the correct information through guessing, memorized knowledge, or incomplete reasoning, while external verification could still substantially improve reliability. Conversely, an unsuccessful bypass attempt does not establish that the operation is consistently necessary.

This distinction is particularly important because the resulting necessity judgments are used both to construct adaptive SFT trajectories and to determine tool-level RL rewards. Although the paper reports human agreement on assigned credits, this does not fully establish the stability of the underlying counterfactual necessity estimates. Further evaluation of bypass reliability across repeated predictions or its relationship to downstream answer correctness would strengthen the central methodological claim.

Related work also deserves clearer differentiation. Adaptive video exploration and backtracking have been considered in Deep Video Discovery and VideoLucy, while Toolformer explores learning when external tools provide useful information. More recent work on epistemic tool necessity and efficiency-aware tool-use optimization also provides relevant context. The paper would benefit from explaining more explicitly how its operational definition of tool necessity differs from prior notions of tool utility, and what additional capability is enabled by the proposed credit-assignment formulation.

  1. The experimental evaluation does not fully substantiate the claimed efficiency advantages or isolate the benefit of the proposed RL formulation relative to alternative approaches.

The paper emphasizes reducing unnecessary tool calls and improving efficiency. However, Table 2 shows that the final AdaVDR model requires more average interaction turns than both the base model and its SFT version: 11.67 versus 5.12 and 10.18 on VideoDR, and 13.26 versus 5.70 and 10.81 on VDR-EE. The final model also makes more external retrieval calls than the SFT model. These observations are not necessarily inconsistent with more effective tool selection, particularly given the improvements in accuracy and the reduction in redundant calls relative to specific RL ablations. Nevertheless, they make the overall efficiency claim difficult to interpret without an explicit accuracy-cost comparison.

Furthermore, while the ablation studies demonstrate the contributions of individual NASA-GRPO components, comparisons with alternative efficiency-aware RL objectives or tool-level credit-assignment strategies remain limited. For instance, cost-regularized GRPO or other process-reward formulations could help determine whether the gains arise specifically from model-conditioned necessity weighting or from additional process supervision. This is particularly relevant because the necessity-aware mechanism constitutes one of the paper's main methodological contributions.

Questions for authors

Q1. How robust is the model-conditioned tool necessity criterion?

In Sections 4.2 and 5.2, tool necessity is determined by whether the target model can recover the expected information when a tool is bypassed. Is this judgment based on a single inference or multiple sampled predictions? How stable are these necessity labels under repeated trials? More importantly, have the authors examined whether tools classified as unnecessary can be removed without degrading final-answer accuracy or reliability? Such evidence would help validate the underlying assumption of the proposed training method.

Q2. How should the efficiency improvements be interpreted given the increased number of tool interactions?

Table 2 indicates that AdaVDR uses more overall interaction turns and external retrieval calls than the base model and SFT checkpoint. Could the authors provide a more systematic accuracy-efficiency comparison, including total tool calls, latency, and, if available, inference cost? It would also be helpful to clarify whether the efficiency claim is intended primarily as a reduction in redundant operations relative to another RL formulation rather than a reduction in absolute computational cost.

Q3. How much of NASA-GRPO's improvement comes from the necessity-specific reward rather than process-level supervision more generally?

Table 3 shows a substantial improvement when the model-conditioned necessity weight is added. Could the authors clarify how the proposed formulation compares with simpler alternatives such as tool-cost penalties, evidence-only process rewards, or other action-level credit-assignment methods under comparable training budgets? This would help establish the methodological contribution beyond the reported component ablations.

6 comments
AuthorsOpenReview ↗

Response(1/4) W1 (How AdaVDR Differs from Prior Work)

W1: Related work also deserves clearer differentiation. Adaptive video exploration and backtracking have been considered in Deep Video Discovery and VideoLucy, while Toolformer explores learning when external tools provide useful information. More recent work on epistemic tool necessity and efficiency-aware tool-use optimization also provides relevant context. The paper would benefit from explaining more explicitly how its operational definition of tool necessity differs from prior notions of tool utility, and what additional capability is enabled by the proposed credit-assignment formulation.

Thank you for pointing out the missing related work. We compare AdaVDR with Deep Video Discovery [1], VideoLucy [2], Toolformer [3], and other adaptive tool-use and efficiency-aware RL methods below. Prior methods use prediction probabilities or answer correctness to assess tool utility at the task or call level. As discussed below, task-level feedback is sparse, and tool-level assessments can miss how several calls work together to recover evidence in video deep research. AdaVDR’s novelty lies in using verifiable evidence acquisition to provide segment-guided supervision, capturing the joint contribution of multiple tool calls, and introducing necessity-aware credit assignment to weight individual tool calls according to their necessity for the current model. In this way, AdaVDR provides more accurate and appropriate supervision for learning tool-use necessity.

Deep Video Discovery [1] and VideoLucy [2] use training-free, prompt-based strategies to guide video exploration and memory backtracking at inference time. Toolformer [3] retains calls that reduce prediction loss, even if the model can already recover the same information. Model-Adaptive Tool Necessity [4] uses ten tool-free trials to assess whether the whole question requires tools and uses the resulting labels to train hidden-state probes that examine the gap between perceived necessity and actual tool use, rather than training the model itself to determine when it needs to use tools. Search-R1 [5] rewards final-answer correctness, EAPO [6] adjusts rewards by question difficulty and tool-call count, and CoBRA [7] compares answer rewards with and without tools. These trajectory-level signals give limited guidance on which calls can be skipped. TRACE [8] measures each call’s immediate contribution to the answer and may undervalue preparatory actions. Spurious Tool Use [9] uses an external judge to identify unnecessary calls and penalizes them in the trajectory reward. Its judgments leave open whether the current model can recover the same intermediate evidence after skipping a call.

However, video deep research often requires multiple pieces of evidence. The model may already know some of them and still need tools to obtain the rest. Task-level necessity judgments cannot distinguish these cases, final-answer rewards provide sparse feedback, and immediate call-level rewards can undervalue preparatory actions whose contribution becomes clear only after a later retrieval succeeds. AdaVDR therefore uses each piece of evidence as a verifiable intermediate target. We test whether the model can recover the same evidence after bypassing a call and use this judgment to filter redundant calls during SFT. During RL, segment-guided credit assignment groups calls that pursue the same evidence and gives them base credit when the segment recovers it, including preparatory calls. Necessity-aware credit assignment then adjusts that credit for each call. Necessary calls retain full credit, while bypassable calls retain half, reducing their contribution to the process advantage.

We have added these comparisons to the Related Work in Section 2.3 of the revised manuscript.

[1] Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understanding. NeurIPS 2025.

[2] VideoLucy: Deep Memory Backtracking for Long Video Understanding. NeurIPS 2025.

[3] Toolformer: Language Models Can Teach Themselves to Use Tools. NeurIPS 2023

[4] Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use. NeurIPS 2026

[5] Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning. COLM 2025.

[6] Learning When Not to Act: Mitigating Tool Abuse in Agentic Reinforcement Learning. ArXiv 2026.

[7] CoBRA: Learning Tool-Use Boundaries via Counterfactual Margins. EMNLP 2026.

[8] TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents. ArXiv 2026.

[9] Spurious Tool Use: When RL Agents Learn the Wrong Reason to Act. arXiv 2026.

AuthorsOpenReview ↗

Response (2/4): W1 & Q1 (Reliability of Tool Necessity Judgments)

W1 & Q1: The reliability of model-conditioned tool necessity and its relationship to existing approaches need stronger justification. In Sections 4.2 and 5.2, tool necessity is determined by whether the target model can recover the expected information when a tool is bypassed. Is this judgment based on a single inference or multiple sampled predictions? How stable are these necessity labels under repeated trials? More importantly, have the authors examined whether tools classified as unnecessary can be removed without degrading final-answer accuracy or reliability? Such evidence would help validate the underlying assumption of the proposed training method.

R: Thank you for raising this question. In both SFT and RL, we label a call as bypassable only if all eight sampled responses pass. Our experiments show that the resulting labels are stable and reliable and that removing calls labeled as bypassable does not reduce final-answer accuracy in our evaluation. Together, these results support the rationale for our method and show that necessity-aware training can make tool use more efficient while improving final-answer accuracy.

1. Stability of necessity labels. We compared the 8/8, 16/16, and 32/32 rules on the same 200 bypass checks, requiring all sampled responses to pass in each setting. The 16/16 and 32/32 rules agreed with the 8/8 rule on 98.0% (196/200) and 96.5% (193/200) of labels, respectively. This high agreement indicates that eight samples are sufficient to obtain stable labels in our setting. During RL training, we sample the eight responses in parallel to limit the additional wall-clock latency of each bypass check.

ComparisonLabel agreement
8/8 vs. 16/1698.0% (196/200)
8/8 vs. 32/3296.5% (193/200)

To limit the impact of potential labeling errors during RL, NASA-GRPO assigns bypassable calls a necessity weight of 0.5. A bypassable call that contributes useful evidence therefore retains half of its evidence reward.

2. Effect of necessity filtering and weighting on final-answer accuracy. We evaluate how removing calls classified as unnecessary from SFT trajectories affects final-answer accuracy. We also compare the full training pipeline with a variant that omits both SFT necessity filtering and RL necessity weighting.

SettingVideoDRVDR-EE
SFT without necessity filtering54.046.4
SFT with necessity filtering52.048.0
SFT without necessity filtering + RL without necessity weighting62.053.2
SFT with necessity filtering + RL with necessity weighting66.055.2

We evaluate necessity filtering at the SFT stage and jointly remove necessity filtering and necessity weighting in the end-to-end training ablation. Filtering alone has mixed effects across the two benchmarks. After RL, the full method improves accuracy by 4.0 percentage points on VideoDR and 2.0 percentage points on VDR-EE over the variant with both components removed. This complements the ablation analysis in Section 6.4 (Table 5), where adding necessity weighting improves accuracy while reducing the average number of interaction turns. These findings are consistent with Theorem 1 in Section 3.2, which shows that misleading tool outputs can increase prediction risk when the model already has sufficient information.

3. Effect of counterfactual tool removal on final-answer accuracy. To test whether calls labeled bypassable can be skipped at inference time, we evaluated Qwen3.5-9B on 85 questions associated with 200 evidence checks. Both conditions used the same checkpoint and pre-call contexts, and all other tools remained available. The first condition retained the original tool results. The second skipped calls that passed the 8/8 bypass check and withheld their results.

Inference settingRun 1Run 2Run 3Mean accuracy
Retain the original tool results30.628.231.830.2
Skip calls labeled bypassable32.930.628.230.6

Mean final-answer accuracy remained similar after skipping bypassable calls (30.6 vs. 30.2). This provides direct evidence that the calls flagged by the 8/8 rule can be skipped with little change in aggregate accuracy in this setting. The base model still made these calls in its original trajectories, which motivates training it to skip them on its own.

We have clarified the 8/8 bypass rule and necessity weighting in Sections 4.2 and 5.2. We have added the label agreement and counterfactual tool removal results to Appendix F.2 (Tables 19–20) and the training ablations to Appendix H.1 (Table 26). All changes are marked in blue.

AuthorsOpenReview ↗

Response(3/4): W2 & Q2 (AdaVDR’s Tool-Use Efficiency and Inference Latency)

W2 & Q2: How should the efficiency improvements be interpreted given the increased number of tool interactions?

The increased tool calls in AdaVDR mainly arise from reflection, which introduces additional calls to correct unreliable evidence. We conduct additional experiments by removing reflection data while retaining the complete adaptive tool-use method. The results show that AdaVDR reduces tool calls while maintaining strong accuracy, demonstrating its efficiency. Furthermore, even with reflection, AdaVDR achieves lower inference latency than existing specialized video deep research agents, further supporting its practical efficiency.

Tool-use efficiency without reflection. We conduct an additional experiment by removing only the reflection data from training while retaining the full adaptive tool-use method. We further compare the number of tool calls and accuracy on VideoDR, as shown in Table. The average number of calls per question drops from 10.67 to 3.69, below Qwen3.5-9B Base (4.12), VideoSearcher-8B (3.99), and Video-DeepResearch-30B-A3B (6.57). Meanwhile, AdaVDR without reflection achieves 60.0% accuracy, outperforming Qwen3.5-9B Base (37.0%) and VideoSearcher-8B (53.0%). These results demonstrate that our method can reduce tool calls while maintaining high accuracy.

ModelAccuracyVisual callsText callsTotal calls
Qwen3.5-9B Base37.0%0.893.234.12
AdaVDR-9B SFT + RL (without reflection data)60.0%2.311.383.69
AdaVDR-9B SFT + RL (with reflection data)66.0%3.347.3310.67
VideoSearcher-8B53.0%2.531.463.99
Video-DeepResearch-30B-A3B62.0%2.334.246.57

Inference latency analysis. We further measure the actual wall-clock inference latency and output token counts on both VideoDR and VDR-EE, as shown below. Despite the additional tool calls introduced by reflection, AdaVDR achieves lower inference latency than both specialized baselines on VideoDR (98.3s vs. 153.8s/174.9s) and VDR-EE (101.2s vs. 162.7s/236.7s). This efficiency stems from our SFT and RL training, which reduces reasoning output length and thereby helps offset the latency overhead of additional tool calls. Results demonstrate that AdaVDR maintains competitive efficiency while benefiting from reflection-enhanced accuracy.

ModelVideoDR output tokensVideoDR latency (s)VDR-EE output tokensVDR-EE latency (s)
Qwen3.5-9B Base1,44971.41,82070.5
AdaVDR-9B1,78698.31,381101.2
VideoSearcher-8B5,403153.81,022162.7
Video-DeepResearch-30B-A3B2,642174.93,480236.7

In the revised manuscript, we clarify that necessity weighting reduces redundant tool calls relative to the RL variant without necessity weighting, while reflection introduces additional calls to correct unreliable evidence. In Section 6.3, we have added a tool-use comparison with and without reflection data (Table 3) and a wall-clock inference latency comparison with existing baselines (Table 4). Both tables and their accompanying discussion are marked in blue.

Reviewer_SkVvOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

Research Motivation: This paper explores the "Video Deep Research" (VDR) task, which requires an agent to both comprehend video content and retrieve external knowledge from the web. Existing approaches often rely on lengthy or rigid sequences of tool calls; when the underlying support or retrieval tools are unreliable, this can lead to redundant tool invocations, increased latency, and the propagation of errors. The paper propose AdaVDR, an adaptive agent for Video Deep Research. It dynamically selects tools based on task requirements, the model’s own capabilities, and its internal knowledge base. If the necessary information is already accessible, the agent skips superfluous support or retrieval steps; it engages in selective reflection only when intermediate evidence appears unreliable. Training Methodology: We construct task-specific, adaptive tool-use trajectories and employ a "model-conditioned tool necessity filtering" mechanism to prune tools that are unnecessary for the target model to complete a given task. We first perform supervised fine-tuning (SFT) on verified trajectories, followed by the application of NASA-GRPO—a necessity-aware, segment-guided credit assignment method—which assigns rewards to individual tool calls based on their contribution to the evidence, their necessity, and a normalization factor derived from the tool type.

Key Findings: Experiments on the novel VDR-EE benchmark and the existing VideoDR benchmark demonstrate that AdaVDR significantly outperforms baseline models, achieving gains of up to 15 percentage points on VDR-EE and up to 29 percentage points on VideoDR. Ablation studies further reveal that simply reducing the number of tool calls is insufficient for optimal performance; rather, the key to improvement lies in explicitly learning which tool calls are truly necessary.

Critical strengths
  1. Clear and important problem formulation: The paper identifies a practical limitation of existing Video Deep Research agents: more tool calls do not necessarily lead to better performance, especially when grounding and retrieval tools are noisy. Framing the problem around tool necessity is both intuitive and relevant to broader agent research.
  2. Adaptive and model-aware tool use: AdaVDR does not rely on a fixed workflow. Instead, it adapts tool selection based on task type and model capability, which is a meaningful step toward more efficient and capable multimodal agents.
  3. Well-designed training pipeline: The paper combines adaptive trajectory construction, selective reflection, SFT, and NASA-GRPO into a coherent training framework. In particular, the model-conditioned necessity filtering and tool-level credit assignment are well aligned with the paper’s motivation.
  4. Strong empirical results and useful ablations: AdaVDR achieves substantial gains on both VDR-EE and VideoDR, and the ablation studies clearly show that simply reducing tool calls is insufficient; explicitly modeling tool necessity provides the largest benefit.
Critical weaknesses
  1. Limited scale of the new benchmark: VDR-EE contains only 250 questions, which may be too small to fully establish generalization across diverse video domains and tool-use patterns.
  2. Potential dependence on the target model: The tool necessity filtering is explicitly model-conditioned, meaning that trajectories
  3. Generality beyond Video Deep Research is not fully demonstrated: Although the core idea of capability-aware tool use is broadly applicable, the experiments are mainly focused on video deep research. It remains unclear how well NASA-GRPO and model-conditioned necessity filtering transfer to other multimodal or general-purpose tool-using agents.
Questions for authors
  1. How well does the model-conditioned tool necessity filtering generalize to different backbone models?

  2. What is the additional computational cost of the adaptive trajectory construction and NASA-GRPO training?

  3. Can the authors provide more analysis on when the reflection mechanism succeeds or fails?

4 comments
AuthorsOpenReview ↗

Response (1/3): W1 (VDR-EE Scale and Coverage) and W2 (Adaptation to New Backbones)

W1: Limited scale of the new benchmark: VDR-EE contains only 250 questions, which may be too small to fully establish generalization across diverse video domains and tool-use patterns.

VDR-EE is comparable in scale to existing video deep research benchmarks. As shown in Table below, recent benchmarks typically contain 200–307 questions, while VDR-EE includes 250 manually verified questions. Thus, the size of VDR-EE is consistent with current evaluation practices in video deep research. Some of these benchmarks were not publicly available at the time of submission, so we did not evaluate AdaVDR on them in the original manuscript.

BenchmarkQuestions
VideoDR-Bench [1]200
VDR-EE (ours)250
VideoSearch-QA [2]278
VideoRover-Bench [3]300
OneSearch-Video-Bench [4]307

VDR-EE covers seven semantic domains and includes entity-centric and event-centric questions across a range of video lengths. These questions require different tool-use patterns, such as identifying a visual entity or localizing an event before retrieving external information. Appendix D summarizes the benchmark composition, and Appendix E.1 reports accuracy by domain and video length (Tables 8 and 9).

References

[1] Video-DeepResearch: Towards the Next-Generation Multimodal Deepresearch Agent. arXiv:2608.03979.

[2] VideoSearcher: Empowering Video Deep Research with Multi-Tool Agentic Reasoning via Reinforcement Learning. EMNLP 2026.

[3] Thinking Beyond Videos: Unifying Video Reasoning and Deep Research for Open-World Video Agents. arXiv:2608.23329.

[4] OneSearch-VL: Unified Multimodal Deep Research Agent for Image and Video. arXiv:2610.12419.

W2: Potential dependence on the target model: The tool necessity filtering is explicitly model-conditioned, meaning that trajectories

Model-conditioned necessity filtering is essential because backbones differ in their video understanding capabilities and internal knowledge, making the same tool call necessary for some models but redundant for others. Moreover, our training trajectories can be easily adapted to other backbones. We will release the complete trajectories. The necessity filtering and post-filtering rewriting prompts are already provided in Appendices J.6 and J.8, respectively. By applying this pipeline to a new target model, model-specific training trajectories can be readily constructed. We further validate the extensibility of the proposed tool necessity filtering through experiments on additional backbones.

First, model-conditioned filtering is necessary to provide appropriate tool-use supervision for different backbones. Backbones differ in their video understanding capabilities and internal knowledge, so the same tool evidence may be redundant for one model but necessary for another. Using fixed necessity labels across backbones may therefore encourage unnecessary tool calls or discourage essential ones, leading to suboptimal tool-use behavior. This motivates our model-conditioned filtering strategy. Existing methods also adopt model-conditioned approaches to assess tool necessity, providing further support for the feasibility of this design. We discuss the detailed differences in the Related Work section.

Second, our training trajectories can be easily adapted to different backbones. We will release the complete trajectories. The necessity filtering and post-filtering rewriting prompts are already provided in Appendices J.6 and J.8, respectively. To adapt to a new backbone, we only need to rerun necessity filtering on the existing trajectories using the target model, then apply the rewriting prompt to construct new reasoning trajectories from the retained evidence and tool calls. No additional data collection or evidence acquisition is required. The appendix already includes the initial trajectory generation prompt (J.4), the pre-execution trajectory refinement prompt (J.5), and the tool necessity filtering prompts (J.6). In the revised manuscript, we have added the post-filtering SFT reasoning prompt (Appendix J.8).

We further evaluate AdaVDR on additional backbones to validate its extensibility. We conduct experiments with Qwen3-VL-8B-Instruct and Qwen3.5-35B-A3B on both VideoDR and VDR-EE. As shown in Table, AdaVDR consistently improves accuracy across both backbones and benchmarks. These results demonstrate that our approach can be effectively extended to backbones with different capabilities.

BackboneVideoDR (Agentic)VideoDR (AdaVDR)GainVDR-EE (Agentic)VDR-EE (AdaVDR)Gain
Qwen3-VL-8B-Instruct30.0058.00+28.028.4048.60+20.2
Qwen3.5-35B-A3B42.0069.00+27.046.4058.40+12.0

We will release the complete trajectories before filtering. Appendix J.6 provides the necessity-filtering prompts.

AuthorsOpenReview ↗

Response (2/3): W3 (Generalization to New Tasks) and Q1 (Generalization Across Backbones)

W3: Generality beyond Video Deep Research is not fully demonstrated: Although the core idea of capability-aware tool use is broadly applicable, the experiments are mainly focused on video deep research. It remains unclear how well NASA-GRPO and model-conditioned necessity filtering transfer to other multimodal or general-purpose tool-using agents.

Thank you for raising this point. Our method can be effectively extended to multimodal and general-purpose tool-using agents beyond video deep research. We designed additional experiments on WebShop (shopping) and SlideVQA (document question answering) to test whether our training data construction pipeline and NASA-GRPO transfer beyond video deep research. Both tasks involve tool sets distinct from those used in video deep research, such as click_element in WebShop and slide retrieval in SlideVQA. The results show that both our training data construction pipeline and NASA-GRPO can be adapted to tasks beyond video deep research, including both multimodal and general-purpose tool-use settings.

To evaluate transfer to other tool-use settings, we use Qwen3.5-9B on two tasks, with 2K SFT trajectories and 1K RL tasks per setting. WebShop uses search_products and click_element, and SlideVQA uses search (slide retrieval) and bbox (region cropping).

SettingWebShop scoreCalls/taskSlideVQA scoreCalls/task
Qwen3.5-9B Base31.28.2431.23.18
Full-trajectory SFT38.7 (+7.5)10.3638.0 (+6.8)4.26
Necessity-filtered SFT39.2 (+8.0)9.0838.3 (+7.1)3.51
Filtered SFT + standard GRPO41.7 (+10.5)9.7240.8 (+9.6)3.89
Filtered SFT + NASA-GRPO45.7 (+14.5)8.6743.4 (+12.2)3.36

The results show two consistent patterns across all two benchmarks. First, necessity-filtered SFT reduces average tool calls across all tw0 benchmarks while maintaining or slightly improving task scores relative to full-trajectory SFT. Second, starting from the same filtered SFT checkpoint, NASA-GRPO achieves higher scores with fewer tool calls than standard GRPO across all two settings. These results show that our data construction pipeline and NASA-GRPO can improve tool-use efficiency beyond video deep research, across tasks with different tool sets.

We have added the experimental setup and cross-task results for WebShop and SlideVQA to Appendix E.6 (Table 14) of the revised manuscript.

Q1: How well does the model-conditioned tool necessity filtering generalize to different backbone models?

Our tool necessity filtering procedure is well applicable across backbone models. When switching backbones, we use the same procedure and reassess which tool calls are necessary using the new backbone’s own responses. To evaluate this generalization, we applied AdaVDR to Qwen3-VL-8B-Instruct and Qwen3.5-35B-A3B. As to both backbones, AdaVDR improves accuracy over the corresponding agentic baseline on both benchmarks. These results show that the AdaVDR pipeline, including model-conditioned tool necessity filtering, generalizes across the backbones tested.

BackboneVideoDR (Agentic)VideoDR (AdaVDR)GainVDR-EE (Agentic)VDR-EE (AdaVDR)Gain
Qwen3-VL-8B-Instruct30.058.0+28.028.448.6+20.2
Qwen3.5-35B-A3B42.069.0+27.046.458.4+12.0
AuthorsOpenReview ↗

Response (3/3): Q2 (Computational Cost) and Q3 (When Reflection Succeeds or Fails)

Q2: What is the additional computational cost of the adaptive trajectory construction and NASA-GRPO training?

Thank you for raising this question. The added cost is small relative to the overall data construction and training time and is therefore entirely manageable. Evidence filtering adds about 2 hours in total for the entire training dataset. NASA-GRPO’s average time per training step approaches that of standard GRPO by the final reported interval, with most of the additional overhead occurring early in training.

1. Adaptive trajectory construction. The additional cost comes from model-conditioned evidence filtering. We collect candidate evidence using the standard procedure, then use a locally deployed model to assess its necessity before constructing trajectories from the retained evidence. This filtering step takes approximately 2 hours for the training dataset and requires no additional evidence retrieval calls.

2. NASA-GRPO training. The additional cost comes primarily from tool necessity verification and is concentrated early in training. By the final reported interval, NASA-GRPO’s average wall-clock time per training step is close to that of standard GRPO (37.1 vs. 36.0, as shown in the table). Without necessity weighting, bypassable calls that provide useful evidence still receive full process credit. This variant makes more tool calls, and its training steps generally take longer than those of standard GRPO.

Mean wall-clock time per training step (minutes)

Training stepsStandard GRPONASA-GRPO (w/o necessity weighting)NASA-GRPO
0–1933.741.648.1
20–3930.642.547.2
40–5932.546.241.2
60–7928.846.436.4
80-9036.049.137.1

We have added this computational cost analysis to Appendix G.3 of the revised manuscript, with a reference in Section 5.2. The appendix reports the one-time evidence filtering cost and compares the mean wall-clock time per training step across GRPO variants (Table 25). The additions are marked in blue.

Q3: Can the authors provide more analysis on when the reflection mechanism succeeds or fails?

Thanks for pointing it out. We analyzed when reflection successfully recovers key evidence and why some attempts fail. Overall, reflection successfully recovered key evidence in 71.4% of events on VideoDR and 68.3% on VDR-EE, while 28.6% and 31.7% remained unsuccessful, respectively, demonstrating its effectiveness. Among successful reflections, the most common strategies were refining search criteria and correcting the search target or visual localization. In contrast, most failures occurred when the model continued searching for the wrong fact or pursuing the wrong target.

1. When does reflection succeed?

Of all reflection events, 71.4% on VideoDR and 68.3% on VDR-EE recovered key evidence. The table below uses the number of reflection events that recovered key evidence on each benchmark as the denominator. Each event is assigned to one category based on its primary strategy.

Successful strategyVideoDRVDR-EE
Refining the search criteria and verifying facts. The model searches again after specifying the required attribute, condition, or relationship.52.4%85.2%
Correcting the search target or visual localization. The model corrects or disambiguates the entity or event, or locates the relevant content in the video again before searching.44.3%10.6%
Switching the retrieval path or source. After the original tool call fails, the model looks for evidence through another retrieval path or source.3.3%4.2%

2. What issues remain unresolved after reflection?

The percentages below are computed over the 393 reflection events across both benchmarks that failed to recover key evidence. The four categories below account for approximately 92% of unsuccessful reflection events.

  • Searching for the wrong fact or condition (41.0%). The model searches for the wrong attribute, condition, or version. For example, it repeatedly looks up the series score and never finds the single-game score asked for in the question.
  • Continuing to pursue the wrong entity, event, or visual content (34.9%). After retrying, the model still fails to locate the target referred to in the question.
  • Insufficient evidence, conflicting sources, or disputed judgments (9.2%). After retrying, the evidence remains insufficient, the sources give conflicting information, or the original evaluation judgment is disputed.
  • Errors in comparing or integrating multiple pieces of information (6.9%). The model fails to compare multiple targets correctly or to connect information from different sources.

We have added this analysis to Appendix F.3 (Reflection Analysis) of the revised manuscript, with a reference in Section 6.3. Tables 21 and 22 summarize the strategies that recovered key evidence and the errors that persisted after reflection, respectively. The additions are marked in blue.

AuthorsOpenReview ↗

A Brief Follow-Up: We Would Be Grateful for Any Further Feedback

Dear Reviewer SkVv,

Thank you again for your careful review and detailed questions. As the discussion period draws to a close, we would like to briefly summarize our responses to your questions and the additional results we shared in the rebuttal.

  • W1: VDR-EE coverage. VDR-EE covers diverse video domains and tool-use patterns. To address your concern about its scale, we summarized its coverage and compared its size with existing video deep research benchmarks.
  • W2 & Q1: Ease of transfer and generalization across backbones. The model-conditioned pipeline adapts to different backbones. To address your question, we applied the pipeline to two additional backbones and found that AdaVDR improved accuracy on both benchmarks with each.
  • W3: Transfer to other tasks. The data construction pipeline and NASA-GRPO also transfer beyond video deep research. Our experiments on WebShop and SlideVQA showed gains on both tasks using their respective tool sets.
  • Q2: Data Construction and Computational cost. The extra verification overhead is concentrated early in RL training. To address your question about cost, we quantified data construction costs and measured training step times. NASA-GRPO’s step time approached that of standard GRPO later in training.
  • Q3: When reflection helps and fails. Reflection often recovers key evidence by refining searches or correcting the target. In response to your question, we analyzed successful and failed reflection events and found that common failures involved pursuing the wrong fact or finding insufficient evidence.

We would sincerely appreciate even a brief note on whether our responses have helped address your concerns. We would be glad to clarify any remaining points. Thank you again for your time and consideration.

Sincerely,

The Authors

Reviewer_o2FzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents AdaVDR, a video deep research agent with adaptive tool use and reflection. The main idea is to skip unnecessary grounding/retrieval steps based on the model's own capabilities, and retry when intermediate evidence is unreliable.

The authors build model-conditioned training trajectories and propose NASA-GRPO for tool-level credit assignment. They also introduce VDR-EE, a new benchmark with 250 questions. Results show fairly large gains over Qwen3.5-9B on VideoDR and VDR-EE.

Critical strengths
  1. The motivation makes sense. Different video questions need different tools, and unnecessary retrieval can introduce errors. I like the idea of checking tool necessity against the target model's own capability, rather than always following a fixed pipeline.

  2. The results are quite good, especially considering the 9B model size. The ablations in Table 3 are helpful, and the necessity analysis in Table 13 gives some evidence that the model is learning to use tools more appropriately.

Critical weaknesses
  1. I'm not fully convinced by the efficiency claim. In Table 2, AdaVDR actually uses considerably more tools than the base model. Average turns increase from 5.12 to 11.67 on VideoDR, and 5.70 to 13.26 on VDR-EE. I understand that the necessity weight reduces redundant calls compared with the RL variant without it, but this is not quite the same as improving overall efficiency. Actual latency/cost numbers would help.

  2. I have some concerns about the theory in Sec. 3.2. The Markov assumption used in Appendix B seems questionable since external search can provide information not contained in the video/question. Also, a negative lower bound on the risk difference doesn't imply that using tools improves prediction. I think the theoretical discussion is a bit stronger than what the theorem actually proves.

Questions for authors

Can the authors report actual inference latency or an accuracy-vs-tool-budget comparison? This would help clarify the efficiency claim.

Could the authors explain the Markov assumption in Appendix B, particularly for questions requiring external knowledge?

How stable is the tool necessity judgment? For example, would repeated bypass tests produce similar labels? I wonder how much a single model prediction can tell us about whether a tool is really unnecessary.

4 comments
AuthorsOpenReview ↗

Response (1/2): W1 & Q1 (Tool-Use Efficiency and Inference Latency)

W1 & Q1: I'm not fully convinced by the efficiency claim. In Table 2, AdaVDR actually uses considerably more tools than the base model. Average turns increase from 5.12 to 11.67 on VideoDR, and 5.70 to 13.26 on VDR-EE. I understand that the necessity weight reduces redundant calls compared with the RL variant without it, but this is not quite the same as improving overall efficiency. Actual latency/cost numbers would help. Can the authors report actual inference latency or an accuracy-vs-tool-budget comparison? This would help clarify the efficiency claim.

The increased tool calls in AdaVDR mainly arise from reflection, which introduces additional calls to correct unreliable evidence. We conduct additional experiments by removing reflection data while retaining the complete adaptive tool-use method. The results show that AdaVDR reduces tool calls while maintaining strong accuracy, demonstrating its efficiency. Furthermore, even with reflection, AdaVDR achieves lower inference latency than existing specialized video deep research agents, further supporting its practical efficiency.

Tool-use efficiency without reflection. We conduct an additional experiment by removing only the reflection data from training while retaining the full adaptive tool-use method. We further compare the number of tool calls and accuracy on VideoDR, as shown in Table. The average number of calls per question drops from 10.67 to 3.69, below Qwen3.5-9B Base (4.12), VideoSearcher-8B (3.99), and Video-DeepResearch-30B-A3B (6.57). Meanwhile, AdaVDR without reflection achieves 60.0% accuracy, outperforming Qwen3.5-9B Base (37.0%) and VideoSearcher-8B (53.0%). These results demonstrate that our method can reduce tool calls while maintaining high accuracy.

ModelAccuracyVisual callsText callsTotal calls
Qwen3.5-9B Base37.00.893.234.12
AdaVDR-9B SFT + RL (without reflection data)60.02.311.383.69
AdaVDR-9B SFT + RL (with reflection data)66.03.347.3310.67
VideoSearcher-8B53.02.531.463.99
Video-DeepResearch-30B-A3B62.02.334.246.57

Inference latency analysis. We further measure the actual wall-clock inference latency and output token counts on both VideoDR and VDR-EE, as shown below. Despite the additional tool calls introduced by reflection, AdaVDR achieves lower inference latency than both specialized baselines on VideoDR (98.3s vs. 153.8s/174.9s) and VDR-EE (101.2s vs. 162.7s/236.7s). This efficiency stems from our SFT and RL training, which reduces reasoning output length and thereby helps offset the latency overhead of additional tool calls. Results demonstrate that AdaVDR maintains competitive efficiency while benefiting from reflection-enhanced accuracy.

ModelVideoDR output tokensVideoDR latency (s)VDR-EE output tokensVDR-EE latency (s)
Qwen3.5-9B Base1,44971.41,82070.5
AdaVDR-9B1,78698.31,381101.2
VideoSearcher-8B5,403153.81,022162.7
Video-DeepResearch-30B-A3B2,642174.93,480236.7

In the revised manuscript, we clarify that necessity weighting reduces redundant tool calls relative to the RL variant without necessity weighting, while reflection introduces additional calls to correct unreliable evidence. In Section 6.3, we have added a tool-use comparison with and without reflection data (Table 3) and a wall-clock inference latency comparison with existing baselines (Table 4). Both tables and their accompanying discussion are marked in blue.

AuthorsOpenReview ↗

Response (2/2): W2 & Q2 (Theoretical Assumptions) and Q3 (Stability of Tool Necessity Judgments)

W2 & Q2: I have some concerns about the theory in Sec. 3.2. The Markov assumption used in Appendix B seems questionable since external search can provide information not contained in the video/question. Also, a negative lower bound on the risk difference doesn't imply that using tools improves prediction. I think the theoretical discussion is a bit stronger than what the theorem actually proves. Could the authors explain the Markov assumption in Appendix B, particularly for questions requiring external knowledge?

Thank you for pointing out the need to clarify the scope and interpretation of our theoretical analysis. We address the two issues separately below.

Markov assumption. We agree that deriving a tool query from the video/question does not by itself guarantee Y→X→ZY \to X \to Z, since external retrieval also accesses a knowledge source beyond X=(V,q)X=(V,q). Importantly, the risk decomposition in Appendix B does not require this assumption: Rtool−Rbase=−I(Y;Z∣HM)+ξtool−ξM.R_{\mathrm{tool}}-R_{\mathrm{base}}=-I(Y;Z\mid H_M)+\xi_{\mathrm{tool}}-\xi_M. Retaining the information-gain term explicitly, the subsequent argument gives Rtool−Rbase≥2ϵδ2−I(Y;Z∣HM)−ξM.R_{\mathrm{tool}}-R_{\mathrm{base}}\ge 2\epsilon\delta^2-I(Y;Z\mid H_M)-\xi_M. In the revised manuscript, we present this form for general external retrieval and explicitly state that replacing I(Y;Z∣HM)I(Y;Z\mid H_M) with ΓM\Gamma_M requires the additional Markov assumption.

Interpretation of the lower bound. We agree that a negative lower bound does not imply that tool use reduces prediction risk. A positive lower bound provides a sufficient condition for increased risk, whereas a non-positive bound is inconclusive. We have remove the converse interpretation in Section 3.2 in the revised manuscript. The general decomposition instead characterizes the trade-off between the information gained from tools and the change in model approximation error.

Q3: How stable is the tool necessity judgment? For example, would repeated bypass tests produce similar labels? I wonder how much a single model prediction can tell us about whether a tool is really unnecessary.

R: Thank you for raising this question. Our tool necessity judgments are robust to increases in the number of sampled responses. In practice, we sample eight responses for each bypass check and label a tool call as bypassable only if all eight responses pass. The single-response rule agrees with the 8/8 rule on only 84.5% of labels. We therefore label a tool call as bypassable only if all eight sampled responses pass. In our supplementary experiments, increasing the sample count to 16 or 32 yielded labels that remained highly consistent with those from the original 8/8 rule, supporting the robustness of our necessity judgments.

In a supplementary experiment on the same 200 bypass checks, we increased the sample count from 8 to 16 and 32, requiring all responses to pass in each setting. As shown below, both larger sample sizes yield labels that are highly consistent with the original 8/8 rule, supporting the robustness of our necessity judgments.

ComparisonLabel agreement
8/8 vs. 16/1698.0% (196/200)
8/8 vs. 32/3296.5% (193/200)

To further reduce the impact of rare labeling errors during GRPO training, we assign bypassable calls a necessity weight of 0.5. A bypassable call that provides useful evidence therefore still receives half of the evidence reward it would otherwise earn.

We have clarified the 8/8 bypass rule in Sections 4.2 and 5.2 and added the label agreement results for 16 and 32 sampled responses to Appendix F.2 (Table 19). These revisions are marked in blue.

AuthorsOpenReview ↗

A Brief Follow-Up: We Would Be Grateful for Any Further Feedback

Dear Reviewer o2Fz,

Thank you again for your thoughtful questions. As the discussion period draws to a close, we would like to briefly summarize our responses to your questions, including the main findings from the additional experiments and analyses we conducted to address them.

  • W1 & Q1: Inference efficiency. Our ablation shows that most of AdaVDR’s additional tool calls come from reflection. To address your question about actual inference costs, we measured wall-clock latency and output token counts. With reflection enabled, AdaVDR was still faster than both specialized baselines on both benchmarks.
  • W2 & Q2: Scope of the theory. The exact risk decomposition holds without the Markov assumption. We retained the information-gain term to account for external retrieval and clarified that the lower bound alone leaves the direction of the risk change undetermined.
  • Q3: Stability of tool-necessity judgments. The necessity labels remain stable across sampling budgets. Following your suggestion, we repeated the same bypass checks with more samples and compared the resulting labels.

We would be grateful for any further thoughts on whether our responses helped address your main concerns. We would be happy to clarify any remaining points. Thank you again for your time and constructive feedback.

Sincerely,

The Authors

Dear Reviewer o2Fz,

With less than 12 hours remaining in the discussion period, we wanted to briefly follow up on our rebuttal. We would appreciate any feedback you have time to share and would be happy to clarify any remaining points before the discussion closes.

If our response has helped address your concerns, we hope the additional results and clarifications will be helpful as you finalize your assessment.

Thank you again for your time and consideration.

Sincerely,

The Authors

Comments on the paper

3 comments
AuthorsOpenReview ↗

Follow-up Comment to Reviewers

Dear Reviewers,

Thank you again for your time and constructive feedback. In response to your comments, we have provided the following experiments and clarifications:

  • Reviewer o2Fz: Inference-latency and output-token analyses; clarification of the theoretical assumptions and risk decomposition; stability checks for tool-necessity judgments across sampling budgets.
  • Reviewer SkVv: Clarification of VDR-EE’s scale and coverage; experiments on two additional backbones; transfer experiments on WebShop and SlideVQA; data-construction and training-cost analyses; analysis of successful and failed reflection events.
  • Reviewer GPk6: Comparison with prior work on adaptive tool use; necessity-label stability checks and counterfactual call-removal experiments; inference-efficiency analyses; ablations isolating necessity weighting and comparisons with simpler reward variants.
  • Reviewer 5LRj: Clarification of the necessity criterion’s novelty and reliability; generalization experiments with unseen tool combinations and across tasks; clarification of the prompt-tuned baseline and matched evaluation settings; analysis of the benefits and cost-effectiveness of task-specific training.

As the discussion period draws to a close, we would be very grateful for any feedback on whether these responses have addressed your main concerns and whether they inform your overall assessment. If any questions remain, we would be happy to provide further clarification during the remaining discussion period.

Thank you again for your time and consideration. Even a brief update would be greatly appreciated.

Sincerely,
The Authors

AuthorsOpenReview ↗

Our Earlier Rebuttal Included Results on Generalization to Unseen Tools on WebShop and SlideVQA in Response(3/4)

Thank you for your feedback. In Response (3/4), Our earlier rebuttal has reported additional experiments on AdaVDR’s generalization to unseen tools on WebShop (online shopping) and SlideVQA (document question answering).

  • WebShop. AdaVDR uses search_products to find candidate products in the simulated store’s catalog and click_element to open product pages, select options, and make purchases. It scored 38.2, compared with 31.2 for the base model.
  • SlideVQA. AdaVDR uses search to retrieve relevant slides from the slide deck provided with the question and bbox to crop regions such as tables or charts for closer inspection. It scored 39.1, compared with 31.2 for the base model.
AuthorsOpenReview ↗

Clarifying AdaVDR’s Novelty Relative to Prior Work

Thank you for your feedback. To our knowledge, AdaVDR is the first work to study tool credit assignment and model-conditioned tool necessity in video deep research (VDR). Existing methods are difficult to apply directly to VDR for three main reasons.

  1. They lack a training data construction pipeline tailored to VDR.
  2. Trajectory-level supervision is too coarse, and evaluating calls individually can overlook dependencies between them (e.g., Search-R1 and EAPO use outcome rewards, while TRACE assigns credit at the turn level).
  3. Different tool types receive positive rewards at different rates, which can lead to an imbalance in the learning signal (IG-Search and STAMP assign credit primarily to text retrieval actions, and VDR also needs to account for reward differences between visual grounding and search).

To address these challenges, AdaVDR introduces a model-conditioned data construction pipeline and an RL training method tailored to VDR.

  1. Model-conditioned data construction. We construct training data based on which tool calls the current model needs to acquire intermediate evidence.
  2. Reinforcement learning for VDR. NASA-GRPO includes three components.
    • Segment-guided credit assignment. We assign credit to multiple calls that jointly acquire a piece of evidence.
    • Model-conditioned necessity testing. Building on the evidence segments, we assess which calls the current model can bypass, using distinct tests for visual grounding and search. Calls deemed bypassable receive less credit.
    • Tool-type reward normalization. We normalize process rewards separately for each tool type to prevent tools that receive positive rewards more easily from dominating the learning signal.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.