- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces AdaVDR, a video agent that selects tools and reflects on unreliable intermediate results. The approach constructs model-conditioned tool-use trajectories for SFT and proposes NASA-GRPO, which assigns tool-level rewards based on evidence contribution and tool necessity. The authors further present VDR-EE, a new benchmark for video deep research.
Critical strengths
Important problem: The paper tackles unnecessary tool invocation and error propagation in video deep research—practically relevant challenges for multimodal agents.
Comprehensive experiments: The paper includes ample ablations, tool-use analyses, and evaluations on additional image and video benchmarks, all of which support the effectiveness of the overall framework.
Critical weaknesses
Limited novelty and missing related work: The core idea of model-conditioned tool necessity has been explored in prior works—like Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use—yet this work is not discussed.
Questionable tool necessity criterion: A single correct prediction does not guarantee that the tool is unnecessary. This may introduces unreliable labels into training.
Insufficient justification for task-specific training: The paper lacks comparisons with carefully prompt-optimized agents that use the same tools. In addition, stronger general-purpose models already surpass AdaVDR on VDR-EE.
Questions for authors
-
See weakness 1.
-
Can AdaVDR perform on unseen tools?
-
See weakness 3.
6 comments
Response(2/4): W2 & Q1 (Tool Necessity Label Stability and Counterfactual Tool Removal)
W2 & Q1: Questionable tool necessity criterion: The proposed method considers a tool unnecessary if the model correctly predicts its target information without invoking it. However, a single correct prediction does not guarantee that the tool is unnecessary (e.g., a model may answer correctly with 60% probability without the tool but achieve 95% accuracy with it). This potentially introduces unreliable necessity labels into both SFT and RL training. How reliable are its labels under repeated predictions or counterfactual tool removal?
R: Thank you for raising this question. In both SFT and RL, we label a call as bypassable only if all eight sampled responses pass. Our experiments show that the resulting labels are stable and reliable and that removing calls labeled as bypassable does not reduce final-answer accuracy in our evaluation. Together, these results support the rationale for our method and show that necessity-aware training can make tool use more efficient while improving final-answer accuracy.
1. Stability of necessity labels. We compared the 8/8, 16/16, and 32/32 rules on the same 200 bypass checks, requiring all sampled responses to pass in each setting. The 16/16 and 32/32 rules agreed with the 8/8 rule on 98.0% (196/200) and 96.5% (193/200) of labels, respectively. This high agreement indicates that eight samples are sufficient to obtain stable labels in our setting. During RL training, we sample the eight responses in parallel to limit the additional wall-clock latency of each bypass check.
| Comparison | Label agreement |
|---|---|
| 8/8 vs. 16/16 | 98.0% (196/200) |
| 8/8 vs. 32/32 | 96.5% (193/200) |
To limit the impact of potential labeling errors during RL, NASA-GRPO assigns bypassable calls a necessity weight of 0.5. A bypassable call that contributes useful evidence therefore retains half of its evidence reward.
2. Effect of necessity filtering and weighting on final-answer accuracy. We evaluate how removing calls classified as unnecessary from SFT trajectories affects final-answer accuracy. We also compare the full training pipeline with a variant that omits both SFT necessity filtering and RL necessity weighting.
| Setting | VideoDR | VDR-EE |
|---|---|---|
| SFT without necessity filtering | 54.0 | 46.4 |
| SFT with necessity filtering | 52.0 | 48.0 |
| SFT without necessity filtering + RL without necessity weighting | 62.0 | 53.2 |
| SFT with necessity filtering + RL with necessity weighting | 66.0 | 55.2 |
We evaluate necessity filtering at the SFT stage and jointly remove necessity filtering and necessity weighting in the end-to-end training ablation. Filtering alone has mixed effects across the two benchmarks. After RL, the full method improves accuracy by 4.0 percentage points on VideoDR and 2.0 percentage points on VDR-EE over the variant with both components removed. This complements the ablation analysis in Section 6.4 (Table 5), where adding necessity weighting improves accuracy while reducing the average number of interaction turns. These findings are consistent with Theorem 1 in Section 3.2, which shows that misleading tool outputs can increase prediction risk when the model already has sufficient information.
3. Effect of counterfactual tool removal on final-answer accuracy. To test whether calls labeled bypassable can be skipped at inference time, we evaluated Qwen3.5-9B on 85 questions associated with 200 evidence checks. Both conditions used the same checkpoint and pre-call contexts, and all other tools remained available. The first condition retained the original tool results. The second skipped calls that passed the 8/8 bypass check and withheld their results.
| Inference setting | Run 1 | Run 2 | Run 3 | Mean accuracy |
|---|---|---|---|---|
| Retain the original tool results | 30.6 | 28.2 | 31.8 | 30.2 |
| Skip calls labeled bypassable | 32.9 | 30.6 | 28.2 | 30.6 |
Mean final-answer accuracy remained similar after skipping bypassable calls (30.6 vs. 30.2). This provides direct evidence that the calls flagged by the 8/8 rule can be skipped with little change in aggregate accuracy in this setting. The base model still made these calls in its original trajectories, which motivates training it to skip them on its own.
We have clarified the 8/8 bypass rule and necessity weighting in Sections 4.2 and 5.2. We added the label agreement and counterfactual tool removal results to Appendix F.2 (Tables 19–20) and the training ablations to Appendix H.1 (Table 26). All changes are marked in blue.
Response(3/4): W4 & Q2 (Generalization to Unseen Tools) and W3 (Gains over a Prompt-Optimized Baseline)
W4 & Q2: Limited tool-Set generalization: Both trajectory construction and NASA-GRPO depend on predefined tool types and evidence-acquisition patterns. The paper does not evaluate whether the trained agent can adapt to newly introduced or upgraded tools without additional data construction or training, which limits the demonstrated extensibility. Can AdaVDR generalize to previously unseen tools without additional training? And how does its performance change when the available tool set is modified?
Thank you for raising this question. Without additional training, we replace the original video tools with unseen tools and outperform Qwen3.5-9B Base by 25.0 on VideoDR and 12.8 on VDR-EE. We also transfer AdaVDR-9B to WebShop and SlideVQA with their task-specific tools, outperforming the base model by 7.0 pp and 7.9 pp, respectively. These results show that AdaVDR can adapt to new tools on video tasks and transfer to new tasks with different tools, outperforming the base model in both settings.
1. Tool replacement on video tasks. On VideoDR and VDR-EE, we combine existing operations into three tools that were not used to train AdaVDR and use them to replace the corresponding original tools at inference time. AdaVDR-9B still achieves strong results with the replacement tools.
search_and_visitsearches the web and opens result pages in a single call.crop_search_and_visitcrops a region of a video frame, uses it for image search, and opens result pages in a single call.inspect_videoprovides a single tool for selecting frames, locating time segments, and cropping regions within frames.
| Benchmark | Qwen3.5-9B Base | AdaVDR-9B (ours) |
|---|---|---|
| VideoDR | 39.0 | 64.0 (↑ +25.0) |
| VDR-EE | 41.2 | 54.0 (↑ +12.8) |
2. Transfer to new tasks with task-specific tools. We further evaluate AdaVDR on WebShop and SlideVQA. Both models receive the same tools on each task. AdaVDR outperforms the base model on both benchmarks, demonstrating that it can transfer to new tasks with different tools without additional training.
- WebShop provides
search_productsto find products by keyword andclick_elementto click a product, option, or button on the current page. - SlideVQA provides
searchto retrieve page images from a text query andbboxto crop a region of a retrieved page using specified coordinates.
| Benchmark | Qwen3.5-9B Base | AdaVDR-9B (ours) |
|---|---|---|
| WebShop | 31.2 | 38.2 (↑ +7.0) |
| SlideVQA | 31.2 | 39.1 (↑ +7.9) |
We added the tool replacement and cross-task transfer results to Appendix E.7 (Tables 15 and 16) of the revised manuscript.
W3: Insufficient justification for task-specific training: Although AdaVDR outperforms the untrained Qwen3.5-9B agentic baseline, the paper lacks comparisons with carefully prompt-optimized agents that use the same tools.
All models evaluated in the agentic setting in Table 1 use the same prompts tuned for video deep research and have access to the same tools. We selected the shared prompt by comparing candidate prompts on a held-out development set with Qwen3.5-9B before evaluating on VideoDR and VDR-EE. Qwen3.5-9B and AdaVDR-9B also share the same backbone. Under this matched setup, task-specific training improves accuracy from 40.0% to 55.2% on VDR-EE and from 37.0% to 66.0% on VideoDR. We have clarifyed this evaluation setup in the revised manuscript.
Response(1/4): W1 & Q1 (How AdaVDR’s Tool Necessity Criterion Differs from Prior Work)
Thank you for pointing out the missing related work. We compare AdaVDR with Model-Adaptive Tool Necessity [1] and other adaptive tool-use and efficiency-aware RL methods below. Prior methods use prediction probabilities or answer correctness to assess tool utility at the task or call level. As discussed below, task-level feedback is sparse, and tool-level assessments can miss how several calls work together to recover evidence in video deep research. AdaVDR’s novelty lies in using verifiable evidence acquisition to provide segment-guided supervision, capturing the joint contribution of multiple tool calls, and introducing necessity-aware credit assignment to weight individual tool calls according to their necessity for the current model. In this way, AdaVDR provides more accurate and appropriate supervision for learning tool-use necessity.
Existing adaptive tool-use and efficiency-aware RL methods differ in how they select and reward tool calls. Model-Adaptive Tool Necessity [1] uses ten tool-free trials to assess whether the whole question requires tools and uses the resulting labels to train hidden-state probes that examine the gap between perceived necessity and actual tool use, rather than training the model itself to determine when it needs to use tools. Toolformer [2] retains calls that reduce prediction loss, even if the model can already recover the same information. Search-R1 [3] rewards final-answer correctness, EAPO [4] adjusts rewards by question difficulty and tool-call count, and CoBRA [5] compares answer rewards with and without tools. These trajectory-level signals give limited guidance on which calls can be skipped. TRACE [6] measures each call’s immediate contribution to the answer and may undervalue preparatory actions. Spurious Tool Use [7] uses an external judge to identify unnecessary calls and penalizes them in the trajectory reward. Its judgments leave open whether the current model can recover the same intermediate evidence after skipping a call.
However, video deep research often requires multiple pieces of evidence. The model may already know some of them and still need tools to obtain the rest. Task-level necessity judgments cannot distinguish these cases, final-answer rewards provide sparse feedback, and immediate call-level rewards can undervalue preparatory actions whose contribution becomes clear only after a later retrieval succeeds. AdaVDR therefore uses each piece of evidence as a verifiable intermediate target. We test whether the model can recover the same evidence after bypassing a call and use this judgment to filter redundant calls during SFT. During RL, segment-guided credit assignment groups calls that pursue the same evidence and gives them base credit when the segment recovers it, including preparatory calls. Necessity-aware credit assignment then adjusts that credit for each call. Necessary calls retain full credit, while bypassable calls retain half, reducing their contribution to the process advantage.
We have added these comparisons to the Related Work in Section 2.3 of the revised manuscript.
[1] Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use — NeurIPS 2026
[2] Toolformer: Language Models Can Teach Themselves to Use Tools — NeurIPS 2023
[3] Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning — COLM 2025
[4] Learning When Not to Act: Mitigating Tool Abuse in Agentic Reinforcement Learning — arXiv 2026
[5] CoBRA: Learning Tool-Use Boundaries via Counterfactual Margins — EMNLP 2026
[6] TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents — arXiv 2026
[7] Spurious Tool Use: When RL Agents Learn the Wrong Reason to Act — arXiv 2026