- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces ReTool-Video, a video agent with a Planner and an LLM-based Resolver. The Planner decides what evidence is needed and when to answer. The Resolver turns these requests into actual tool calls by fixing arguments, choosing suitable tools, or breaking a request into smaller steps. The authors also build a library of 134 tools and train the Planner with GRPO. The system is evaluated on three video-understanding benchmarks.
Critical strengths
-
The Planner can ask for the evidence it needs without getting every tool name and parameter right. The Resolver handles these details and returns the results for further reasoning.
-
Without RL, adding it raises MLVU accuracy from 61.5% to 74.6%. The paper also tests another backbone, providing some evidence that the gains are not limited to Qwen models.
-
Operations such as filtering and merging time intervals are useful for video questions. Removing meta tools lowers accuracy on all three benchmarks.
Critical weaknesses
-
The Planner acts as the main agent controller, while the Resolver translates requests into tool calls. Argument repair, tool substitution, and recursive task decomposition are useful, but they build on familiar ideas from hierarchical agents.
-
Freezing the Resolver provides a stable interface, as the authors explain, but a sensible Planner request can still be poorly translated or unnecessarily decomposed. The Planner may learn to phrase requests around these weaknesses, while the Resolver itself receives no training signal to improve. Since tool grounding is central to the method, the paper would benefit from explaining when this separation helps and when it becomes a bottleneck.
-
The Resolver checks whether a tool call can run, but this does not ensure that it fulfils the Planner’s request: Appendix C.10 gives an example where frame-retrieval requests were repeatedly mapped to OCR without obtaining new evidence.
Questions for authors
On MLVU, w/o Recursion, in Table 3 scores 63.2%, while One-step grounding in Table 4 scores 72.8%. It is unclear how these settings differ, and the authors acknowledge that inference compute is not strictly matched. Figure 6 also assigns 58% of calls to visual/video tools, whereas Video_Clip_QA and Inspect_Frame alone total 81.55% in Table 8. Different settings or counting rules may explain these gaps, but they need to be stated so readers can interpret the results.
Thanks for your response. My concerns have been addressed. Now I've increased my rating to 3.