- Rating
- 4
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies when RL-trained crop-and-zoom tools improve VLM performance and why their use is often decorative.
The authors first identify a pixel-budget gate: the value of a correct crop is determined by the gap between an image’s native resolution and the model’s working pixel budget. Furthermore, by decomposing the tool-use protocol, they show that although presenting a crop improves performance, delivering the same crop through a standard tool-call exchange can introduce substantial performance losses. To measure whether a model actually relies on the returned visual content, the paper proposes counterfactual ablation credit, which replaces the crop with a shifted region and credits the call only when the answer changes. The authors find that most calls made by an RL-trained model are non-causal despite accurate localization. Finally, experiments with above-budget data and alternative advantage estimators show that causal rewards alone do not produce durable, selective tool use.
Critical strengths
-
Important and well-supported findings. The paper challenges a common assumption that frequent or accurate tool calls necessarily indicate effective tool use. Through a carefully designed set of experiments, it examines the problem from several complementary perspectives, such as the pixel-budget gate, the cost of tool-call serialization, etc., and reaches the conclusion that current RL-trained crop-and-zoom behavior is largely decorative and becomes genuinely useful only under specific pixel-budget, presentation, and credit-assignment conditions.
-
New insights for evaluating and improving tool use. The paper provides a useful perspective for studying whether tool calls really contribute to a model’s predictions, rather than merely correlating with successful outcomes. Its evaluation framework could be extended to other visual or multimodal tools, and motivate future work targeting more effective tool-using, such as tool-necessity training, better credit assignment, and more selective policies that invoke tools only when they provide additional information.
Critical weaknesses
-
Limited generality of the training conclusions. Finding 3 focuses on the behavior of RL training, yet almost all training experiments are conducted on Qwen2.5-VL-3B. The current evidence is insufficient to establish that the observed collapse or decorative tool use generalizes broadly across different VLM backbones.
-
Minor unclarity of the presentation. The paper contains many experiments, including the initial observation of decorative tool use and the three findings. Each experiment has substantially different protocols, datasets, metrics. Though the experiment setup is explained very clearly in the text, I would recommend that the paper can present a simple overview figure to show how the experiments are implemented and related, together with concrete case examples showing the complete pipeline of each experiment, which would make the methodology and conclusions much easier to follow.
Questions for authors
-
(See Weakness 2) Could the authors add an overview diagram of the experimental pipeline or several concrete cases showing how each experiment is conducted instead of solely using textual explanations?
-
The substantial performance degradation caused by the tool-call protocol in Finding 2 is somewhat counterintuitive. Could the authors provide a representative example, including the exact tool-call serialization, model outputs, and repeated-call behavior, to illustrate how this failure occurs?
-
For locally-deployed small VLMs, it may not always be necessary to impose a tight working pixel budget. However, when using large proprietary models (e.g., GPT6-Astra), the number of visual tokens greatly affect the fee paid. I wonder whether the conclusions of Findings 1 and 2 also apply in this setting? I would be grateful if the authors could provide practical guidance on crop-and-zoom tool-using for large proprietary models (e.g., whether users should directly downsample full images or first crop them and use crop-only inputs to save token cost).