- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper studies tool overuse in RL-trained KBVQA agents, which often collapse to a fixed image-search-then-text-search workflow even when retrieval is unnecessary. It first probes the model’s entity recognition, entity-conditioned answering, and direct answering capabilities for each training instance. It then converts these capability estimates into preferred tool-use paths and uses them as an additional RL reward, encouraging the agent to call external tools only when its own capabilities are insufficient.
Critical strengths
- The paper identifies a practically relevant failure mode in RL-based KBVQA agents and motivates the use of a capability-aware tool well.
- The capability probing and path-based reward provide an intuitive and interpretable mechanism for controlling tool use.
- The paper provides useful behavioral analyses that help illustrate how capability-aware supervision changes the agent’s tool-use patterns.
Critical weaknesses
- Offline capability probing may become stale as the model’s capabilities evolve during RL, since path preferences are fixed before training.
- The method may generalize poorly to new tools or task structures because it relies on a task-specific capability decomposition and four predefined path categories.
- The approach may be difficult to scale to less annotated datasets because probing requires ground-truth answers and relevant entity labels.
- The method’s robustness in realistic agent settings remains unclear because evaluation is limited to closely related KBVQA benchmarks with relatively simple tool spaces.
Questions for authors
- The benefit of the proposed RL reward is not fully disentangled from direct path supervision. A comparison with simpler alternatives, such as directly supervising the preferred path or using a routing classifier, would help isolate the contribution of the RL formulation.
- The authors should explain how the framework would extend to richer tool spaces or tasks where the reasoning structure cannot be reduced to direct, image, text, and image-plus-text paths
- The capability boundary is estimated before RL and remains fixed even though the model’s capabilities may change during training. The authors should provide stronger empirical evidence on capability drift, or compare against dynamic or periodically updated probing.
- The current evaluation remains within relatively similar KBVQA settings and provides limited evidence for generalization to more realistic VQA scenarios. I strongly recommend evaluating on SUPERGLASSES, which is built from real-world egocentric images captured by smart glasses and covers diverse domains and query types with annotated search trajectories and reasoning processes.
3 comments
Thank you for the response. However, the restricted action space (Q2) is a fundamental limitation of the proposed method. In addition, the limited evaluation (Q4) does not sufficiently demonstrate the method's efficiency and generalizability. Therefore, I will maintain my original score.
Response to Reviewer civd [Supplementary Response]
For Restricted Paths
Thank you for your follow-up reply! We conducted a evaluation on the same MMHops test examples.
| Probed target group | Forced Direct | Forced Image | Forced Text | Forced Image+Text |
|---|---|---|---|---|
| Direct | 62.24 / 0.00 | 62.10 / 1.67 | 62.05 / 2.24 | 61.90 / 3.60 |
| Image-only | 28.42 / 0.00 | 55.52 / 2.13 | 33.67 / 2.15 | 54.28 / 3.69 |
| Text-only | 24.46 / 0.00 | 28.13 / 2.40 | 52.57 / 2.56 | 52.68 / 3.74 |
| Image+Text | 11.39 / 0.00 | 18.27 / 2.49 | 13.43 / 2.68 | 44.43 / 3.72 |
The results support an accuracy–cost interpretation of the probed targets. BoundarySA therefore does not indiscriminately suppress retrieval: it retains the full path when both tools are needed.
Allowing both ImageSearch and TextSearch gives the model access to more complete external evidence and can therefore remain beneficial for some examples. However, full retrieval is not uniformly advantageous across capability groups. Compared with the capability-compatible paths, forcing Image+Text decreases accuracy from 62.24% to 61.90% in the Direct group and from 55.52% to 54.28% in the Image-only group, while also requiring substantially more tool calls. This observation is consistent with our discussion in the original Tool Calls and Top- ablation: excessive retrieval may introduce irrelevant information and retrieval noise, increasing the burden on evidence summarization and subsequent reasoning and potentially interfering with questions that the model could already solve directly or with a single tool. In contrast, capability-aware routing selects tools according to the capability gap of each example, reducing the average number of calls from 3.71 to 3.10 while maintaining overall answer accuracy. Its benefit therefore does not come from simply minimizing search, but from avoiding redundant retrieval and its potential noise while preserving necessary evidence acquisition.
For More Evaluation
Thank you for this valuable suggestion. We agree that SUPERGLASSES is an important benchmark for evaluating VLM agents in realistic egocentric scenarios.
Although SUPERGLASSES is related to our task through its use of external knowledge retrieval, its focus on heterogeneous, real-world egocentric smart-glasses scenarios differs in scope from the long-tail, knowledge-intensive KBVQA setting studied in this work. We consider this an important direction and plan to extend BoundarySA to broader and more complex scenarios in future work.
Response to Reviewer civd
We thank the reviewer for the thoughtful questions.
Q1. Are all three capability probes necessary?
We added two simpler alternatives:
Correct-gated efficiency: fewer tool calls are rewarded only when the final answer is correct.
Direct-only probing: only pd is used to divide examples into Direct and Non-direct groups, without distinguishing Image-only, Text-only, and Image+Text requirements.
Direct-only probing improves over the Ans+format baseline because it can identify some examples that do not require retrieval. However, it cannot distinguish whether a Non-direct example requires ImageSearch, TextSearch, or both tools. Consequently, its Non-direct predictions remain concentrated on the conservative Image+Text workflow. Full BoundarySA improves Direct-only probing while reducing average tool use , demonstrating the complementary value of pe and pa.
Correct-gated efficiency achieves lower accuracy than BoundarySA because it applies the same “fewer calls are better” preference to every correctly answered instance, without modeling why different instances require different numbers or types of tool calls. For example, a correct Direct instance should ideally use no tools, whereas a correct Image+Text instance may require both ImageSearch and TextSearch, and therefore inherently needs more calls. Correct-gated efficiency observes only answer correctness and trajectory length; it cannot distinguish the necessary multi-tool calls of the latter from genuinely redundant retrieval. This can disrupt the appropriate multi-tool behavior of instances that genuinely require both tools.
In contrast, BoundarySA provides instance-specific supervision: three probes therefore distinguish zero-tool, single-tool, and multi-tool requirements, allowing the model to reduce redundant calls without uniformly penalizing necessary retrieval. This complementary capability decomposition explains why BoundarySA achieves a better accuracy–efficiency trade-off than either Correct-gated efficiency or Direct-only probing.
Q3. Does the pre-RL capability target remain applicable after SFT and RL?
We reported results further as follows:
All three capability scores increase after SFT and RL, confirming that the model’s capability improves during training. This is consistent with our original formulation. As stated in the Capability-Boundary Shifts during Training paragraph, BoundarySA does not assume that the target remains fixed.
Our appendix shows that, even when the boundary shifts, the original answer and format rewards are preserved:
RBoundarySA=rans+βrformat+λransrbound.
The additional boundary term is non-negative and gated by answer correctness. It therefore cannot reduce the baseline reward, and an incorrect trajectory cannot receive a bonus merely by matching the initial target.
The results provide direct empirical support: 92.00% of the SFT targets and 86.14% of the final RL targets remain identical to their Base targets. Moreover, BoundarySA improves accuracy from 43.80% to 47.91% while reducing average tool calls from 3.90 to 3.10. Thus, the initial target remains sufficiently stable to provide useful guidance while preserving the model’s ability to improve during training.
Q2,Q4: Questions about more tools and datasets.
In future work, we plan to extend our investigation of capability boundaries to a broader range of datasets and more diverse tool settings.