- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates LLM agents' overreliance on unreliable tool outputs. The authors evaluate 14 models across three tool types: web search, LLM sub-agents, and code executors, using controlled output corruption. Results reveal substantial adoption of incorrect tool outputs, even when models can independently produce correct answers. The paper further investigates this behavior through reasoning traces and evaluates mitigation strategies. Verification-based prompting demonstrates promising improvements in reliability.
Critical strengths
-
Important and practical problem. The paper highlights a critical reliability issue in tool-augmented LLM agents. Distinguishing tool-output adoption from the overriding of previously correct answers provides valuable insights.
-
Extensive empirical analysis. The evaluation spans diverse models, tools, and corruption settings. The investigation of reasoning behavior and multiple mitigation strategies further strengthens the contribution.
Critical weaknesses
-
The evaluation relies primarily on synthetically corrupted tool outputs, which may not fully capture the diversity of naturally occurring tool failures.
-
Although the mitigation experiments are informative, the computational and tool-calling overhead introduced by additional verification remains insufficiently explored.
-
There seems to be many API calls in the experiments, but the author did not provide any cost information.
Questions for authors
-
Could the authors clarify how well the proposed corruption settings represent naturally occurring tool errors and whether verification overhead can be quantified?
-
Providing detailed API cost in the experiments could help to verify the real attack strength. The authors should add a figure or a table in corresponding sections.
-
The related work discussion could more clearly position this study alongside existing evaluations of tool-using agents, such as AgentDojo, ToolSandbox, and ToolEmu, Sponge Tool Attack and Efficiency in LLM Multi-Agent Systems: A Survey. These connections would help clarify the distinct contributions and practical implications of the proposed approach.
I will adjust my rating according to the authors' rebuttal.
Thanks for your response and the additional experiments. The 64.6% rejection of correct answers suggests that rejecting a candidate does not imply detecting an error, so relying on tools in such cases is not necessarily unreasonable. The detectability of errors in the original tasks, especially P1 and Sub-Agent, also remains insufficiently established. I will maintain my rating.