- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper studies how tool-using agents handle changes in tool interfaces, outputs, and availability. EvoToolAgent links failures to system components and uses this diagnosis for local recovery and offline evolution. The paper also introduces EvoToolBench-D, a benchmark with controlled tool changes. Experiments test online adaptation across three backbones and separately evaluate a fixed configuration chosen through offline evolution.
Critical strengths
- The experiments use matched execution budgets and three random seeds, with gains across three backbones.
Critical weaknesses
- Table 3 removes the whole Recovery Loop, including repair and partial replanning. It shows that recovery helps, but not whether component-level attribution helps more than a simpler repair policy with the same actions and budget. This leaves the main contribution insufficiently tested.
- The benefit of co-evolution is unclear. Population evolution is disabled in the main comparisons, including for baselines built around evolution. Table 4 compares only the evolved configuration’s development and frozen-test results. A small gap does not show that evolution helped. The paper needs a comparison with the initial configuration and another search method under matched budgets and test rules.
Questions for authors
- Do you have results that keep the repair actions, evidence preservation, and execution budget fixed, but replace component-level attribution with a generic repair policy?
- Under Protocol B, how does the evolved system compare with the initial configuration and untargeted evolution? Can you compare them using the same search budget and frozen-test rules?