- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes AnyTool-Agent for learning tool use in API ecosystems where there is a lack of API data. The authors propose API-guided trajectory synthesis and also propose Execution-Grounded Policy Optimization (EGPO), uses execution feedback and verified reference trajectories to reweight outcome-level advantages at the action level. Experiments cover AppWorld and BFCL with two Qwen3 backbones. With Qwen3-4B-Instruct, EGPO improves average success over GRPO from the same SFT initialization by 5.5 percentage points on AppWorld and 5.7 points on BFCL.
Critical strengths
- EGPO shows ~5-6 points gain on the two test sets over GRPO, while also outpeforming other GRPO variants like DAPO and GSPO, suggesting that execution feedback and action level re-weighting helps improve tool-calling performance in LLMs.
Critical weaknesses
-
The API-guided synthesis contribution is not sufficientely ablated and evaulated compared to existing baselines. The paper presents targeted synthesis for unfamiliar, data-scarce APIs as a central contribution, but there are no experiments comparing their method to existing API-guided generation methods. The authors only include a comparision againt blind exploration in Figure 3 a. This does not establish an advantage over existing specification-guided synthesis methods or show that the additional coverage improves downstream learning. It is important to compare their synthesis method to existing baselines, for instance: APIGen (NeurIPS 2024) [1] and APIGen-MT (NeurIPS 2025) [2] and show how API-guided synthesis fairs against them in a constant data setting for initial SFT and how the SFT trained models perform after EGPO.
-
For EGPO, the paper lacks controlled baseline comparisons against existing action-level advantage methods like SALT [3] and GiGPO [4]. These methods already assign step-level credit without training a separate process reward model, and SALT has also been evaluated on AppWorld. The main comparisons against GRPO, DAPO and GSPO do not establish whether EGPO improves over these existing action-level methods. A comparison should using the same SFT initialization, training tasks and evaluation setup would help establish the value of EGPO over these methods.
References
[4] Group-in-Group Policy Optimization for LLM Agent Training. NeurIPS 2025.
Questions for authors
-
How does your method differ operationally from APIGen-MT’s verified blueprint construction? Under matched settings, does this distinction improve accepted trajectory yield or task success?
-
What is the exact formula for aggregating multiple API calls into r_api^t?
-
How sensitive is the additive baseline to its coefficient? Does EGPO retain its advantage over a tuned additive control using identical feedback?
9 comments
Response to Reviewer PdN1 [2/2]
Q2
When an executable action contains multiple API calls, how are their call-level scores aggregated into one action-level API score?
Response
We score the calls within an executable action before computing one action-level API credit. Let denote the credit for call in action , the set of reference APIs first hit in that action, and the scarcity multiplier for API . For normal calls, the implemented aggregation is
m_t=\begin{cases} \max_{c\in H_t}\omega_c,&H_t\ne\varnothing,\\ 1,&H_t=\varnothing, \end{cases} \qquad r_{\mathrm{api}}^{(t)}=m_tq_t.$$ When no API is called, $q_t$ is initialized to zero. Taking the maximum call credit avoids increasing the action score simply by putting more calls in the same code block. When the action first hits one or more reference APIs, the largest corresponding scarcity multiplier is applied; the call giving the maximum credit need not be the call associated with that multiplier. Error and unsupported-constant rules are applied as specified by the execution scorer, while repetition and formatting feedback are handled in their respective score terms. The resulting action score is clipped, centered within the trajectory, and used to reweight the final outcome advantage. This **call aggregation followed by scarcity weighting** is the implemented rule. We have made the order explicit in Section 2.3 and Appendix F.4, Equation (6), including the case with no reference hit. ### **Q3** *How sensitive is fixed-scale additive action feedback to its coefficient, and does EGPO improve upon these controls when execution feedback is held constant?* ### **Response** We compared five fixed-addition coefficients using the same SFT initialization, execution-derived action scores, and AppWorld evaluation. For clarity, the table separates fixed-scale addition, a multiplicative-only control, and the complete EGPO configuration: | Training configuration | avg@4 (%) | best@4 (%) | | --- | ---: | ---: | | GRPO / $\lambda=0$ | 53.72 | 75.00 | | Fixed addition, $\lambda=0.06$ | 52.53 | 73.21 | | Fixed addition, $\lambda=0.2$ | 53.42 | 72.62 | | Fixed addition, $\lambda=0.4$ | 49.11 | 70.83 | | Fixed addition, $\lambda=0.6$ | 52.00 | 73.30 | | Multiplicative-only, $\alpha=0.6$, no scarcity | 54.17 | 75.00 | | **Full EGPO, $\alpha=0.6$** | **56.25** | **77.38** | The strongest tested nonzero fixed-addition coefficient reaches **53.42% avg@4**, compared with **54.17%** for the multiplicative-only version and **56.25%** for EGPO. A separate matched round-60 scarcity ablation measures **53.27% / 73.81% without scarcity** and **56.25% / 77.38% with scarcity**. For the same centered action score $d_t$, magnitude-scaled addition is precisely equivalent to the multiplicative form when $\lambda=\alpha$: $$A_o+\lambda|A_o|d_t=A_o\bigl(1+\lambda\operatorname{sgn}(A_o)d_t\bigr).$$ This equation explains the advantage of scaling action feedback with the final outcome: EGPO adjusts credit among actions in proportion to the trajectory advantage rather than adding an outcome-independent offset. **The coefficient sweep and the full EGPO results demonstrate stronger task completion than the tested fixed-addition alternatives.** Section 2.3, Section 3.4, and Appendix F.3 (Table 14) now present the mechanism and comparisons together.Thank you for adding the APIGen-MT comparison. The improved generation yield and downstream SFT performance are encouraging. However, I still do not understand which specific methodological difference explains these gains. APIGen-MT already samples APIs and environment context, constructs task blueprints, and verifies execution and semantic correctness.
Could you clarify:
- What exactly differs between the two synthesis procedures? A side-by-side example would help explain how your API selection and entity grounding differ from APIGen-MT.
- Why does APIGen-MT accept only 179 candidates versus your 399? Please break down where candidates fail and clarify whether both methods use the same models, retry budgets, and verification criteria.
- Does the subsequent EGPO comparison use identical RL tasks and references? Otherwise, it combines differences in SFT data and RL supervision. These details would help establish what makes the synthesis method better beyond the reported performance difference.
Response to Reviewer PdN1 [1/2]
Thank you for identifying the need for comparisons with established API-guided synthesis and action-level optimization methods. We added controlled APIGen-MT synthesis and learning experiments, together with GiGPO and SALT baselines, to clarify AnyTool's contribution beyond existing approaches.
W1 & Q1
How does API-guided synthesis compare with APIGen and APIGen-MT? Does its additional coverage lead to more verified demonstrations and stronger SFT and EGPO policies under comparable conditions?
Response
We adapted APIGen-MT to the same AppWorld sandboxes, API catalog, and verification procedure. We first compared generation efficiency with 1,500 candidates per method:
AnyTool yields 2.23× more verified demonstrations and 1.75× more API coverage at comparable recorded token usage. Per recorded token, these become 2.12× the verified-demonstration yield and 1.66× the API coverage. We also report the original APIGen adaptation in the revised paper.
We then separately expanded the APIGen-MT corpus and compared 399 verified demonstrations from each method using the same Qwen3-4B backbone, supervised assistant-token budget, training updates, and evaluation protocol:
AnyTool improves avg@4 by 10.26 points after SFT, and the advantage remains 4.32 points after EGPO. These results show that the synthesized examples improve actual task completion, not just API coverage.
Our synthesis method selects target API combinations before writing tasks, grounds each instruction in real sandbox entities, verifies task execution, and re-executes its core actions in a fresh sandbox. The full expert trajectory trains the initial policy through SFT; the verified core sequence also supplies action-level reference evidence during EGPO. This direct connection between data generation and subsequent policy optimization is a central contribution of AnyTool. Section 3.2 and Appendix F.2 (Tables 12–13) now give the complete comparison and protocol.
W2
EGPO should be compared with existing action-level credit methods such as GiGPO and SALT, using the same initial policy, training tasks, and evaluation setup.
Response
We added GiGPO and SALT to the AppWorld comparisons, alongside GRPO, DAPO, and GSPO. All action-level methods use the same Qwen3-4B SFT checkpoint, RL task pool, held-out evaluation, and executable-action interface, with a common mapping from action advantages to action tokens:
EGPO exceeds GiGPO by 2.83 / 6.55 points and SALT by 8.18 / 8.93 points in avg@4 / best@4, as measured by actual environment task success.
The difference lies in the action evidence used for learning. EGPO combines core API sequences verified during synthesis, deterministic execution feedback, and the frequency with which reference operations are reached by current rollouts. It uses these signals to redistribute the trajectory's final outcome advantage across individual actions, without training a separate process reward model. The controlled results show that this integration improves policy optimization beyond the tested action-level alternatives. Section 3.2 and Table 1 report the comparison, and Appendix F.1 documents the baseline adaptations.