- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents EnvFactory, an automated pipeline for constructing executable, stateful tool-use environments and synthesizing training trajectories. It combines documentation-grounded environment construction, a parameter-level dependency graph, topology-aware sampling, and implicitness injection to generate executable tasks with more natural user requests. The authors construct 85 environments and 2,575 SFT and RL episodes, and evaluate post-trained Qwen3 models at three scales on four tool-use benchmarks. The results demonstrate the practical utility of the overall pipeline, while the advantages of individual components are less conclusively established.
Critical strengths
-
A coherent system addressing practical training bottlenecks. The pipeline connects environment construction, prerequisite resolution, query synthesis, and execution-based verification. In particular, backward dependency resolution explicitly addresses missing tool prerequisites, while implicitness injection targets overly explicit synthetic requests. These components form a useful and practically motivated approach to improving synthetic tool-use training data.
-
Broad empirical evaluation supporting the overall pipeline’s utility. The evaluation covers three model scales and four benchmark families, including conversational tasks and cross-server workflows. Some improvements over the base models are substantial: for example, the 4B model improves from 33.50 to 48.50 on BFCL multi-turn and from 7.67 to 16.00 on VitaBench. Although improvements over trained baselines are less uniform, the overall results suggest that the synthesized environments and trajectories are useful resources for tool-use post-training.
Critical weaknesses
-
The incremental benefits of the proposed components require stronger uncertainty analysis. Most gains in Table 2 are modest, and the average improvements from implicitness injection in Table 3 are only 0.37–1.00 percentage points. With stochastic decoding and no reported repeated-run variability or confidence intervals, it is difficult to assess their robustness. The topology-aware sampling and implicitness-injection ablations in Tables 2–3 are conducted with SFT, leaving their effects on the final RL pipeline less directly established. The data-efficiency interpretation also requires caution because episode counts across different training pipelines do not directly establish comparable training exposure or computational cost.
-
The relationship between the composite reward and actual task completion is insufficiently validated. As specified in Section 3.5 and Appendix C.3, the RL reward matches reference tool calls and terminal states and penalizes redundant or malformed calls, but includes no explicit correctness check of the final natural-language response. Thus, for a read-only task, two rollouts with identical tool calls, terminal states, and penalties can receive the same reward even if one provides an incorrect final answer. Although Appendix C.3 acknowledges limitations of exact-state matching, a small independent audit of student rollouts would help establish how well the composite reward reflects actual task success, particularly for read-only tasks and alternative valid execution paths.
Questions for authors
-
Uncertainty analysis: Could the authors report repeated-run variability or task-level paired uncertainty estimates for the Table 2 and Table 3 ablations at one representative scale, such as 4B? This would help determine whether the modest component gains are consistent across runs.
-
Reward validation: Is any additional assessment of the final response used during RL beyond the reward described in Appendix C.3? If not, could the authors provide a small audit comparing reward values with independently assessed task completion, distinguishing read-only and state-changing tasks?
-
Training configuration and accounting: Table 8 specifies initialization from SFT epoch 1, whereas Section 4.2 compares against a three-epoch SFT model. Which checkpoint actually initializes RL, and what is its pre-RL performance? Please also report the number of supervised records after dialogue expansion to clarify the data-scale comparison.
2 comments
Thank you to the authors for their efforts in addressing my comments. My main concern has been clarified to some extent. The 4B SFT+RL ablations show gains of 1–2 points on BFCL multi-turn, but their robustness remains unclear. I have decided to maintain my original score.
Thank you for your insightful feedback. We address your concerns as follow step by step and we have uploaded the revised PDF, which you can directly open the OpenReview of this submission. The relevant changes to your reviews are marked in blue in both main body and appendix part.
We do not use the agent’s self-reported completion as an additional reward signal, since unsupported claims can overestimate task success.
We distinguish task fulfillment—whether the user’s request is satisfied—from a completion claim—whether the agent’s final response says the task is complete. We audited 100 randomly sampled training rollouts using correlation to assess reward–fulfillment alignment and mismatch to assess whether completion claims are supported by execution evidence.
Higher-reward rollouts generally received higher task-fulfillment scores, with the correlation’s confidence interval entirely above zero. This supports reward–fulfillment alignment in the audited sample.
The high false-positive rate shows that agents frequently claimed full completion despite reward-defined non-success. Adding these claims as positive reward signals could inflate reward for rollouts that currently receive partial or zero credit. This supports relying on execution evidence rather than self-reported completion.
Thank you for pointing out, the table 8 had a typo, which we corrected: SFT+RL starts from the same three-epoch SFT checkpoint evaluated in Table 1, not epoch 1.
The pre-RL scores are the SFT rows in Table 1 (e.g., at 4B: 44.25 on BFCL multi-turn and 27.29 Overall). The reported 1,622 SFT and 953 RL quantities count episodes, not expanded training records.