acceptodds
Under review as a conference paper at ICLR 2027

Scaling Tool-Use Agents via Environment-Grounded Query Synthesis

Abstract

Training tool-use agents with agentic reinforcement learning (Agentic RL) is bottlenecked by two challenges: the lack of scalable, robust execution environments and the scarcity of realistic training data that carries learning signal and captures implicit human reasoning. Existing pipelines build the two separately from pre-collected tool corpora and overlook the difficulty and realism of synthesized tasks, limiting their effectiveness for RL. We introduce , a fully automated framework that addresses both challenges. autonomously explores and verifies stateful, executable tool environments from authentic resources, and synthesizes natural multi-turn trajectories through topology-aware sampling and implicitness injection, producing grounded queries with realistic intents. Using 85 environments across 7 domains, generates 2,575 SFT and RL trajectories and beats prior work with significantly fewer environments and data. Across Qwen3 models from 1.7B to 8B, improves over the base models by up to **15.0 points** on BFCL-v3 multi-turn, **8.6** on MCP-Atlas, **4.9** on -Bench, and **8.3** on VitaBench. By automating both environment construction and trajectory synthesis, offers a scalable, extensible foundation for Agentic RL.

Primary area:
reinforcement learning
Keywords:
Agentic RL, Environment Scaling, Tool Agents

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jpo8OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents EnvFactory, an automated pipeline for constructing executable, stateful tool-use environments and synthesizing training trajectories. It combines documentation-grounded environment construction, a parameter-level dependency graph, topology-aware sampling, and implicitness injection to generate executable tasks with more natural user requests. The authors construct 85 environments and 2,575 SFT and RL episodes, and evaluate post-trained Qwen3 models at three scales on four tool-use benchmarks. The results demonstrate the practical utility of the overall pipeline, while the advantages of individual components are less conclusively established.

Critical strengths
  1. A coherent system addressing practical training bottlenecks. The pipeline connects environment construction, prerequisite resolution, query synthesis, and execution-based verification. In particular, backward dependency resolution explicitly addresses missing tool prerequisites, while implicitness injection targets overly explicit synthetic requests. These components form a useful and practically motivated approach to improving synthetic tool-use training data.

  2. Broad empirical evaluation supporting the overall pipeline’s utility. The evaluation covers three model scales and four benchmark families, including conversational tasks and cross-server workflows. Some improvements over the base models are substantial: for example, the 4B model improves from 33.50 to 48.50 on BFCL multi-turn and from 7.67 to 16.00 on VitaBench. Although improvements over trained baselines are less uniform, the overall results suggest that the synthesized environments and trajectories are useful resources for tool-use post-training.

Critical weaknesses
  1. The incremental benefits of the proposed components require stronger uncertainty analysis. Most gains in Table 2 are modest, and the average improvements from implicitness injection in Table 3 are only 0.37–1.00 percentage points. With stochastic decoding and no reported repeated-run variability or confidence intervals, it is difficult to assess their robustness. The topology-aware sampling and implicitness-injection ablations in Tables 2–3 are conducted with SFT, leaving their effects on the final RL pipeline less directly established. The data-efficiency interpretation also requires caution because episode counts across different training pipelines do not directly establish comparable training exposure or computational cost.

  2. The relationship between the composite reward and actual task completion is insufficiently validated. As specified in Section 3.5 and Appendix C.3, the RL reward matches reference tool calls and terminal states and penalizes redundant or malformed calls, but includes no explicit correctness check of the final natural-language response. Thus, for a read-only task, two rollouts with identical tool calls, terminal states, and penalties can receive the same reward even if one provides an incorrect final answer. Although Appendix C.3 acknowledges limitations of exact-state matching, a small independent audit of student rollouts would help establish how well the composite reward reflects actual task success, particularly for read-only tasks and alternative valid execution paths.

Questions for authors
  1. Uncertainty analysis: Could the authors report repeated-run variability or task-level paired uncertainty estimates for the Table 2 and Table 3 ablations at one representative scale, such as 4B? This would help determine whether the modest component gains are consistent across runs.

  2. Reward validation: Is any additional assessment of the final response used during RL beyond the reward described in Appendix C.3? If not, could the authors provide a small audit comparing reward values with independently assessed task completion, distinguishing read-only and state-changing tasks?

  3. Training configuration and accounting: Table 8 specifies initialization from SFT epoch 1, whereas Section 4.2 compares against a three-epoch SFT model. Which checkpoint actually initializes RL, and what is its pre-RL performance? Please also report the number of supervised records after dialogue expansion to clarify the data-scale comparison.

2 comments

Thank you for your insightful feedback. We address your concerns as follow step by step and we have uploaded the revised PDF, which you can directly open the OpenReview of this submission. The relevant changes to your reviews are marked in blue in both main body and appendix part.

W2 & Q2: Reward alignment with task fulfillment

We do not use the agent’s self-reported completion as an additional reward signal, since unsupported claims can overestimate task success.

We distinguish task fulfillment—whether the user’s request is satisfied—from a completion claim—whether the agent’s final response says the task is complete. We audited 100 randomly sampled training rollouts using correlation to assess reward–fulfillment alignment and mismatch to assess whether completion claims are supported by execution evidence.

  1. Correlation analysis tests whether the reward aligns with independently judged task fulfillment. A DeepSeek-V4.1-Flash judge assessed how well each rollout satisfied the user’s request using the final response, tool executions, and environment states, without access to the reward—not merely whether the agent claimed completion.
AssessmentSpearman ρ with reward95% CI
Independently judged task fulfillment0.7250.573–0.839

Higher-reward rollouts generally received higher task-fulfillment scores, with the correlation’s confidence interval entirely above zero. This supports reward–fulfillment alignment in the audited sample.

  1. Mismatch analysis tests whether the agent’s self-reported completion is supported by execution evidence. We compared final-response claims of full completion with reward-defined success, treating partial completion as non-success.
MeasureDefinitionResult
Claim–reward mismatchFull-completion claims on non-successes / all full-completion claims60/89 (67.4%)
False-positive rateFull-completion claims / non-successes60/71 (84.5%)
False-negative rateNon-completion claims / successes0/29 (0%)

The high false-positive rate shows that agents frequently claimed full completion despite reward-defined non-success. Adding these claims as positive reward signals could inflate reward for rollouts that currently receive partial or zero credit. This supports relying on execution evidence rather than self-reported completion.

Q3: Training configuration and data accounting

Thank you for pointing out, the table 8 had a typo, which we corrected: SFT+RL starts from the same three-epoch SFT checkpoint evaluated in Table 1, not epoch 1.

TrainingInitializationBudget
SFTBase model3 epochs
RLBase model100 RL steps
SFT+RLSFT epoch 3100 RL steps

The pre-RL scores are the SFT rows in Table 1 (e.g., at 4B: 44.25 on BFCL multi-turn and 27.29 Overall). The reported 1,622 SFT and 953 RL quantities count episodes, not expanded training records.

Thank you to the authors for their efforts in addressing my comments. My main concern has been clarified to some extent. The 4B SFT+RL ablations show gains of 1–2 points on BFCL multi-turn, but their robustness remains unclear. I have decided to maintain my original score.

Reviewer_YTvxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper builds a data pipeline for training tool-use agents. The idea is to have LLM agents write their own practice environments: they look up API docs online, implement each API as a stateful tool server, and test it until it works. On top of these environments, the pipeline generates multi-turn tasks that are actually solvable (by making sure every required input can come from the user or an earlier tool call) and that read like real user requests rather than step-by-step instructions. With 85 environments and about 2.6K trajectories, the authors fine-tune Qwen3 models (1.7B to 8B) with SFT and then RL, and report gains on BFCL-v3, tau2-Bench, MCP-Atlas and VitaBench. They also argue the approach is more data-efficient than two concurrent works, AWM and EnvScaler.

Critical strengths
  1. A simple, useful fix for unsolvable synthetic tasks. Instead of chaining tools forward at random, the sampler works backward from the tool it wants to use and adds whatever tools are needed to produce its inputs, so every task is solvable by construction. The idea is easy to reuse, and the paper motivates it well with the observation that naively generated tasks tend to be all-success or all-failure, which gives GRPO nothing to learn from.

  2. Solid engineering. The environments are stateful, deterministic and resettable, the reward checks both the calls and the final state, baselines are re-run under the same evaluation setup, and the authors plan to release everything.

Critical weaknesses

1. The training environments look built around the evaluation benchmarks.

  • Several environments in Figure 2 are near-copies of the test environments: TradingBot and TravelBooking match BFCL's multi-turn backends in both name and number of tools, and Airline / Retail / Telecom are exactly tau2-Bench's three domains.
  • These APIs only exist inside the benchmarks, yet the paper says environments are built from online API documentation.
  • The biggest gain is exactly on BFCL multi-turn, and there is no overlap analysis.
  • By contrast, EnvScaler derives its environments from general tool datasets (API-Bank, ToolACE) and explicitly checks that its gains don't depend on similarity to BFCL.
  • So the results can't tell us whether the pipeline teaches general tool use or just familiarity with the test APIs.

2. The improvement over prior work is small and not clearly attributable to the method.

  • Most of the gain over the base model comes from SFT on trajectories from a stronger same-family model, and the RL reward also rewards imitating a stronger model's reference trajectories.
  • EnvScaler uses the same SFT teacher and base models, and the margin over it is about 1 point Overall, from single runs.
  • Given the difference in how close the training environments are to the test sets, a gap this small is easily explained by distribution match rather than by the pipeline itself.
Questions for authors

1. Does it generalize beyond the benchmark APIs?

  • Several training environments match BFCL's multi-turn backends (e.g., TradingBot and TravelBooking, same names and tool counts) and tau2-Bench's domains. How were these found from online API documentation?
  • If you remove the benchmark-matching environments and train only on environments built from real-world API docs, do the gains on BFCL multi-turn and tau2 hold?
  • A similarity split like EnvScaler's (train on the environments most vs. least similar to the benchmark) would also help.
  • In Figure 4a the number of trajectories grows with the number of environments. What happens if the trajectory budget is fixed and only the number of environments changes?

2. How much of the gain is distillation?

  • Please report the SFT teacher (Qwen3-30B-A3B-Thinking-2507) and the RL reference model (DeepSeek-V3.2) under your own evaluation setup, so readers can see the gap between students and teachers.
  • The RL reward is largely about matching DeepSeek-V3.2's reference calls and final state, which is still imitation. Could you ablate the RL reference model, e.g., use the SFT teacher instead?
  • How much of the ~1-point margin over EnvScaler (same SFT teacher) do you attribute to the pipeline itself?

3. Where does the model improve, and where does it still fail?

  • Please add a case study with a few concrete success and failure examples.
  • A breakdown of error types before and after training (wrong tool, missing dependency, unnecessary questions to the user, wrong state change) would show which errors account for the gain, especially over EnvScaler.
5 comments

Thank you for your feedback. We have addressed all concerns below:

W1: Benchmark overlap and transfer

Ten of our 85 environments share tool specifications with BFCL multi-turn backends or τ²-Bench domains because the Search Agent retrieves public API and open-source tool specifications from the web:

Our environmentBenchmark APIShared tool names / benchmark toolsIdentical input schemas
FileSystemBFCL gorilla_file_system15 / 1810
TradingBotBFCL trading_bot14 / 227
VehicleControlBFCL vehicle_control10 / 224
MessageBFCL message_api7 / 102
PostingBFCL posting_api7 / 142
TicketManagementBFCL ticket_api6 / 94
TravelBookingBFCL travel_booking6 / 173
Telecomτ² telecom9 / 137
Retailτ² retail10 / 154
Airlineτ² airline7 / 144

This overlap is only at the interface level: EnvGen implements every server, initial state, and task from scratch. To test whether our gains depend on these ten environments, we removed all of them and retrained on the remaining 75 environments (Subset):

TypeSFT SamplesRL Samples
Full33,2543,207
Subset25,4562,404
BenchmarkQwen3-4BEnvFactory-4B-RLEnvFactory-4B-RL (Subset)
BFCL Single-turn83.4982.2780.11
BFCL Multi-turn24.8838.5036.88
τ²-Bench Avg.25.2522.9128.88
VitaBench Avg.7.6721.0021.00
MCP-Atlas Pass Rate4.1213.0611.00
MCP-Atlas Mean Coverage12.8623.2222.14

Removing all ten overlapping environments preserves the gains across all four benchmarks.

W2 & Q2: Gains, distillation, and attribution

We evaluated the SFT teacher and RL reference model across all four benchmarks (added to Table 1):

ModelBFCL Single / MultiMCP-Atlas Passτ²-Bench Avg.VitaBench Avg.
Qwen3-4B85.15 / 33.504.1225.257.67
EnvFactory-4B (SFT+RL)85.46 / 48.509.9730.1316.00
EnvFactory-8B (SFT+RL)86.02 / 49.0013.7533.6718.67
Qwen3-30B-A3B-Thinking-2507 (SFT teacher)84.19 / 47.0013.7553.0134.33
DeepSeek-V3.2 (RL reference)58.21 / 33.2537.1176.7357.33
  • Beyond distillation: Direct RL without SFT improves BFCL multi-turn and VitaBench at every scale (Table 4), RL after SFT further improves Overall scores at all scales (Table 1), and the 4B/8B models surpass both the SFT teacher and RL reference on BFCL multi-turn.
  • Outcome-based RL: The policy learns from its own rollouts against verified terminal states and required calls; state-only reward (α = 0, without reference calls) already recovers most of the BFCL multi-turn gain (39.38 vs. 40.87; Figure 5).
  • Pipeline contribution: Using the same base models and SFT teacher as EnvScaler, EnvFactory reaches higher Overall scores with fewer environments (85 vs. 191) and trajectories (2,575 vs. 11,572; Table 1). Fixed-budget ablations isolate the gains from topology-aware sampling (Table 2) and implicitness injection (Table 3).

Q1: Generalization beyond benchmark APIs

Beyond holding up after removing the ten overlapping environments (W1), our 4B model shows its largest relative gains on VitaBench and MCP-Atlas rather than BFCL or τ²-Bench (Table 1), and preserves out-of-domain math reasoning (average of AIME24, AIME25, and MATH-500; Table 5):

Model1.7B4B8B
Qwen3 (Base)59.2777.9580.54
EnvFactory (SFT)59.0878.8578.25
EnvFactory (SFT+RL)60.3779.3779.04

To separate environment diversity from trajectory volume (Figure 4a), we also fix the budget at 5,000 SFT samples and vary the number of environments (4B):

Number of Environments20406080
BFCL Multi37.5038.2540.8842.25
MCP Atlas Pass5.845.507.5610.31

Both BFCL multi-turn and MCP-Atlas improve as more environments are included under a fixed sample budget.

Thanks again for your insightful feedback and assistant. We have uploaded the revised PDF, which you can directly open the OpenReview of this submission. The relevant changes to your reviews are marked in blue in both main body and appendix part.

In addition, we include case study to address your last concern

Q3: Successes and failure modes

Below are paired before/after examples from a 4B GRPO run on AutomationBench (initial vs. final checkpoint) illustrating how training resolves the four error types:

Error TypeBefore TrainingAfter Training
Wrong tool (Twitter engagement)Publicly replies to all mentions via twitter_post_tweet, ignoring mention type.Retrieves the engagement SOP, likes positive mentions (twitter_like_tweet), replies only to the product question, and escalates the complaint on Slack (slack_send_channel_message).
Missing dependency (ROI report)Reads the spreadsheet and emails the report immediately without checking reporting guidelines, omitting the reference code and ROI breakdown.Retrieves sroi_policy first, then computes the channel ROI breakdown and includes the required reference code (SROI-772-Q1) in the email.
Unnecessary questions (Journal summary)Makes no tool calls and asks the user for the spreadsheet location, worksheet title, and posting procedures.Searches Gmail, Drive, and Sheets to locate the posting procedures and recurring schedule, then emails the summary with the source amounts.
Wrong state change (Badge request)Updates a medium-priority item (itm_413) instead of the eligible request.Verifies priority and status, updates the eligible critical contractor request (itm_411) to In Progress, and sends the required alerts.

Training resolves these errors by teaching the agent to retrieve prerequisite policies before acting, select the appropriate tool for each operation, gather discoverable context via tools rather than asking the user, and verify target records before modifying state.

Thank you for the detailed response and the large body of new experiments. I found the fixed-budget environment scaling and the teacher/reference scores useful, and they resolve some of my questions. That said, my main concern remains: whether the gains come from general tool use or from familiarity with the evaluation interfaces.

  1. I appreciate your transparency about the ten overlapping environments. The explanation that the Search Agent retrieved the benchmarks' own tool specifications suggests that the overlap may originate in the retrieval step itself. I would find it helpful to know whether the pipeline could include a safeguard against this, beyond simply removing the environments after the fact.
  2. The de-overlap experiment is a useful addition, though it relies on direct RL alone, while the main results use SFT+RL. Its base score on BFCL multi-turn (24.88) also seems to differ from the value in Table 1 (33.50); relative to Table 1, less than half of the direct-RL gain remains (+3.4 vs. +7.9). A de-overlapped SFT+RL result under the Table 1 setup would make this considerably more convincing.
  3. I still have trouble separating the method's contribution from distillation. The student reaches the SFT teacher's level mainly on BFCL multi-turn, where training most closely matches the test interfaces, and sits roughly 20 points below the teacher on tau2-Bench and VitaBench. In my view, this pattern is also consistent with in-distribution distillation.

For now I will keep my score, but I would be glad to revisit it if these points are clarified.

Reviewer_CtDAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces EnvFactory, which creates environments and tasks for training tool-use agents. It retrieves online API documentation, builds tools that can read and change shared state, and tests and fixes their implementations. It uses a tool graph to identify dependencies and add missing prerequisite calls when creating tasks. It then rewrites user requests to make them more natural and leave some steps for the agent to infer. The authors train Qwen3 models using SFT on 1,622 multi-turn conversations, followed by GRPO on 953 single-request episodes.

Critical strengths
  1. The paper measures task quality in a clear way. It runs each task 8 times and counts how many tasks always succeed, sometimes succeed, or always fail (Fig. 1a). This links task design directly to the training signal in GRPO.

  2. The environments are based on real API documentation. The tool graph keeps “output feeds input” links separate from looser “often comes next” links. Only the first kind is used to fill in missing inputs, which is a clean design.

  3. Evaluation across three model sizes and four benchmarks provides good coverage.

Critical weaknesses

W1: My main concern is that the comparison mixes the effect of the data-generation method with differences in training procedures and budgets. EnvFactory uses teacher-generated SFT data followed by RL, while AWM uses direct RL. EnvScaler uses the same SFT teacher as EnvFactory, but its training procedure and budget are different. These comparisons show how the complete systems perform, but they do not clearly establish how much of the improvement comes from EnvFactory’s data-generation method. The paper’s main idea shows only a small effect. In Table 2, replacing topology-aware sampling with random walks lowers scores by only 0.3-2.8 points. At 1.7B, random walks are even better on τ2τ²-Bench. The overall margins over EnvScaler are also small: 1.2 points at 4B and 0.7 points at 8B.

W2: The ablations do not fully test the main motivation of the method. The ablations of the two data ideas use SFT only: topology-aware sampling (Table 2) and implicitness injection (Table 3). But the paper motivates both ideas with GRPO training. The environment-building parts are not ablated at all. These include building from API documentation (Fig. 1b), the “lifecycle” rule that every state you can enter must have a way out (Section 3.2), and the test-and-fix loop (Section 3.2).

W3: I am not sure the generated environments and reference trajectories are reliable. The pipeline runs real code and checks the database state, which is good. But one model, Kimi-K2, mainly proposes, writes and tests the environments. The test-writing prompt also starts by reading the generated code. So the code and the tests could share the same mistake. Passing these tests shows that the code agrees with itself, not that it behaves like the real API. When a test fails, the revision step asks the model to decide whether the code or the test is wrong. Also, neither embedding similarity nor LLM reasoning guarantees a correct tool graph, and the paper does not measure the graph’s accuracy.

Tasks have a similar problem. A separate solver agent tries each task 8 times. An LLM judge then checks the tool calls, the final database state, and whether the user’s request was met. A task is kept if at least one run succeeds, and dropped if all 8 fail. But consider two tasks:

  • Task A is impossible, for example because a needed tool is missing.
  • Task B is valid but too hard for the solver agent.

Both can fail all 8 times, and this check cannot tell them apart. So the filter may throw away good, hard tasks together with broken ones.

W4: Comparisons with closely related methods are missing. AgentScaler [1], VULCAN [2], Simia [3], APIGen-MT [4] and Toucan [5]. The paper cites [1], [3] and [5], but discusses them only in the text, not as baselines.

[1] Fang, Runnan, et al. "Towards general agentic intelligence via environment scaling." Findings of the Association for Computational Linguistics: ACL 2026. 2026.

[2] Saeidi, Amir, et al. "Vulcan: Where agents learn by living in simulated tool environments." ICLR 2026 Workshop on Data-FM. Last modified: April. Vol. 1. 2026.

[3] Li, Yuetai, et al. "Simulating environments with reasoning models for agent training." arXiv preprint arXiv:2511.01824 (2025).

[4] Prabhakar, Akshara, et al. "Apigen-mt: Agentic pipeline for multi-turn data generation via simulated agent-human interplay." Advances in Neural Information Processing Systems 38 (2026).

[5] Xu, Zhangchen, et al. "Toucan: Synthesizing 1.5 m tool-agentic data from real-world mcp environments." arXiv preprint arXiv:2510.01179 (2025).

Questions for authors

See weaknesses above

4 comments

Thank you for your feedback. We have addressed all concerns below:

W1: Training procedure and gains

Table 4 evaluates direct RL without SFT initialization, matching AWM’s setup:

Model sizeMethodBFCL Single-turnBFCL Multi-turnτ²-Bench Avg.VitaBench Avg.
4BAWM85.9740.7522.3711.67
4BEnvFactory (RL)85.2641.3824.8312.74
8BAWM84.8042.2528.4216.48
8BEnvFactory (RL)84.4244.3529.0817.00

EnvFactory (RL) outperforms AWM on BFCL multi-turn, τ²-Bench, and VitaBench at both scales without SFT initialization. With SFT+RL (Table 1), EnvFactory uses fewer environments and trajectories (85 / 2,575) than AWM (526 / 3,315) and EnvScaler (191 / 11,572), while achieving higher Overall scores across all three scales.

W2: Component ablations

To test both data-synthesis mechanisms under GRPO, we extend the 4B SFT ablations in Tables 2 and 3 with RL training on BFCL-v3 multi-turn.

For topology-aware sampling, we synthesize 700 RL tasks per setting and run GRPO from the ablated 4B SFT checkpoints:

ModelBaseMiss FuncMiss ParamLong ContextOverall
Random Walk-4B (SFT+RL)51.545.532.034.040.75
Topology-Aware-4B (SFT+RL)52.047.033.535.041.88

For implicitness injection, we synthesize 300 RL tasks per setting and run GRPO from the ablated 4B SFT checkpoints:

ModelBaseMiss FuncMiss ParamLong ContextOverall
Original-4B (SFT+RL)52.048.539.036.043.88
Injected-4B (SFT+RL)55.047.541.538.045.50

W3: Environment and task reliability

  1. Environment Reliability: Our test agent executes unit tests and checks tool outputs, errors, and database state transitions against the retrieved API specifications and example payloads, rather than relying on text-only judgments.

  2. Edge Accuracy: An independent DeepSeek-V4.1-Flash agent audited 100 randomly sampled edges against tool specifications, checking input–output compatibility for embedding-derived parameter-flow edges and workflow plausibility for LLM-derived edges:

    Edge TypeFalse Positive Ratio
    Embedding (parameter-flow)3.8% (3/78)
    LLM (workflow)0.0% (0/22)
  3. Task Solvability: An independent DeepSeek-V4.1-Flash agent audited 50 randomly sampled all-failure tasks:

    TypeRatio
    Solver Difficulty16% (8/50)
    Solvability Issues84% (42/50)

    Most rejected tasks (84%) have solvability defects. While 16% are valid but hard tasks, our pipeline requires a verified execution trajectory to define reference calls and terminal states for SFT and RL.

W4: Related methods and baselines

We cited all five works in Section 2 and compared their mechanisms in Figure 1(c):

MethodAutonomous DiscoveryTool GraphDependency ResolutionImplicitness Injection
EnvScaler✗✗✗✗
VULCAN✗✓✗✗
Simia✗✗✗✗
APIGen-MT✗✓✗✗
TOUCAN✗✗✗✓
EnvFactory✓✓✓✓

We also evaluated the released 7B–8B checkpoints among these methods—Simia (Qwen3-8B, SFT+RL), TOUCAN (Qwen2.5-7B), and APIGen-MT’s xLAM-2-fc-r (Llama-3.1-8B)—in Table 1:

ModelBFCL SingleBFCL MultiMCP-Atlas PassMCP-Atlas Cov.τ²-Bench Avg.VitaBench Avg.Overall
TOUCAN (Qwen2.5-7B)76.4016.889.6221.6016.499.3320.52
xLAM-2-fc-r (Llama-8B)73.5265.250.003.8533.770.6725.96
Simia (Qwen3-8B, SFT+RL)51.493.632.756.4932.897.0017.55
EnvFactory-4B (SFT+RL)85.4648.509.9721.8930.1316.0030.77
EnvFactory-8B (SFT+RL)86.0249.0013.7525.9833.6718.6733.40

Dear Reviewer,

We have uploaded the revised PDF, which you can directly open the OpenReview of this submission. The relevant changes to your reviews are marked in blue in both main body and appendix part.

Thanks for your insightful feedback and assistant.

Thank you for the rebuttal. Some of my concerns remain:

  • W1: The direct RL comparison only covers AWM, not other baselines under similar training settings. The performance differences are small, and no variance is reported. Could the authors provide results across multiple random seeds?
  • W3: The task audit only examines rejected tasks, finding that 16% were valid but too difficult for the solver. However, it does not establish whether the retained tasks have correct reference trajectories.

I will maintain my original score.

Thank you for the follow-up. We report additional evaluation runs for the direct RL comparison and a new audit of reference trajectories in the retained training set.

W1: Direct-RL baseline and evaluation variability.

We chose AWM for the direct-RL comparison in Table 4 because both methods can be compared at 4B and 8B without a task-specific SFT warm start. Table 1 includes broader comparisons with released checkpoints from EnvScaler, Simia, TOUCAN, and APIGen-MT, but those checkpoints do not provide additional baselines under this direct-RL setting.

To quantify evaluation variability, we evaluated the fixed AWM and EnvFactory direct-RL checkpoints at both model sizes with two additional evaluation seeds, without retraining. The table reports the mean ± sample standard deviation across the original run and the two additional runs.

ModelsBFCL-Multiτ²-Bench Avg.VitaBench Avg.MCP-Atlas Pass
AWM-4B39.88 ± 0.6124.81 ± 2.3612.22 ± 0.964.47 ± 0.34
EnvFactory-RL-4B41.88 ± 0.8626.46 ± 1.4513.47 ± 0.815.96 ± 0.20
AWM-8B43.12 ± 0.6930.81 ± 2.1816.72 ± 1.186.19 ± 0.69
EnvFactory-RL-8B44.88 ± 1.1033.03 ± 3.8519.44 ± 2.507.67 ± 0.40

EnvFactory direct-RL has higher mean scores than AWM on all three benchmarks at both model sizes, with gains of 1.25–2.72 percentage points.

W3: Reference Trajectories Judge

To directly assess the quality of the retained training set, we conducted both an LLM-as-a-judge audit and an independent human audit covering task clarity (whether the request is unambiguous and feasible) and reference trajectory correctness.

  • Human Judge: We recruited six computer science Ph.D. students with experience in LLM agents. Each annotator independently audited 20 randomly sampled tasks (120 tasks in total).
  • LLM Judge: We prompted GPT6.1-Sol to evaluate 500 randomly sampled retained tasks and their reference trajectories on both clarity and trajectory correctness, with no overlap with the human sample.
AdultReference trajectory correctnessTask Clarity
Human82.5% (99/120)96.7% (116/120)
LLM83.6% (418/500)98.4% (492/500)

The two audits yielded similar aggregate rates on separate samples. Most sampled trajectories were judged correct, but roughly one in six was not. We have added both audit results to the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.