acceptodds
Withdrawn from ICLR 2027

Echoverse-MCP: Stateful Tool Environments for Agent Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reinforcement learning for tool-using agents requires environments that maintain state, enforce application rules, and expose the consequences of actions. Building these environments by hand limits the range of workflows available for training. We present Echoverse-MCP, an LLM-assisted pipeline that combines public OpenAPI specifications with reusable domain models to construct isolated, stateful Model Context Protocol environments. Each environment pairs typed tools with a relational database, seeded records, and explicit state transitions. The pipeline generates multi-step tasks by executing reference programs and retaining their tool traces and database changes as evidence. The resulting resource contains 121 environments and 5,916 tools, with 19,041 tasks across 45 environments. Reference solutions average 7.15 calls and 6.24 distinct tools. A separate 600-task evaluation suite covers scripted-user and simulated-user interaction. Mixed GRPO post-training of Qwen3.5-4B raises reported BFCL accuracy by 5.6 percentage points and mean three-domain pass@1 by 3.2 points. The results support the use of generated executable environments for agent learning while showing that aggregate gains can coexist with regressions in individual capabilities.

Primary area:
reinforcement learning
Keywords:
RL, Tool Use, Tool Calling, Agents, Environments, Self-Evolve, Post-training, Rewards, Verification, Synthetic, Data, Task Generation

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_aUDbOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper builds a pipeline that turns public OpenAPI specifications into isolated, stateful MCP environments, each backed by a relational database that enforces application rules. On top of these environments, it generates multi-step tasks from executed reference programs, and each task can be posed with either a scripted or a simulated user. The paper also introduces an evaluation suite and uses the resource for mixed RL training of a small model, reporting gains on BFCL, a three-domain τ benchmark and the suite.

Critical strengths

Grounding tasks in executed reference programs is the right design for RL on stateful tools. Because each request is written from an actual execution and its database changes, every task has a checkable target state, and the reward can rely mainly on state checks rather than only on model judgment.

Critical weaknesses
  1. The contribution is not separated from the closest prior work. Public work already synthesizes stateful, database-backed tool environments from API or tool specifications and trains agents on them with RL: AgentScaler (arXiv 2509.13311), EnvScaler (arXiv 2601.05808) and AWM (arXiv 2602.10090). AWM in particular also exposes database-backed environments as MCP tools, trains small Qwen models with GRPO, and evaluates on BFCL and τ²-bench. The paper neither discusses nor compares against any of these. Its own distinctive choices, starting from real OpenAPI specifications and posing each task in two user formats, are never shown to matter, so the contribution cannot be distinguished from existing pipelines.
  2. The experiments do not show that these environments are good training material. The before/after comparison favours the trained model: the base score is the worst of several runs while the trained model is run once, and on the paper's own suite the same model acts as user simulator, training reward judge and evaluation judge. More importantly, the paper compares only against no training. Without a comparison to training on other environments (e.g., AWM's), even the reported gains do not show that these environments are better than what already exists.
Questions for authors
  1. How does the pipeline differ from AgentScaler, EnvScaler and AWM, both in how environments are built and in the environments it produces? At a matched task budget, does training on Echoverse-MCP outperform training on their environments?
  2. Are the 20 environments in the evaluation suite disjoint from the 45 environments used to generate training tasks?
Reviewer_7igfOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces Echoverse-MCP, an LLM-assisted pipeline which converts OpenAPI specifications and base environment templates into stateful MCP environments. Each environment is a server with its own small database, a set of tools the agent can call, some starting data, and rules that block invalid actions. The pipeline then creates tasks by having an LLM write a Python solution, running it, and saving the tool calls it made and the changes it caused in the database as proof that the task works. In total, the authors build 121 environments with 5,916 tools, and create 19,041 tasks in 45 of them. They also build a separate test set of 600 task suite (300 scripted-user and 300 simulated-user tasks over 20 environments). They train Qwen3.5-4B with reinforcement learning (GRPO) on 1,280 of these tasks. After training, the model scores 5.6 points higher on BFCL and about 3.2 points higher on average across three τ-bench domains.

Critical strengths

The paper addresses an important problem. Reinforcement learning for tool-using agents requires stateful, resettable environments with persistent databases, executable tools, and mechanisms for enforcing application rules. Building these environments by hand is expensive. The proposed LLM-assisted pipeline addresses this bottleneck by generating 121 environments and over 19,000 tasks across 45 of them. This resource could be valuable for training and evaluating tool-using agents.

Critical weaknesses

1. Empirical validation is insufficient.

  • Environment and task quality are insufficiently validated. Although the pipeline includes execution checks and model-based review, no independent human audit measures API fidelity, correctness of application rules, task solvability, or ambiguity.
  • There is no comparison against the same GRPO recipe using existing tool-use data (ToolACE, APIGen, τ-bench, AppWorld), adapted to a comparable training setting, or against SFT on the generated reference trajectories. Consequently, it is unclear how much benefit comes from the proposed environments versus additional tool-use training.
  • The learning benefits are inconsistent across models. Qwen3.5-4B improves on aggregate metrics, but Qwen3-8B Mixed RL regresses on BFCL overall accuracy (40.55 -> 38.85) and memory (23.01 -> 18.92), as well as Tau3 retail (52.6 -> 49.3) and telecom (40.8 -> 39.9) pass@1. These results limit the generality of the claimed training benefits.
  • The benefits and costs of scaling are not established. Training uses only 1,280 of the 19,041 generated tasks, while the task corpus covers 45 of the 121 environments. No experiment examines how increasing task volume or environment diversity affects downstream performance.
  • The low frontier-model success rates lack diagnostic analysis. The selected trajectory examples do not explain whether failures primarily reflect planning difficulty, ambiguous requests, environment defects.

2. The paper lacks clarity and important details.

  • Paper is hard to read as some of the concepts such as "Reusable domain model" and "reusable base environment" are central to the pipeline but never defined, and they appear to be used interchangeably. A lot of terms are used abstractly and difficult to follow.
  • Figure 1 is difficult to understand.
  • Some of crucial details are missing - Appendix A states that the optimiser configuration, reward coefficients, generation settings, and checkpoint-selection rule are not specified in the present manuscript.
Questions for authors

Please see weaknesses.

Reviewer_o2g9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Echoverse-MCP turns public OpenAPI specifications and what the paper calls a reusable domain model into stateful MCP servers, each with a private SQLite database and tools that enforce status transitions. Tasks come from writing a Python program that calls those tools, running it against the seeded state, and keeping the calls, responses and database deltas as evidence. The user request is then written backwards from that execution, in a fixed script for the scripted format or as a brief for a model that plays the user live in the simulated format. The resource has 121 environments and 5,916 tools, 19,041 tasks in 45 of those environments, and a separate 600-task suite over 20 environments. Mixed GRPO on Qwen3.5-4B over 1,280 tasks raises BFCL overall from 51.34 to 56.93, moves the three Tau3 domains by 3.2 points on average, and moves the local suite from 19.3 to 21.3 on scripted tasks and from 40.2 to 51.0 on simulated ones.

Critical strengths
  1. Execution grounding is the right design for stateful tools. The target of a task is a database state rather than a judged reply, the reference calls and deltas are retained, and the request is written so that facts the agent has to discover stay out of the prompt. Tools average 48.9 per environment and references average 7.15 calls over 6.24 distinct tools, so selecting the right tool from about fifty is part of the work rather than an afterthought.

  2. The workload is not toy-sized. Scripted references average 10.02 calls, 64.1% of them reads, over 3 to 5 user turns, and the pool exercises 1,917 distinct tools in 14,981 distinct combinations (Table 7). Deep write chains (39.3%) and fan-in reconciliation (25.3%) are the patterns that single-turn function-calling data lacks.

  3. The reporting is unusually candid. The paper states that the Tau3 base cells are cellwise minima over five runs, that BFCL's Category-aware-2 directly evaluates 155 of 4,906 examples and inherits the rest, that 31 scripted tasks were picked because a specific model had failed them, and that the Qwen3-8B arm regresses on BFCL overall and on two Tau3 domains. Most submissions would have dropped the second block of Table 3.

Critical weaknesses
  1. The gold state and the environment come out of the same generator, so a wrong application rule is invisible everywhere. The reference program runs against the generated server, whatever state it reaches becomes the task's target, and the reward checks equality to that state (L646-650). If a generated refund or capture rule departs from the real service's, the task, its answer and its reward agree with the mistake, and nothing in the pipeline can tell. The checks that do exist are structural: tool counts match decorated functions (L167-168), status guards come out of a regex (L675), dependency edges are heuristic. The authors say as much at L169-170, so this is not hidden, but it leaves the headline number uninterpretable: 21.3% scripted pass@1 could mean weak agents or odd applications, and I cannot tell which from the paper. One check would settle it: give the request alone, with no reference, to a strong model or to a person and ask whether it reaches the same final state. Cases where it does not are ambiguous tasks, and that rate is the number the suite needs.

  2. Tasks are graded whole, so near misses and failures look the same, and the one measurement that separates them is left in a paragraph. §5.2 reports 155 of 180 required checks satisfied over 60 scripted trajectories (86.1%), with 58 of 60 making no extra state change (L431-433). Table 5 reports 21.3% pass@1 on 300 tasks for the same checkpoint. If the batch is drawn from that suite, both numbers hold only if a failing trajectory still satisfies about two and a half of its three checks: the policy usually does the work and loses a single condition at the end. That is the most useful thing in the section, and the text reads it as a success rate instead. It never says how the 60 trajectories relate to the 300 tasks. Report per-condition satisfaction over the full suite; it would show whether the 2.0 point scripted gain comes from better steps or from fewer last-step misses.

Questions for authors
  1. Every environment exposes 45 to 52 tools, and 108 of the 121 sit in 45 to 50 (Table 1, Table 6). Was interface size fixed by the generator? If so, the environments vary in schema and rules but not in how many tools the agent has to choose among, and the 1,412-edge Instagram graph and the 36-edge environment are the same interface at different depths.
  2. Table 3 shows multi-turn up 9.38 and memory up 7.74 while live drops 3.28 and non-live AST drops 1.40, for a net 5.59. No environment connects to a live API, so a live-API gain would have to transfer. Is there any task in the corpus that could train that category, and do the drops look like forgetting to you?
  3. Table 5 uses four attempts per task for the base rows and two for the trained row (L769-770), and no confidence intervals are given. Does the 2.0 point scripted gain survive at equal attempt counts?
  4. The simulated-user gain is judged by the same fixed Qwen3.8-27B that played the user and supplied the training reward (L343-345, L931-932). What does that gain become under a judge that never took part in training?
Reviewer_2QobOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Echoverse-MCP generates database-backed, stateful tool environments from public API specifications and domain models. Executed reference programs, ground task requests, and expected state changes. The resource comprises 121 environments and 5,916 tools; its 19,041 generated tasks cover 45 environments, and a separate 600-task evaluation suite covers 20. Mixed GRPO training combines scripted and simulated users. Qwen3.5-4B improves on the reported aggregate evaluations, although some capabilities regress, and comparison protocols limit interpretation.

Critical strengths
  1. Constructing persistent application state, resettable databases and guarded transitions addresses a real bottleneck in agent training. The breadth of the generated resource could make it useful beyond the particular model studied.

  2. Tasks retain executed reference calls and database changes, providing inspectable evidence rather than relying solely on generated dialogue. Separately reporting scripted and simulated interactions also avoids conflating different tasks and grading procedures.

Critical weaknesses
  1. The transfer comparisons need stronger controls. Tau3 base scores are cellwise minima over five runs, while equivalent selection is not reported for trained scores; this can exaggerate gains (Table 4). Both Qwen3.5-4B BFCL rows use Category-aware-2, but each combines 155 directly evaluated examples with 4,751 results inherited from that checkpoint’s earlier Category-aware run (Appendix D). This limits conclusions about a complete evaluation under the revised profile, rather than proving the reported gain invalid. Representative repeated-run comparisons and a complete direct BFCL evaluation would strengthen the evidence.

  2. The execution checks and separate model review are useful, but no quantified independent audit of application semantics or grading accuracy is reported (Sections 3.1–3.2). Generated code and references could share an incorrect assumption. Reproduction is also incomplete: Appendix A leaves reward coefficients, optimiser settings and the precise checkpoint-selection rule unspecified, although it states that selection uses separate validation. The overlap between training, validation and test environments/tasks is also unclear, limiting assessment of generalisation.

Questions for authors
  1. Evidence request: Can you report Tau3 means and uncertainty across the five base runs alongside equivalently evaluated trained runs, and a complete direct BFCL comparison under matched prompt settings? Do the conclusions hold?

  2. Clarification: Beyond the described checks, is there an independent semantic or grading audit, including whether valid alternative solutions pass? If so, please report its sample size and error rates.

  3. Clarification: Please specify training/validation/test overlap at the environment and task level, including shared task presentations, and provide the missing reward settings and precise checkpoint-selection rule.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.