acceptodds
Under review as a conference paper at ICLR 2027

Davinci-Tool: Scaling Agentic Tool-Use Supervision with Verified Demonstrations

12 views

Abstract

Reliable agentic tool use requires training data that pair diverse multi-step workflows with executable evidence that the requested outcomes were achieved. Scaling either trajectory coverage or verification alone is insufficient as new environments introduce distinct states, rules, and dependencies that must also be reflected in the success criteria. To address this challenge, we introduce **Davinci-Tool**, a B tool-use model, and **Davinci-ToolData**, an execution-grounded dataset designed to scale environment coverage and outcome verification together. Davinci-ToolData preserves restorable states, execution traces, outcomes, and verifiers as a unified substrate for supervision. Two complementary pipelines populate this substrate: Execution-Grounded Task Synthesis (EGTS) derives verified demonstrations from independently authored systems, and Executable Environment Synthesis (EES) synthesizes auditable environments, state-grounded tasks, and executable verifiers across single and coordinated domains. These pipelines yield verified trajectories spanning invoked tool names. We first train on these demonstrations with supervised fine-tuning, then reuse EES states and verifiers to compare alternative actions and construct outcome-grounded preferences for Direct Preference Optimization. Davinci-Tool improves Qwen3-Coder-30B-A3B-Instruct from to on -bench, from to on BFCL v4 multi-turn, and from to on MCP-Atlas, outperforming Qwen3-Coder-480B-A35B-Instruct on all three benchmarks with only of its total parameters.

Primary area:
foundation or frontier models, including LLMs
Keywords:
agentic tool use, large language models, verified demonstrations, executable environments, multi-step tool use, cross-domain tool use, preference optimization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_UmjtOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Davinci-Tool and Davinci-ToolData, an execution-grounded pipeline for scaling multi-step tool-use supervision. Each data instance preserves a restorable initial state, tool trajectory, outcome evidence, and executable verification result. Execution-Grounded Task Synthesis (EGTS) adapts existing environments by exploring feasible outcomes, synthesizing tasks from observed evidence, restoring the initial state, and collecting task-conditioned replays that must reproduce the target outcome. Executable Environment Synthesis (EES) instead builds and audits executable domains, then generates state-grounded tasks and verifiers either within a single domain (EES-Single) or across composed domains with shared entities, policies, and dependencies (EES-Cross).

The authors collect 30,395 verified trajectories spanning 2,260 invoked tool names. They first apply SFT to Qwen3-Coder-30B-A3B-Instruct, then construct 1,150 same-context preference pairs from restorable EES states and use executable downstream outcomes to supervise DPO. Davinci-Tool-30B achieves extensive improvement on three benchmarks after SFT and further DPO.

Critical strengths
  1. A coherent execution-grounded data substrate that supports both demonstrations and preferences.

  2. Strong empirical gains with broad coverage. For example, Davinci-Tool improves the controlled Qwen3-Coder-30B backbone by over 30 points on the three reported benchmark averages after SFT, and DPO adds further gains on all three aggregates.

Critical weaknesses
  1. The methodological novelty and data provenance are not sufficiently clear. A growing body of work has already explored executable-environment-based synthesis for tool-use agents, including EnvScaler as well as AWM [1], ScaleEnv [2], AgentWorld [3], and EnvFactory [4], several of which are not cited or compared. As a result, it is difficult to identify the unique design contribution of Davinci-Tool beyond prior pipelines. In addition, the paper does not clearly explain where the authored systems used by EGTS or the synthesized domains used by EES come from, nor whether decontamination against the three evaluation benchmarks is performed. Given the very large gains (over 30 absolute points on all three benchmarks), stronger analysis of data provenance, overlap, and novelty is necessary.

  2. The experimental design leaves several important gaps. The paper does not report Davinci-ToolData SFT+DPO results for Qwen3-8B and Qwen3-14B, nor corresponding MCP-Atlas results for these models, making it difficult to assess whether the full training recipe generalizes across scales. This is especially important because Qwen3-8B/14B are mixed reasoning models, while Qwen3-Coder-30B-A3B-Instruct is an instruct model, yet the paper does not explain how the data or training format is adapted across these model types. In addition, the proposed executable verification naturally appears suitable for online RL, which is highly relevant to recent agent-training work, but the paper only uses it for DPO. A clearer justification or RL-based comparison would strengthen the contribution. Finally, the results on Airline domain in τ2\tau^2-Bench are missing without any reasons.

[1] Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning, 2026.

[2] ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Training, 2026.

[3] Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence, 2026.

[4] EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RL, 2026.

Questions for authors
  1. Do the constructed SFT data contains reasoning content? If not, how do you use in Qwen3-8B/14B? Are those results without reasoning?

  2. Why do you not present the results on Airline domain of τ2\tau^2-Bench?

  3. Have you analyze the overlaps between your data and benchmark environments?

  4. Why do you not conducting experiments on RL?

7 comments
AuthorsOpenReview ↗

Response to Reviewer Umjt (1/4)

We sincerely thank the reviewer for the thorough and constructive assessment. We especially appreciate your recognition of the shared execution-grounded foundation for demonstrations and preferences, as well as the substantial gains across all three benchmarks. Your careful attention to methodological novelty, data provenance, and evaluation reliability has helped us present these contributions more clearly.

We clarify the relationship between EES and EnvScaler, distinguish our new design choices from prior pipelines, and provide an overlap audit. We also report teacher and frontier-model evaluations, matched-count data-source comparisons, and additional DPO analyses. Together, these results help explain where the gains come from and how the pipeline is evaluated.

Response to Critical Weakness 1: Methodological contribution and data provenance

Critical Weakness 1: The methodological novelty and data provenance are not sufficiently clear. … In addition, the paper does not clearly explain where the authored systems used by EGTS or the synthesized domains used by EES come from, nor whether decontamination against the three evaluation benchmarks is performed. … Given the very large gains (over 30 absolute points on all three benchmarks), stronger analysis of data provenance, overlap, and novelty is necessary.

Our central contribution is a pipeline for constructing high-quality, executable-verified tool-use data from different environment sources. EGTS integrates 11 independently authored systems, including API-Bank, AppWorld, and Toolathlon (full inventory in the appendix), and converts explored outcomes into verified, task-conditioned replays. They share one collection and verification pipeline, with a median system-specific adapter size of about 1.1K lines. EES-Single starts from 191 executable EnvScaler domains (Song et al., 2026). We test their tool implementations, repair issues found during execution, and construct verified tasks on the audited environments. EES-Cross then composes these single-domain environments into 246 cross-domain settings with shared entities, policies, and state dependencies. Its trajectories show a 14.50% domain-switch ratio, with 6.39% involving cross-domain entity transfer. These steps extend the available verified supervision from individual environments to coordinated, multi-domain workflows.

We appreciate the references to EnvScaler (Song et al., 2026), ScaleEnv (Tu et al., 2026), Agent World Model (Wang et al., 2026), Agent-World (Dong et al., 2026), and EnvFactory (Xu et al., 2026). These works contribute to executable environment construction and agent training. Our work connects audited single-domain environments to explicit cross-domain composition, verified replay from authored systems, and the reuse of states and verifiers for SFT and decision-level DPO.

AuthorsOpenReview ↗

Response to Reviewer Umjt (2/4)

Response to Critical Weakness 2: Data quality, model coverage, and post-training scope

Evidence for the reported gains. Our primary goal is to build high-quality, executable-verified agentic training data and demonstrate its value for supervised learning. SFT accounts for most of the improvement over Qwen3-Coder-30B. It raises performance by 32.7 points on τ²-bench, 30.8 on BFCL v4, and 35.9 on MCP-Atlas. We thank the reviewer for the attention to evaluation reliability, which led us to recheck the original score files and correct the base BFCL score from 30.4 to 29.9 in the revised Table 2. Decision-level DPO adds 3.1, 2.1, and 3.4 points, respectively. We also evaluated the collection teacher, DeepSeek-V4-Pro, under the same protocols. The trained 30B model scores 62.75 on BFCL, above the teacher’s 54.88, and reaches 88.16 versus 90.35 on τ²-bench. In our additional frontier comparison under the same protocols, it also exceeds Kimi-K3 (61.38) and stays close to GLM-5.2 (66.38) on BFCL.

Model / training corpusSFT trajectoriesτ²-bench Avg.BFCL v4MCP-Atlas
Qwen3-Coder-30B (base)n/a52.4029.8816.90
Qwen3-Coder-30B + Toucan-1.5M SFT~1.5M77.7041.9022.50
Davinci-Tool-30B-SFT30,39585.0960.6352.81
Davinci-Tool-30B (SFT + DPO)30,39588.1662.7556.18
DeepSeek-V4-Pro (collection teacher)n/a90.3554.8866.29

The Toucan-1.5M SFT baseline offers another comparison of data effectiveness: using the same 30B backbone and SFT settings, our substantially smaller verified corpus yields +7.4 / +18.7 / +30.3 points on τ²-bench, BFCL, and MCP-Atlas. These results are consistent with our data construction process. The teacher is used for exploration and collection. EGTS guides exploration with progressive tool exposure and reusable experience, then collects new task-conditioned replays. Only trajectories that satisfy execution and outcome checks enter SFT. EES supplies additional verified demonstrations from audited domains. This process turns teacher-guided exploration into verified, task-grounded supervision and helps explain the strong BFCL results.

Data source complementarity. We also trained four SFT models with 8,000 trajectories each, the same initialization, and the same training settings. The mixed dataset is best or tied for best on all four reported metrics. For example, on MCP-Atlas it scores 55.06, compared with 48.31 for the strongest single source. Compared with EES-Cross, the mixture improves BFCL by 11.0 points (paired McNemar p < 0.001) and MCP-Atlas by 6.75 points (p = 0.031). The different strengths of EGTS, EES-Single, and EES-Cross support the value of combining their supervision.

SFT data (8,000 trajectories)τ² Retailτ² TelecomBFCL v4MCP-Atlas
EGTS64.9189.4758.1340.45
EES-Single64.9197.3755.0041.57
EES-Cross73.6896.4947.5048.31
Three-source mixture74.5697.3758.5055.06

Preference learning evidence. Across three DPO training seeds, the mean gains over SFT are +2.05 on τ²-bench, +1.83 on BFCL, and +3.75 on MCP-Atlas. The average Retail gain is +5.26 with a task-bootstrap 95% CI of [1.75, 9.65]. We also compared outcome-verified preferences with DeepSeek-V4-Pro relabeling on the same 1,150 decision pairs using the same SFT backbone and DPO settings:

Post-SFT comparison (seed 42)τ² RetailBFCL v4MCP-Atlas
SFT only73.6860.6352.81
DPO, DeepSeek preference labels75.4460.6356.18
DPO, verified-outcome labels79.8262.7556.18

The verified-label model improves Retail by 4.38 points and BFCL by 2.12 points over the LLM-relabeled counterpart. Along with the multi-seed gains, this supports using executable outcomes for decision-level preference supervision.

AuthorsOpenReview ↗

Response to Reviewer Umjt (3/4)

Response to Question 1: Reasoning format and smaller-model transfer

Question 1: Do the constructed SFT data contains reasoning content? If not, how do you use in Qwen3-8B/14B? Are those results without reasoning?

Our smaller-model experiments assess the quality and transferability of Davinci-ToolData as SFT supervision. The demonstrations contain no explicit reasoning traces, and Qwen3-8B/14B use non-thinking mode with model-compatible tool-call serialization. Both backbones show substantial SFT gains on τ²-bench and BFCL, demonstrating that the data benefits models beyond the primary 30B backbone.

Modelτ²-bench Avg.BFCL v4MCP-Atlas diagnostic*
Qwen3-8B (base, non-thinking)32.515.46.75
Qwen3-8B + Davinci-ToolData SFT74.633.96.74
Qwen3-14B (base, non-thinking)33.421.12.25
Qwen3-14B + Davinci-ToolData SFT76.841.43.37

MCP-Atlas scores were returned by the official evaluator at the original 40,960-token limit. Unfinished tasks, including context-overflow runs, count as failures. These are not extended-context evaluations.

We applied SFT followed by DPO to our primary 30B model to further demonstrate that the same environment and verifier framework can produce both verified demonstrations and decision-level preferences. This prioritizes the paper's data-construction contribution while keeping post-training costs manageable. Extending full SFT+DPO to 8B/14B is a natural direction beyond the current experiments.

For MCP-Atlas, the 8B and 14B base and SFT checkpoints used a 40,960-token context window, without YaRN. The benchmark includes extended interactions with real MCP servers, during which tool schemas and accumulated messages can exceed the available window (Bandi et al., 2026). We inspected the runs and confirmed context overflow in some trajectories. The official evaluator scored unfinished tasks as failures. Other failures arose from the smaller models' difficulty completing complex tool workflows. The values in the table are therefore scores under the original context-limited evaluation setting, rather than scores from an extended-context comparison. Extending the window for four checkpoints would also require additional serving adaptation and evaluation resources. Given our time and compute budget, we prioritized the complete τ²-bench and BFCL transfer evaluations. We will include these MCP-Atlas scores and the precise evaluation setting in the revised manuscript.

Response to Question 2: Choice of τ²-bench domains

Question 2: Why do you not present the results on Airline domain of τ²-Bench?

When selecting our τ²-bench evaluation scope, we considered reported concerns about the Airline benchmark. The Claude Opus 4.5 System Card documented policy loopholes in Airline and showed that corrections changed its reported score from 70.1 to 87.8. Subsequent Claude Opus 4.6 and Claude Sonnet 4.6 reports omitted Airline because these fixes had not yet been incorporated into the upstream benchmark. We also noted that the Gemini 3.1 Pro model card presents Retail and Telecom in its main τ²-bench comparison, while the MiniMax M2.5 model card separately reports Telecom. Given these findings and the differences between Airline evaluation versions, we chose Retail and Telecom for our evaluation. Retail measures stateful customer-service workflows, while Telecom evaluates dual-control interactions in which both the agent and user can use tools. We use this two-domain scope for every model and report the macro average of Retail and Telecom.

Reviewer_m4WHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Davinci-ToolData, a dataset of 30,395 verified tool-use trajectories constructed through two complementary pipelines, EGTS and EES, under a shared execution and verification framework. The authors train Davinci-Tool-30B using supervised fine-tuning (SFT) followed by outcome-grounded, decision-level direct preference optimization (DPO). The resulting model achieves substantial improvements on τ²-bench, BFCL v4 multi-turn, and MCP-Atlas.

Critical strengths

1.Unifies two complementary data routes—EGTS from authored systems (execution-first, outcome-grounded) and EES (audited, executable synthetic domains)—under a shared interface that preserves restorable states, execution traces, outcomes, and verifiers.

2.EES-Cross composes audited domains via explicit shared entities, policies, and dependencies, offering verified cross-domain workflows rather than ad hoc tool bags.

3.The decision-level DPO scheme reuses EES restorable states and executable verifiers to branch sibling decisions from the same context, labeling preferences with verified downstream outcomes without additional expert search policies or process evaluators.

Critical weaknesses

1.The unified execution and verification interface is an important component of the proposed framework. However, the experiments primarily demonstrate the effectiveness of the data generated by EGTS and EES and the overall training pipeline. The paper does not directly compare the unified framework with an alternative approach in which the two pipelines are constructed and verified separately before their training data are combined. Consequently, the additional benefits of the unified interface itself remain insufficiently validated.

2.Although Table 3 evaluates EGTS, EES-Single, and EES-Cross individually and in several combinations, the compared configurations also differ in training data size. Moreover, some two-source combinations underperform individual sources. Therefore, it remains unclear to what extent the improvements obtained by combining these data sources arise from their complementary characteristics rather than changes in training data size and composition.

3.The paper demonstrates that adding DPO after SFT improves model performance, but it does not provide controlled comparisons with alternative preference data construction methods. Consequently, it remains unclear what specific advantages the proposed bounded sibling branching and outcome-grounded preference construction offer over existing preference construction strategies.

Questions for authors

1.Unified execution and verification framework: Could the authors further clarify the specific advantages of the unified framework over independently constructing and verifying EGTS and EES data before combining them for training? Could a controlled comparison or quantitative analysis demonstrate the additional benefits of the shared interface for data construction, verification, or subsequent training?

2.Data source composition: Could the authors provide additional experiments that control the number of training samples or the token budget when comparing EGTS, EES-Single, EES-Cross, and their combinations? This would help clarify whether the observed improvements primarily result from complementary data sources or differences in training data size and composition.

3.Decision-level preference construction: Could the authors compare the proposed approach with at least one alternative preference construction strategy under the same SFT backbone and a comparable training budget? Such a comparison would help establish the independent advantages of the proposed decision-level preference construction method.

4.Stability of DPO improvements: DPO provides additional gains of 3.1, 2.2, and 3.4 points on the three benchmarks, respectively. Could the authors report repeated-run statistics or provide a task-level analysis to clarify the stability of these improvements?

5 comments
AuthorsOpenReview ↗

Response to Reviewer m4WH (1/2)

We sincerely appreciate the reviewer's thoughtful assessment and encouraging recognition of three central design choices: combining EGTS and EES under one execution interface, building cross-domain workflows with explicit dependencies, and reusing verified outcomes for decision-level preferences. Your questions have helped us clarify the distinct value of the shared infrastructure, data composition, and preference labeling.

To address them, we completed a four-way, matched-count SFT comparison, a DeepSeek-based preference-labeling ablation, and two additional DPO runs. We also clarify how the common interface supports environment integration and reuse of execution evidence across SFT and DPO.

Response to Critical Weakness 1 and Question 1: Value of the shared interface

Critical Weakness 1: The unified execution and verification interface is an important component of the proposed framework. However, the experiments primarily demonstrate the effectiveness of the data generated by EGTS and EES and the overall training pipeline. The paper does not directly compare the unified framework with an alternative approach in which the two pipelines are constructed and verified separately before their training data are combined. Consequently, the additional benefits of the unified interface itself remain insufficiently validated.

Question 1: Unified execution and verification framework: Could the authors further clarify the specific advantages of the unified framework over independently constructing and verifying EGTS and EES data before combining them for training? Could a controlled comparison or quantitative analysis demonstrate the additional benefits of the shared interface for data construction, verification, or subsequent training?

The unified interface is the foundation of our execution and verification substrate. It standardizes environment states, tool interactions, execution evidence, and outcome checks across 11 heterogeneous EGTS systems and the EES environments. The EGTS systems share roughly 42K lines of collection and verification code, while the median system-specific adapter is only 1.1K lines. This gives us a common route for producing verified demonstrations rather than a separate collection and verification implementation for every environment.

The benefit extends beyond SFT. We reuse EES states and executable verifiers to construct 1,150 decision-level DPO preference pairs without building a separate preference verification pipeline. The shared interface thus enables heterogeneous environment integration, consistent verification, and reuse of execution assets across SFT and DPO. We will make this design contribution clearer in the revised manuscript.

Response to Critical Weakness 2 and Question 2: Data composition under a fixed sample count

Critical Weakness 2: Although Table 3 evaluates EGTS, EES-Single, and EES-Cross individually and in several combinations, the compared configurations also differ in training data size. Moreover, some two-source combinations underperform individual sources. Therefore, it remains unclear to what extent the improvements obtained by combining these data sources arise from their complementary characteristics rather than changes in training data size and composition.

Question 2: Data source composition: Could the authors provide additional experiments that control the number of training samples or the token budget when comparing EGTS, EES-Single, EES-Cross, and their combinations? This would help clarify whether the observed improvements primarily result from complementary data sources or differences in training data size and composition.

We trained four SFT models with 8,000 trajectories each, using the same initialization and optimization settings. The three-source mixture uses approximately equal numbers from EGTS, EES-Single, and EES-Cross.

SFT data (8,000 trajectories)τ² Retailτ² TelecomBFCL v4MCP-Atlas
EGTS64.9189.4758.1340.45
EES-Single64.9197.3755.0041.57
EES-Cross73.6896.4947.5048.31
Three-source mixture74.5697.3758.5055.06

The strongest source differs by benchmark: EGTS leads on BFCL, EES-Single on Telecom, and EES-Cross on Retail and MCP-Atlas. The mixture is best or tied for best on all four metrics. It improves MCP-Atlas from the best single-source score of 48.31 to 55.06 (+6.75, paired McNemar p = 0.031). On BFCL, the mixture scores 58.50, close to EGTS (58.13) and above EES-Cross (47.50). The matched-count results support the complementary coverage of the three sources.

AuthorsOpenReview ↗

Response to Reviewer m4WH (2/2)

Response to Critical Weakness 3 and Question 3: Outcome-based preference labeling

Critical Weakness 3: The paper demonstrates that adding DPO after SFT improves model performance, but it does not provide controlled comparisons with alternative preference data construction methods. Consequently, it remains unclear what specific advantages the proposed bounded sibling branching and outcome-grounded preference construction offer over existing preference construction strategies.

Question 3: Decision-level preference construction: Could the authors compare the proposed approach with at least one alternative preference construction strategy under the same SFT backbone and a comparable training budget? Such a comparison would help establish the independent advantages of the proposed decision-level preference construction method.

Previous agent preference methods include contrastive trajectories in ETO (Song et al., 2024) and guided search in Agent Q (Putta et al., 2024). Our approach builds on this direction by using verified downstream outcomes to label alternative decisions sampled from the same executable context. We compared these labels with DeepSeek-V4-Pro preference judgments on the same 1,150 candidate pairs. The judge sees the decision context and both candidates, but not the downstream outcomes. It preferred the originally rejected candidate in 380 pairs (33.0%), whose labels were reversed. For 96 ties or uncertain judgments, we retained the original verified labels. The candidate pairs, SFT initialization, sample count, and DPO training settings were identical.

Trainingτ² Retailτ² TelecomBFCL v4MCP-Atlas
SFT73.6896.4960.6352.81
DPO with DeepSeek relabeling75.4493.8660.6356.18
DPO with verified outcomes79.8296.4962.7556.18

Verified-outcome labels yield 4.38 more points on Retail and 2.12 more on BFCL than DeepSeek relabeling, with the same MCP-Atlas score. With identical candidate decisions and DPO budgets, this ablation directly evaluates the value of executable-outcome preference labels. The stronger Retail and BFCL results support using verified downstream evidence to supervise decision refinement. We will include this preference-labeling comparison in the revised paper.

Response to Question 4: DPO stability across seeds

Question 4: Stability of DPO improvements: DPO provides additional gains of 3.1, 2.2, and 3.4 points on the three benchmarks, respectively. Could the authors report repeated-run statistics or provide a task-level analysis to clarify the stability of these improvements?

We trained two additional DPO seeds from the same SFT model with the same 1,150 pairs and training settings.

Modelτ²-bench Avg.BFCL v4MCP-Atlas
SFT85.0960.6352.81
DPO seed 4288.1662.7556.18
DPO seed 4386.8461.7556.18
DPO seed 4486.4062.8857.30
Three-seed DPO mean87.1362.4656.55

All three seeds improve the aggregate scores over SFT. Mean gains are +2.05 on τ²-bench, +1.83 on BFCL, and +3.75 on MCP-Atlas. Retail shows a mean gain of +5.26 (task-bootstrap 95% CI [1.75, 9.65]), and MCP-Atlas gains +3.75 ([0.75, 7.87]). BFCL Missing-Parameter improves by 6.0 points on average. These repeated runs support the benefit of outcome-based decision refinement after SFT.

These experiments distinguish three contributions: the interface makes execution evidence reusable across environments and training stages, the data sources provide complementary task coverage, and executable outcomes provide informative preference labels. We will add the completed comparisons and repeated-run results to the revised manuscript and clarify the interface's role in the training pipeline.

AuthorsOpenReview ↗

A Friendly Reminder to Review Our Responses

Dear Reviewer m4WH,

We sincerely appreciate your thoughtful review and constructive feedback. We are especially grateful for your recognition of the complementary EGTS and EES pipelines, explicit cross-domain dependencies, and the reuse of executable verification for decision-level preference learning.

In response to your concerns, we have provided additional evidence for the unified interface, including quantitative implementation details and verification reuse across SFT and DPO. We also conducted a controlled comparison using 8,000 trajectories per data source, introduced a DeepSeek-based preference-labeling baseline, and evaluated DPO stability across three training seeds.

These additional analyses help clarify the contributions of our shared verification infrastructure, complementary data sources, and outcome-verified preference supervision.

We would be very grateful if you could take another look at our responses when convenient. If the new evidence has satisfactorily addressed your concerns, we would sincerely appreciate your consideration of revisiting your assessment.

Thank you again for your valuable feedback and time.

Best regards, The Authors

Reviewer_u7qVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Davinci-Tool (a 30B-A3B model specifically tailored for tool-use tasks) and Davinci-ToolData, an execution-grounded dataset that contains 30395 verified trajectories spanning 2260 tools. Davinci-ToolData preserves restorable states, execution traces, outcomes, and verifiers as a unified substrate for supervision. Two complementary pipelines populate this substrate: Execution-Grounded Task Synthesis (EGTS) derives verified demonstrations from independently authored systems, and Executable Environment Synthesis (EES) synthesizes auditable environments, state-grounded tasks, and executable verifiers across single and coordinated domains. Empirical results on τ2\tau^2-Bench, BFCL v4 multi-turn and MCP-Atlas demonstrate that Davinci-Tool achieves performances on par with frontier models, obtained through Davinci-ToolData.

Critical strengths

The construction reuses one execution substrate for both stages of training: restorable EES states and executable verifiers that certify SFT demonstrations (Eq. 2–3) are re-used to build same-context decision-level preference pairs, where alternatives are sampled from the same fork state and labeled by verified downstream outcomes (Sec. 4.2, App. D.2). This is consequential because matched-context, outcome-verified preference labels are exactly the ingredient that is hardest to obtain for agent DPO, and the construction adds only bounded sibling rollouts (at most 1 + B(K−1) = 21 paths per scenario) rather than tree search.

Critical weaknesses
  1. The provenance of the SFT corpus undermines the central framing. All of Davinci-ToolData's trajectories are generated by DeepSeek-V4-Pro (Sec. 5.1), and the paper reports neither the generation-and-verification compute behind the 30,395 trajectories nor the teacher's own scores under the evaluation protocol (it is confusing why DeepSeek-V4-Pro does not appear in Table 2). The Toucan comparison compounds this: 30K vs. 1.5M trajectories differs in teacher and data source simultaneously, so even the matched-backbone deltas (7.4 / 18.7 / 30.3) cannot isolate the framework.

  2. The reference models are one generation out of date, so the positioning claim cannot be assessed. The frontier model comparison points are GPT-5.2, Claude Sonnet 4.5, GLM-4.7, MiniMax-M2.1, and Kimi-K2 (Sec. 5.1, Table 2), and the abstract's comparison is against Qwen3-Coder-480B; none of the current generation of frontier agent models (e.g., GPT-5.5/5.6, Claude Sonnet 4.6, GLM-5.2, Kimi-K3, MiniMax-M2.7) appears anywhere in the paper. Moreover, recently there have been many works on data synthesis for tool-use agents [1,2,3], yet none of them is mentioned in this work.

References

[1] ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Training. ICML 2026

[2] Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning. ICML 2026

[3] EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis. ACL 2026 Findings

Questions for authors
  1. (Clarification) What does DeepSeek-V4-Pro itself score on τ2\tau^2-bench, BFCL v4 multi-turn, and MCP-Atlas under your protocol, and what was the total generation-plus-verification compute spent building Davinci-ToolData? Without the teacher's own numbers, the reader cannot tell how much headroom remains between the trained 30B and its teacher.

  2. (Clarification) Could you add at least one current-generation frontier agent (e.g., GPT-5.5, Claude Sonnet 4.6, or Kimi-K3) as a reference point under your harness? Given the pattern in Table 2 (GPT-5.2: 89.9 on τ2-bench vs. 28.9 on BFCL v4), please also check protocol compatibility for closed models — strict decoding or harness settings may be penalizing them.

  3. (Clarification) Could you report one size-matched mixture comparison (e.g., the full mixture subsampled to the trajectory count of its best single-source run, such as the 8,545-trajectory EES-Cross row of Table 3)? This is the missing control for the complementarity claim.

  4. (Clarification) Are the Table 2 numbers single evaluations of single training runs? How stable are the DPO deltas (3.1 / 2.2 / 3.4) and the per-category changes (e.g., Missing-Function −1.5) across evaluation repeats or additional seeds, particularly on the 89-task MCP-Atlas split?

  5. (Clarification) Preference pairs are built only from EES-Single/EES-Cross scenarios (Sec. 4.2), even though EGTS-authored systems also expose restoration ρ and verification. Does decision-level preference construction transfer to authored environments, or does it silently depend on EES-controlled predicates — i.e., what bounds the applicability of the post-SFT stage?

  6. (Suggestion) The paired analysis (Table 12) shows 78 matched tasks regressing from success to failure after SFT; a brief characterization of what SFT breaks (e.g., which BFCL categories) would make the trade-off concrete for practitioners.

4 comments
AuthorsOpenReview ↗

Response to Reviewer u7qV (1/3)

We sincerely thank the reviewer for the thoughtful and constructive assessment. We particularly appreciate your recognition of our reuse of restorable execution states and verifiers to construct same-context, outcome-verified DPO preferences. As you noted, obtaining reliable decision-level preference supervision is challenging, and this reuse is central to Davinci-Tool.

We have now evaluated the collection teacher and four newer frontier models, quantified collection resources, and completed a matched-count data-source study and two additional DPO runs. We also examined BFCL regressions and the applicability of preference construction to EGTS. These results provide additional evidence regarding data provenance, evaluation reliability, and training stability.

Response to Critical Weakness 1 and Question 1: Teacher performance and construction cost

Teacher performance. We evaluated the data collection teacher, DeepSeek-V4-Pro, using the same benchmark protocols. Davinci-Tool scores 62.75 on BFCL v4, compared with 54.88 for the teacher. SFT already reaches 60.63, and DPO further improves it to 62.75. On τ²-bench, Davinci-Tool is close to the teacher (88.16 vs. 90.35).

Model / training corpusSFT trajectoriesτ²-bench Avg.BFCL v4 multi-turnMCP-Atlas
DeepSeek-V4-Pro (collection teacher)n/a90.3554.8866.29
Qwen3-Coder-30B (base)n/a52.4029.8816.90
Qwen3-Coder-30B + Toucan-1.5M SFT~1.5M77.7041.9022.50
Davinci-Tool-30B-SFT30,39585.0960.6352.81
Davinci-Tool-30B (SFT + DPO)30,39588.1662.7556.18

We rechecked Table 2 against the original score files and found that the base model's Missing-Function score is 15.0 rather than 17.0, which changes its BFCL average from 30.4 to 29.9. The revised paper reports the corrected value.

DeepSeek-V4-Pro serves as the collection policy. In EGTS, progressive tool exposure and reusable guidance help it discover achievable outcomes. We then synthesize user requests, restore the initial states, and collect new task-conditioned replays. We retain trajectories that pass execution and outcome checks. EES adds verified supervision from audited environments. These verified, task-conditioned demonstrations help explain why the student can outperform the teacher on BFCL. The Toucan-1.5M baseline in the table uses the same Qwen3-Coder-30B backbone and SFT hyperparameters. Our 30,395 verified demonstrations outperform its approximately 1.5M trajectories by 7.4 points on τ²-bench, 18.7 on BFCL, and 30.3 on MCP-Atlas. This highlights the effectiveness of Davinci-ToolData as a compact, high-quality SFT corpus. We appreciate the reviewer's point that the two corpora also differ in their collection teachers, and will clarify the comparison's purpose in the revised paper. The matched-count experiment below further examines the complementarity of our three data sources.

Construction cost. EGTS and EES ran concurrently on a shared deployment of 16 H200 GPUs, with generation concurrency capped at 16. EGTS took about two weeks and EES about one week within that period. The SFT collection window corresponds to at most 5,376 provisioned H200 GPU-hours. Subsequent preference collection took 2 to 3 days on 4 H200 GPUs and produced 1,150 pairs. Environment execution and verification ran on CPUs.

AuthorsOpenReview ↗

Response to Reviewer u7qV (2/3)

Response to Critical Weakness 2 and Question 2: Current models, evaluation protocol, and related work

We evaluated GLM-5.2, Kimi-K3, MiniMax-M3, and GPT-5.6-Sol with our benchmark protocols. Davinci-Tool reaches 62.75 on BFCL v4. It exceeds Kimi-K3 (61.38), GPT-5.6-Sol (46.63), and MiniMax-M3 (38.75), and is close to GLM-5.2 (66.38). Its 88.16 on τ²-bench is also within 1.31 points of Kimi-K3. The results show that our specialized 30B model remains competitive with current frontier models, especially on multi-turn function calling.

Modelτ²-bench Avg.BFCL v4 multi-turnMCP-Atlas
GLM-5.291.6766.3867.42
Kimi-K389.4761.3858.43
MiniMax-M390.7938.7555.06
GPT-5.6-Sol (medium)73.6846.6377.53
Davinci-Tool-30B88.1662.7556.18

To investigate the low GPT-5.2 BFCL score, we inspected its BFCL evaluation records. Of 583 failed tasks, 264 contained a textual response at a point where a tool call was required, including refusals and requests for additional information. We observed no decoding or tool-call parsing errors in the inspected records. Thus, a frequent failure mode was the model choosing text over the expected tool action. Following the reviewer's suggestion, we also checked the Table 2 entries against the original score files. Two Missing-Function entries did not match these files: GPT-5.2 and the Qwen3-Coder-30B base model score 15.5 and 15.0, so their BFCL averages are 27.1 and 29.9 rather than 28.9 and 30.4. We have corrected these entries in the revised paper, and the comparisons with Davinci-Tool are unchanged.

Relation to prior work. We appreciate the references to ScaleEnv (Tu et al., 2026), Agent World Model (Wang et al., 2026), and EnvScaler (Song et al., 2026). ScaleEnv and AWM develop executable environments for agent training. EES-Single uses 191 executable domains from EnvScaler. We test their tool functions and state transitions, correct implementation issues found during execution, and construct verified tasks on the audited domains. EES-Cross then composes these domains through shared entities, policies, and state dependencies to create coordinated workflows. Along with EGTS task-conditioned replay and the reuse of execution evidence for SFT and DPO, these steps define our contribution.

Response to Question 3: Complementarity at a matched trajectory count

Question 3: (Clarification) Could you report one size-matched mixture comparison (e.g., the full mixture subsampled to the trajectory count of its best single-source run, such as the 8,545-trajectory EES-Cross row of Table 3)? This is the missing control for the complementarity claim.

We trained four SFT models with 8,000 verified trajectories each and the same initialization and training configuration. The mixture uses approximately equal amounts of EGTS, EES-Single, and EES-Cross data.

SFT data (8,000 trajectories)τ² Retailτ² TelecomBFCL v4MCP-Atlas
EGTS64.9189.4758.1340.45
EES-Single64.9197.3755.0041.57
EES-Cross73.6896.4947.5048.31
Three-source mixture74.5697.3758.5055.06

The strongest single source varies by benchmark: EGTS leads on BFCL, EES-Single on Telecom, and EES-Cross on Retail and MCP-Atlas. The mixture matches or exceeds the best single-source score on every metric. Its clearest additional gain is on MCP-Atlas, where it improves over the strongest single source by 6.75 points (paired McNemar p = 0.031). On BFCL, the mixture reaches 58.50, preserving EGTS's strength (58.13) while outperforming EES-Cross by 11.0 points (p < 0.001). The results show complementary coverage at a fixed trajectory count.

AuthorsOpenReview ↗

Response to Reviewer u7qV (3/3)

Response to Question 4: DPO stability across seeds

Question 4: (Clarification) Are the Table 2 numbers single evaluations of single training runs? How stable are the DPO deltas (3.1 / 2.2 / 3.4) and the per-category changes (e.g., Missing-Function −1.5) across evaluation repeats or additional seeds, particularly on the 89-task MCP-Atlas split?

The original Table 2 uses one training checkpoint per setting. We added two DPO seeds (43 and 44) using the same SFT model, 1,150 preference pairs, and training configuration.

Modelτ²-bench Avg.BFCL v4MCP-Atlas
SFT85.0960.6352.81
DPO seed 4288.1662.7556.18
DPO seed 4386.8461.7556.18
DPO seed 4486.4062.8857.30
Three-seed DPO mean87.1362.4656.55

All three seeds improve the aggregate scores over SFT. The average gains are +2.05 on τ²-bench, +1.83 on BFCL, and +3.75 on MCP-Atlas. The clearest gains are on Retail (+5.26, task-bootstrap 95% CI [1.75, 9.65]) and MCP-Atlas (+3.75, [0.75, 7.87]). On BFCL, Missing-Parameter improves by 6.0 points on average, while the Missing-Function decrease in the original run does not persist across seeds.

Response to Question 5: Applicability to authored environments

Question 5: (Clarification) Preference pairs are built only from EES-Single/EES-Cross scenarios (Sec. 4.2), even though EGTS-authored systems also expose restoration ρ and verification. Does decision-level preference construction transfer to authored environments, or does it silently depend on EES-controlled predicates — i.e., what bounds the applicability of the post-SFT stage?

Yes. EGTS systems expose restoration and outcome-verification interfaces, so the same preference construction procedure can be applied when intermediate decision states can be restored for alternative continuations. We collected the 1,150 DPO pairs from EES for efficiency: repeated branching and verification are easier to orchestrate in simulated environments than across EGTS systems involving Docker, MCP tools, and external services. We prioritized EES given our collection time and resource budget, and will clarify this choice in the revised paper.

Response to Question 6: SFT regression analysis

Question 6: (Suggestion) The paired analysis (Table 12) shows 78 matched tasks regressing from success to failure after SFT; a brief characterization of what SFT breaks (e.g., which BFCL categories) would make the trade-off concrete for practitioners.

We examined the 58 BFCL tasks that changed from success to failure after SFT, out of the 78 regressions in the paper's paired analysis. The most common first errors are wrong argument values (23), missing required steps (8), and acting before enough information is available (8). In 16 cases, the model replaces a value given by the user with a common default. These errors point to argument grounding and step completion as the main areas for improvement. At the same time, SFT changes 304 BFCL tasks from failure to success, giving a clear picture of the overall trade-off.

The new evidence places the 30B model's performance in context: verified demonstrations deliver most of the gain, the sources cover different tool-use tasks, and decision-level DPO adds improvements across repeated runs. We will incorporate the completed evaluations, construction-cost details, error analysis, and related-work discussion into the revised manuscript.

AuthorsOpenReview ↗

A Friendly Reminder to Review Our Responses

Dear Reviewer u7qV,

We sincerely thank you again for your careful review, positive feedback, and constructive suggestions. We particularly appreciate your recognition of our unified execution framework and outcome-verified preference construction.

We have provided detailed responses and additional experimental evidence addressing your concerns. Specifically, we evaluated the collection teacher and newer frontier models, clarified data construction costs, conducted a matched-count comparison of our three data sources, and added multiple DPO training seeds. We also analyzed BFCL regression cases and clarified the applicability of preference construction to EGTS.

We hope these additional results help address your concerns and strengthen the evidence for our approach.

When convenient, we would be very grateful if you could review our responses and let us know whether any questions remain. If you find our clarifications and additional evidence satisfactory, we would sincerely appreciate your consideration of updating your assessment.

Thank you again for your time and thoughtful feedback.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.