acceptodds
Under review as a conference paper at ICLR 2027

Know What You Don’t Know: Agentic Data Synthesis by Evolving Generation Space for Closed-Loop Learning

Abstract

The continued improvement of large language models increasingly depends on synthetic experience, yet effective data synthesis for interactive agents requires not only executable and verifiable trajectories, but also interactions that provide learning value for the current task agent. Existing methods increasingly incorporate environment feedback, data validation, or learner failures, but still face a feedback-to-decision gap: observed weaknesses do not directly specify how future synthesis should change, or whether they can be represented by the current synthesis space. To address this gap, we propose EvoSyn, a learning-feedback-driven multi-agent framework that organizes synthesis around a revisable state specifying what interaction factors can be deliberately generated and where synthesis effort should be allocated. A role-specialized multi-agent system translates learner feedback into synthesis revisions, while a Designer allocates synthesis directions and a Planner grounds them into executable experience. This process enables within-space adaptation toward weak or under-covered regions and synthesis-space expansion that promotes newly discovered factors into explicit generation controls. Experiments on τ²-Bench and CAR-Bench show that EvoSyn improves average task success by 13.3 points over the strongest synthesis baseline, while producing higher-quality and structurally broader synthetic experience and discovering controllable factors missed without learner feedback.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Data Synthesis, Multi-Agent LLMs, Self-Evolving Agents

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_yNngOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EVOSYN, a feedback-driven framework for agentic data synthesis. It iteratively generates verified teacher trajectories, fine-tunes a student model using SFT, and adjusts subsequent data generation based on the student's weaknesses. The method is evaluated on τ²-Bench and CAR-Bench, showing improvements in task success rates.

Critical strengths
  1. The paper is clearly written and well organized.

  2. The experiments are extensive, including multiple benchmarks, model scales, ablation studies, and analyses of synthetic data quality.

Critical weaknesses
  1. The novelty is unclear. The method seems to be iterative teacher-student distillation with adaptive data generation. What is the key novelty beyond existing adaptive data synthesis methods? Also, since training tasks are generated in the same environments as the benchmarks, they may be highly similar to test tasks, raising concerns about whether the reported performance reflects real generalization.

  2. Repeated SFT on synthetic data may hurt the model's generalization ability. However, the paper does not evaluate performance on unrelated tasks or benchmarks.

  3. The experiments do not clearly separate the effects of the synthesis strategy, data size, and training budget. Table 2 matches the candidate generation budget, but EVOSYN retains more examples for SFT. The size-matched comparison only measures trajectory coverage, not downstream task success. A comparison with matched training data sizes and training budgets is needed to better isolate the benefit of the synthesis strategy.

  4. The need to add task factors dynamically is not directly tested. Although the paper includes an ablation without the Reflector, a more direct baseline would provide all final factors, such as num_write and num_read, from the first round. It would then adjust only the sampling proportions based on student feedback. This would help show whether the gains come from dynamic factor discovery or simply from having a more complete set of task attributes.

  5. It is unclear how new factors are discovered. Appendix D shows that the Reflector’s input can already include suggested factor names and values, and the prompt asks it to prioritize these suggestions. The authors should explain how the diagnostic probes and suggestions are produced and whether they rely on predefined candidates. Otherwise, it is hard to tell whether the system discovers new task attributes or simply turns attributes already supplied in its input into generation controls.

Questions for authors

See weaknesses.

7 comments
AuthorsOpenReview ↗

Response to Reviewer yNng

We sincerely thank you for the careful review and constructive feedback. Your insightful questions and suggestions have helped us further clarify the technical novelty of EvoSyn and better characterize its generalization and factor-discovery mechanisms. We hope the following detailed responses, additional analyses, and new experiments adequately address your main concerns and provide stronger evidence for the contributions of our work.


Reply to Weakness

Reply to W1 -- Clarifying the Technical Novelty and Generalization of EvoSyn

We sincerely thank you for raising these important questions. Our novelty lies not in iterative distillation itself, but in making the controllable synthesis representation explicitly revisable through learner feedback. We clarify this distinction and provide supporting evidence below.

1. Technical novelty: evolving the synthesis space, not merely the generated tasks.

Existing adaptive synthesis methods typically use feedback to generate harder tasks, explore new environments, or adjust task distributions. EvoSyn addresses a different question: what if the learner's remaining weaknesses cannot be adequately expressed by the current synthesis controls?

Formally, EvoSyn maintains a synthesis state Sr=(Or,r), where Or defines the controllable interaction factors and r specifies their generation allocation. Feedback drives two distinct decisions:

  • Within-space adaptation: Reallocate synthesis effort over existing factors by updating r.
  • Synthesis-space expansion: Discover previously unrepresented, failure-relevant factors and incorporate them into Or+1 as actionable generation controls.

This distinction goes beyond conventional adaptive generation:

ApproachPrimary Adaptation MechanismDistinction from EvoSyn
R-Zero / Socratic-ZeroGenerate increasingly challenging reasoning problems from solver feedbackNo explicit revision of environment-grounded interaction-factor controls
AutoPlay / ExplorerExplore environment states and construct executable tasksEnvironment grounding rather than post-training learner-guided synthesis-space revision
CoEvolve / Agent-WorldAdapt tasks, interaction distributions, or environments using agent feedbackDo not explicitly demonstrate learner-grounded promotion of missing interaction factors into reusable generation controls
EvoSynRevise both synthesis allocation and the controllable factor representationFeedback-driven factor discovery, operational controllability, and targeted resynthesis

Importantly, our claim is not unrestricted ontology invention. EvoSyn identifies measurable, previously unrepresented interaction factors and promotes them into reusable synthesis controls based on learner feedback.

2. Mechanistic evidence: discovery, controllability, and learning utility.

Our experiments validate this distinction at multiple levels:

  • Feedback-grounded discovery (Appendix B.6): The feedback-conditioned Reflector identifies both subsequently adopted factors, num_write and num_read (2/2), whereas the no-feedback control fails to recover either (0/6).
  • Operational controllability (Appendix B.7): Requested versus realized factor values achieve 99.6–100% exact-match accuracy, demonstrating that newly introduced dimensions are actionable generation controls rather than post-hoc labels.
  • Non-redundant failure information (Appendix B.10): The newly promoted CAR-Bench factors num_write and num_read provide 0.168 and 0.134 nats of additional learner-failure information after conditioning on the existing synthesis representation.
  • Downstream effectiveness (Tables 1–2): EvoSyn achieves 77.8% average success, compared with 63.2% for R-Zero-style and 64.5% for AutoPlay-style synthesis adapted to our agentic setting. Removing the Reflector reduces CAR-Bench success from 72.6% to 63.3%, directly supporting the contribution of synthesis-space revision.

These results establish a concrete distinction: EvoSyn does not simply use feedback to generate different examples; it converts learner weaknesses into new, verifiable synthesis capabilities.

3. Generalization: unseen tasks within executable environments.

We appreciate the concern regarding training–test similarity. Importantly, sharing an environment does not imply sharing task instances or test supervision. Our setting evaluates in-environment generalization: the agent learns from synthesized interactions and is evaluated on held-out tasks under the same tool interfaces and domain policies.

As specified in Appendix B.1:

  • Synthesis feedback is obtained from a held-out diagnostic set disjoint from the official test set.
  • Official benchmark test tasks are reserved for final evaluation; test labels are never used for synthesis or training.
  • Improvements are consistently observed across four distinct evaluation domains (Airline, Retail, Telecom, and CAR-Bench), rather than a single task distribution.

Beyond task success, Appendix B.9 provides complementary evidence of structural generalization: under matched sample counts, EvoSyn covers 1.32× more distinct executable interaction sequences on CAR-Bench and 2.19× more on Airline than Static Synthesis. These findings indicate that the method expands interaction coverage rather than merely accumulating repetitive trajectories.


Reply to W2 -- General Capability Retention Under Repeated Synthetic SFT

We thank the reviewer for raising this important concern. To examine whether repeated synthetic SFT compromises general-purpose capabilities, we conducted additional evaluations on four independent benchmarks covering instruction following, mathematical reasoning, and general knowledge. We compare the original Qwen3-8B, an R-Zero-style synthesis baseline, EvoSyn-R1, and EvoSyn-Final under identical evaluation protocols, without additional training or benchmark-specific tuning.

1. Generalization results on unseen benchmarks.

All models are trained only on CAR-Bench synthetic data, while the following benchmarks are entirely outside the training domain.

BenchmarkQwen3-8B BaseR-Zero StyleEvoSyn-R1EvoSyn-Final
IFEval (Instruction Following)80.075.480.680.6
GSM8K (Mathematical Reasoning)87.691.491.191.1
BBH (Complex Reasoning)76.979.476.580.1
MMLU-Pro (General Knowledge)69.169.266.367.9

2. Repeated SFT preserves general capabilities without measurable cumulative degradation.

The results provide three key observations:

  • General capability retention: Compared with the original Qwen3-8B, EvoSyn-Final maintains comparable IFEval performance (80.6 vs. 80.0) and MMLU-Pro performance (67.9 vs. 69.1), while significantly improving GSM8K by +3.5 pp (95% CI [+1.6, +5.4]) and BBH by +3.2 pp ([+0.7, +5.9]).
  • No measurable additional degradation across rounds: From EvoSyn-R1 to EvoSyn-Final, performance remains unchanged on IFEval (80.6 → 80.6) and GSM8K (91.1 → 91.1), improves on MMLU-Pro (66.3 → 67.9), and significantly improves on BBH (76.5 → 80.1, +3.6 pp). None of these four benchmarks exhibits a statistically significant decline from R1 to Final.
  • Comparison with learner-feedback synthesis: EvoSyn-Final significantly outperforms the R-Zero-style baseline on IFEval (+5.2 pp, 95% CI [+1.9, +8.3]) while achieving comparable GSM8K and MMLU-Pro performance.

These additional experiments show that EvoSyn's repeated synthesis-learning rounds do not introduce measurable cumulative degradation across the evaluated general-purpose capabilities. Instead, the final model preserves instruction-following and general-knowledge performance while improving mathematical and complex reasoning. These results provide complementary evidence that EvoSyn's specialized agentic improvements need not come at the expense of broader LLM capabilities. We will incorporate these evaluations and their protocols into the revised manuscript.

Reply to W3 -- Isolating the Benefit of Synthesis Strategy Beyond Data Size and Training Budget

To address this, we conducted a new Budget-Matched Static Synthesis experiment, explicitly controlling the final SFT data volume and aggregate training-token budget while keeping the Static synthesis strategy unchanged.

1. New controlled experiment: matched data and training budgets.

We expanded Static Synthesis using its original fixed generation procedure, without learner feedback or synthesis-space evolution. The resulting baseline matches EvoSyn's final SFT dataset size and processes 235.3M training tokens, slightly exceeding EvoSyn's 227.6M (+3.4%) across the complete six-epoch training runs.

MethodSFT ExamplesR1 / R2 / R3 Training TokensTotal TokensSuccess Rate
Static (Original)63937.5M / 20.0M / 49.3M106.8M61.3%
Static (Budget-Matched)~1,28737.5M / 97.6M / 100.2M235.3M64.6%
EvoSyn1,28737.5M / 91.1M / 99.0M227.6M72.6%

Importantly, R1 is identical across methods, and Budget-Matched Static receives more training tokens than EvoSyn in both subsequent rounds.

2. The synthesis strategy retains a substantial independent advantage.

Increasing Static's data and compute improves its success rate by +3.3 pp (61.3% → 64.6%), confirming that training volume contributes to performance. However, EvoSyn still outperforms Budget-Matched Static by +8.0 pp (72.6% vs. 64.6%), retaining approximately 71% of its original performance advantage despite using slightly fewer training tokens.

The structural-quality results further explain this persistent gap:

MetricStatic (Budget-Matched)EvoSyn
Factor Coverage0.8120.893
Factor Balance0.4470.508
Interaction Coverage0.3580.373
Trajectory Diversity0.7840.938

These results complement Figure 12, which already demonstrates 1.32× and 2.19× greater size-matched structural support on CAR-Bench and Airline, respectively. Crucially, our new experiment extends the evidence to actual downstream task success under comparable training budgets, rather than structural coverage alone.


Reply to W4 -- Does Dynamic Factor Discovery Matter Beyond a Complete Synthesis Space?

We thank the reviewer for this insightful suggestion. To directly distinguish dynamic factor discovery from simply having a complete set of task attributes, we implemented the proposed Oracle-Full-Space + Adaptive Allocation baseline on CAR-Bench.

1. New experiment: Dynamic expansion vs. oracle-defined synthesis space.

The Oracle baseline receives all factors eventually discovered by EvoSyn (including num_write and num_read) from R1, while retaining feedback-guided allocation through the Designer. Its synthesis space remains fixed, whereas EvoSyn progressively introduces new factors based on learner feedback.

MethodR1R2R3R1→R3 Gain
Fixed-State Synthesis60.7%62.7%61.3%+0.6 pp
Oracle-Full-Space + Adaptive Allocation64.0%67.3%69.3%+5.3 pp
EvoSyn (Dynamic Expansion)60.7%65.5%72.6%+11.9 pp

2. Key finding: Dynamic expansion ultimately surpasses the oracle baseline.

Despite having privileged access to all final factors, Oracle's initial advantage progressively disappears:

  • Early advantage: Oracle outperforms EvoSyn by +3.3 pp in R1, confirming the usefulness of additional factors.
  • Stronger evolution: EvoSyn achieves +11.9 pp improvement from R1 to R3, compared with +5.3 pp for Oracle.
  • Final advantage: EvoSyn reaches 72.6% vs. 69.3% (+3.3 pp), with a substantially larger R2→R3 improvement (+7.1 vs. +2.0 pp).

3. Why dynamic introduction matters.

These findings distinguish factor availability from learning-guided factor introduction. Rather than exposing all factors initially, EvoSyn progressively introduces num_write and num_read as the learner's weaknesses shift from write-intensive execution to read-intensive decision making (Figure 3). This is further supported by our feedback-conditioned discovery results (2/2 vs. 0/6, Appendix B.6) and factor controllability (99.6–100%, Appendix B.7).

The oracle-controlled comparison provides direct evidence that EvoSyn's benefit extends beyond a richer predefined synthesis space. Its stronger multi-round improvement and higher final performance support the value of introducing controllable factors in response to evolving learner weaknesses. We will incorporate this new comparison into the revised manuscript.

Reviewer_MZjQOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EvoSyn, a closed-loop multi-agent framework for synthetic data generation and iterative improvement of LLM agents. The key idea is to adapt the data synthesis process according to the evolving capability gaps of the downstream task agent. Specifically, EvoSyn introduces a revisable synthesis space that maintains both the controllable interaction factors and the allocation of generation effort. Based on learner feedback, the framework performs two types of updates: within-space adaptation, which reallocates synthesis budgets over existing factors, and synthesis-space expansion, which introduces new controllable factors to address previously uncovered capability gaps. The framework employs multiple specialized agents, including a Reflector, Designer, Planner, Generator, and Verifier, to generate executable trajectories and iteratively improve the downstream agent through supervised fine-tuning. Experiments on τ²-Bench and CAR-Bench demonstrate substantial improvements over several synthetic data generation baselines, supported by extensive ablation studies and analyses of synthesis diversity, factor discovery, and multi-round learning.

Critical strengths

Important research problem. The paper addresses an important challenge in agentic post-training: generating synthetic experiences that are not only executable and verifiable but also targeted toward the evolving weaknesses of the downstream learner.

Comprehensive and practical system design. EvoSyn integrates learner feedback, structured synthesis-space management, specialized synthesis agents, trajectory verification, and iterative training into a unified framework. Although the individual components are relatively straightforward, their integration provides a practical approach to closed-loop agent improvement.

Extensive experimental evaluation. The paper presents a substantial amount of experimental work across multiple agentic benchmarks, synthesis backbones, and downstream model sizes. The authors also provide comprehensive component ablations, multi-round analyses, synthetic data diversity measurements, and factor controllability evaluations. The reported downstream performance improvements are encouraging.

Critical weaknesses

W1. The paper is primarily a system integration work rather than a methodologically novel research contribution. The proposed EvoSyn framework combines several established techniques, including multi-agent collaboration, learner-feedback-driven data generation, structured task representation, trajectory verification, and iterative supervised fine-tuning. While the overall system is comprehensive and the experimental evaluation is extensive, most of these individual components have already been investigated in prior work. The main contribution appears to be the organization and integration of these existing mechanisms into a structured closed-loop synthesis framework. In particular, the paper does not introduce a fundamentally new learning algorithm, data generation model, or optimization mechanism. Much of the proposed synthesis evolution is implemented through LLM-based reflection and modifications to a structured synthesis ontology. Although such system-level integration can be practically valuable, the paper does not sufficiently demonstrate which technical innovations are uniquely responsible for the reported improvements. W2. The core idea of adapting and expanding synthesis dimensions according to learner feedback is closely related to existing recursive self-improvement approaches. The paper emphasizes its ability to revise the synthesis space by introducing new controllable interaction factors according to the downstream agent's capability gaps. However, the broader idea of continuously modifying data generation strategies based on learner feedback has already been explored in recursive self-improvement (RSI) and agent–data co-evolution research. For example, CoEvolve investigates mutual evolution between agent capabilities and training experiences through feedback-driven task synthesis. Other self-evolving approaches similarly adapt task distributions, generation strategies, or curricula according to the learner's performance. The main distinction of EvoSyn appears to be the explicit representation and expansion of synthesis factors through a structured ontology. While this formulation may improve interpretability and controllability, it is unclear whether it enables fundamentally new capabilities beyond existing RSI methods that dynamically adjust generation prompts, task constraints, or synthesis strategies. Therefore, the methodological novelty of the proposed synthesis-space evolution mechanism remains insufficiently established. W3. Insufficient experimental comparison with representative RSI and agent–data co-evolution methods. Although the paper includes several synthetic data generation baselines and controlled synthesis variants, its evaluation does not sufficiently cover representative RSI and agent–data co-evolution frameworks. In particular, closely related approaches such as CoEvolve, Agent-World, and SEAD are discussed in the related work but are not directly evaluated. While the authors include AgenticQwen and an R-Zero-style baseline, these comparisons do not fully establish the advantages of EvoSyn over existing RSI methods that also use iterative learner feedback to revise subsequent data generation. This omission is important because the central contribution of EvoSyn is its closed-loop synthesis evolution rather than the initial construction of executable trajectories. Consequently, the current results demonstrate the effectiveness of EvoSyn relative to the selected data synthesis methods, but do not convincingly establish its superiority over representative state-of-the-art RSI approaches. More direct comparisons under matched synthesis backbones, generation budgets, and downstream training settings would help clarify the incremental contribution of the proposed framework. W4. The unique contribution of synthesis-space expansion is not sufficiently isolated from ordinary feedback-driven synthesis adaptation. The central methodological claim of EvoSyn is that it can expand the set of controllable synthesis factors, rather than merely reallocating generation effort within an existing task space. The authors provide useful component ablations, including removing learner feedback, the Reflector, and the Designer. However, these experiments modify multiple aspects of the framework simultaneously and therefore do not fully isolate the benefits of synthesis-space expansion itself. For example, removing the Reflector affects the broader feedback interpretation and synthesis revision process, rather than exclusively disabling new-factor discovery. A more convincing evaluation would preserve the complete feedback-driven synthesis pipeline while disabling only the introduction of new synthesis factors. Furthermore, the main discovered factors, such as the numbers of read and write operations, are relatively straightforward properties of tool-use trajectories. It remains unclear whether explicitly adding such factors provides substantial benefits over simply modifying generation prompts based on the same learner feedback. This makes it difficult to determine whether the improvements arise from the proposed synthesis-space expansion mechanism or from more general feedback-driven curriculum adaptation. W5. The computational overhead and practical efficiency of the complex multi-agent synthesis framework are insufficiently characterized. EvoSyn employs multiple specialized agents, strong synthesis LLMs, environment verification, trajectory repair, and repeated rounds of downstream model training. Although the authors control candidate generation budgets in several comparisons, the total computational cost may differ substantially across methods due to variations in LLM calls, generated tokens, repair attempts, accepted trajectories, and downstream training data volume. The paper provides extensive analyses of synthetic data diversity and downstream performance, but the additional complexity of the proposed multi-agent framework is not sufficiently evaluated in terms of end-to-end computational efficiency. A more comprehensive quality–cost comparison would help establish whether the performance improvements justify the additional system complexity compared with simpler RSI or learner-adaptive synthesis pipelines.

Questions for authors
  1. Methodological novelty: Beyond integrating existing multi-agent synthesis, learner feedback, and iterative training mechanisms, what are the fundamental methodological innovations introduced by EvoSyn? Which components provide capabilities that cannot be achieved through simpler feedback-driven synthesis pipelines?

  2. Difference from existing RSI methods: How does EvoSyn fundamentally differ from existing recursive self-improvement and agent–data co-evolution approaches, particularly CoEvolve? What additional capability is enabled by explicitly expanding the synthesis ontology rather than dynamically modifying generation prompts or task constraints based on learner feedback?

  3. Comparison with RSI baselines: Why does the paper not include direct experimental comparisons with representative RSI frameworks such as CoEvolve, Agent-World, and SEAD? Can the authors evaluate these approaches under comparable synthesis and downstream training budgets?

  4. Contribution of synthesis-space expansion: Can the authors provide an ablation that retains learner feedback, the Reflector, Designer, and within-space adaptation, but disables only the introduction of new synthesis factors? How does this compare with the complete EvoSyn framework?

  5. Autonomous factor discovery: Given that the Reflector can receive diagnostic reports containing suggested factor names and values, to what extent does the framework autonomously discover previously unrepresented synthesis dimensions? Can the authors evaluate factor discovery without these explicit suggestions?

8 comments
AuthorsOpenReview ↗

Response to Reviewer MZjQ

We sincerely thank you for your careful and insightful review. We particularly appreciate your thoughtful comments regarding the connection to existing RSI and agent–data co-evolution research, which have helped us better clarify our methodological contributions and strengthen our experimental comparisons. We hope the following detailed responses, additional analyses, and new experiments adequately address your main concerns and further demonstrate the contributions of our work.


Reply to Weakness

Reply to W1 -- Methodological Novelty Beyond System Integration

Our methodological contribution is not a new neural optimizer or an additional agent role, but a feedback-to-decision mechanism that explicitly revises what the synthesis process can control, rather than merely changing which tasks it generates.

1. Technical distinction from existing adaptive synthesis.

EvoSyn maintains a revisable synthesis state Sr=(Or,r), where Or represents active controllable interaction factors and r determines their generation allocation. Unlike conventional adaptive synthesis, EvoSyn supports two distinct synthesis-level operations: reallocating effort over existing factors and promoting previously unrepresented, empirically supported factors into new generation controls.

ApproachPrimary Adaptation MechanismDistinction from EvoSyn
R-Zero / Socratic-ZeroLearner-feedback-driven problem generation and difficulty adaptationNo explicit revision of environment-grounded synthesis controls
AutoPlay / ExplorerEnvironment exploration and executable trajectory generationFocus on environment grounding rather than feedback-driven control-space revision
CoEvolve / Agent-WorldAdaptive task or environment generation through agent feedbackDo not explicitly demonstrate evidence-grounded promotion of missing interaction factors into reusable synthesis controls
EvoSynRevise both synthesis allocation and active controllable factorsFeedback-grounded factor promotion, dynamic introduction, and targeted generation

2. Controlled evidence isolating the proposed mechanism.

Beyond overall pipeline effectiveness, we provide complementary experiments that directly test the contributions of synthesis-space revision:

Technical QuestionExperimental EvidenceKey Result
Does empirical feedback determine factor selection?Feedback-Only vs. Suggestion-Only Reflector (new ablation)6/6 vs. 1/6 correct recoveries
Does dynamic factor introduction matter?Oracle-Full-Space + Adaptive Allocation (new baseline)72.6% vs. 69.3% final success
Are promoted factors operational generation controls?Requested vs. realized factor values (Appendix B.7)99.6–100% exact match
Are gains explained simply by more data or compute?Budget-Matched Static (new experiment)72.6% vs. 64.6%, despite Static processing 235.3M vs. 227.6M training tokens

Two findings are particularly important. First, even without suggested factor names and with anonymized probe identifiers, the Reflector consistently selects the appropriate factors from empirical diagnostics (6/6), whereas suggestions without feedback rarely suffice (1/6). Second, EvoSyn ultimately outperforms an Oracle baseline that receives all final synthesis factors from the first round and retains feedback-guided allocation. EvoSyn improves by +11.9 pp across rounds, compared with +5.3 pp for Oracle, supporting the value of introducing controls according to evolving diagnostic evidence rather than exposing all factors upfront.

Importantly, our factor-discovery mechanism operates over a library of measurable diagnostic probes; we do not claim unrestricted invention of arbitrary attributes. Its innovation lies in determining which diagnostic properties should become actionable synthesis controls, when they should be introduced, and how they guide subsequent experience generation.

Reply to W2 -- What Does EvoSyn Contribute Beyond Existing RSI Frameworks?

We also think that EvoSyn belongs to the broader RSI paradigm, and that feedback-driven task synthesis and iterative agent improvement are not new. Our contribution addresses a more specific question within RSI: how should diagnostic feedback change not only the distribution of generated experience, but also the set of interaction properties that the synthesis process can explicitly control?

1. Methodological distinction: From experience adaptation to synthesis-state revision.

Existing RSI frameworks already adapt training experience, but they differ in what their feedback mechanisms explicitly update:

FrameworkFeedback-Driven AdaptationEvoSyn's Additional Focus
CoEvolveUses forgetting and uncertainty signals to guide task re-explorationExplicitly revising reusable generation controls rather than only guiding exploration
Agent-WorldDiagnoses capability gaps to expand environments and generate targeted tasksRevising the active interaction-factor representation within an executable environment
SEADAdapts user profiles and training curriculaPromoting newly relevant interaction properties into controllable synthesis dimensions
EvoSynUpdates both synthesis allocation and active controllable factorsFeedback-to-decision mapping with measurable, executable synthesis controls

The key distinction is not whether an RSI system can generate new tasks or modify generation prompts. Rather, EvoSyn explicitly distinguishes two decisions that conventional feedback-guided resynthesis often leaves coupled:

  • Distribution adaptation: When existing controls are sufficient, redistribute synthesis effort through r.
  • Representation expansion: When existing controls cannot adequately express a diagnosed weakness, promote a measurable factor into Or+1, making it an explicit target for future generation.

Thus, EvoSyn makes the representation of controllable training experience itself a revisable synthesis-level state, rather than relying exclusively on changing task instances, exploration guidance, or curriculum priorities.

2. Why does this distinction matter empirically?

We provide complementary evidence beyond conceptual differences:

  • Feedback-grounded factor promotion: Without suggested factor names, the Reflector recovers the relevant factors in 6/6 trials, versus 1/6 with suggestions but no empirical evidence.
  • Operational realization: Newly promoted factors achieve 99.6–100% exact agreement between requested and realized values, confirming that they are actionable controls rather than descriptive labels.
  • Beyond a richer predefined space: Even an Oracle baseline given all final factors from R1 reaches only 69.3%, compared with 72.6% for EvoSyn, supporting the value of feedback-guided factor introduction.
  • Beyond conventional RSI adaptation: Under shared-R1, SFT-adapted comparisons, EvoSyn achieves +11.9 pp improvement across rounds, compared with +4.6 pp for CoEvolve-style and +7.9 pp for Agent-World-style synthesis (detailed in Reply to W3).

We emphasize that factor promotion operates over measurable diagnostic probes, rather than claiming unrestricted invention of arbitrary attributes.

EvoSyn does not propose a new RSI paradigm; it contributes a distinct mechanism within it: converting diagnostic evidence into explicit decisions about both the allocation and the active representation of controllable synthesis factors. The new oracle and RSI comparisons, together with factor-promotion and controllability analyses, demonstrate that this distinction is operationally meaningful and contributes to downstream learning.

Reply to W3 -- Direct Comparisons with Representative RSI and Agent–Data Co-Evolution Methods

We sincerely thank the reviewer for this important suggestion. We agree that comparisons with representative RSI methods are essential for evaluating EvoSyn's incremental contribution beyond conventional feedback-driven synthesis. To address this, we conducted two additional RSI baseline experiments, adapting the core synthesis mechanisms of CoEvolve and Agent-World to our CAR-Bench setting.

1. New experiments: Controlled comparisons with RSI approaches.

We implement two representative feedback-driven synthesis strategies:

  • CoEvolve-style: Uses agent feedback, including forgetting, capability-boundary, and rare-interaction signals, to guide subsequent task re-exploration and synthesis.
  • Agent-World-style: Uses capability diagnosis to dynamically revise task-generation guidance and synthesize experience targeting identified weaknesses.

Both are SFT-adapted variants, rather than reproductions of the original end-to-end RL frameworks, allowing us to focus on differences in synthesis strategies. Importantly, both baselines and EvoSyn share the identical R1 training data and task-agent checkpoint (60.7%), enabling a controlled comparison of subsequent feedback-driven synthesis.

MethodR1R2R3R1→R3 Gain
CoEvolve-style (SFT-adapted)60.7%66.7%65.3%+4.6 pp
Agent-World-style (SFT-adapted)60.7%64.7%68.6%+7.9 pp
EvoSyn60.7%65.5%72.6%+11.9 pp

2. Key findings: Stronger and more sustained improvement through synthesis-space evolution.

The results reveal three important observations:

  • RSI-style feedback is effective but not sufficient. Both baselines improve over the shared R1 checkpoint, confirming the value of feedback-guided resynthesis. However, EvoSyn ultimately outperforms CoEvolve-style by +7.3 pp and Agent-World-style by +4.0 pp.
  • EvoSyn achieves the strongest multi-round improvement: +11.9 pp, compared with +4.6 pp and +7.9 pp for the two RSI baselines.
  • The advantage emerges in later rounds. CoEvolve-style initially exceeds EvoSyn in R2 (66.7% vs. 65.5%) but declines slightly in R3. In contrast, EvoSyn improves by +7.1 pp from R2 to R3, consistent with its feedback-driven introduction of num_read to address newly identified read-intensive weaknesses.

3. Implications for the methodological contribution.

These comparisons complement our mechanism-specific evidence: unlike feedback-guided re-exploration or dynamic generation guidance alone, EvoSyn explicitly promotes diagnostic factors into reusable, controllable synthesis dimensions and updates generation allocation accordingly. The resulting stronger late-round improvement supports the value of synthesis-space revision as an additional mechanism within the broader RSI paradigm.

Reviewer_bSA3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how to continually synthesize useful training experience for interactive tool-using agents. Its motivation is that executable, successful trajectories are not necessarily informative for the current learner, while observed failures do not directly specify actionable generation decisions.

EvoSyn maintains a revisable synthesis state describing explicitly controllable interaction factors and the allocation of generation effort. Post-training performance, coverage, and failure diagnostics guide either reallocation over existing factors or the introduction of new controls. Specialized roles translate these decisions into task blueprints, trajectories, and environment-verified supervision for supervised fine-tuning.

Experiments cover three τ²-Bench domains and CAR-Bench Base. With Opus 4.5 synthesis and a Qwen3-8B learner, the reported final average success rate is 77.8%, exceeding the strongest synthesis-variant baseline by 13.3 percentage points. Additional experiments examine learner scales, synthesis backends, component ablations, and structural properties of the generated data.

Critical strengths

S1. A coherent control interface connects learner diagnosis to data generation

Sections 1 and 3 identify an important problem in agentic data synthesis: an executable, successful trajectory is not necessarily useful for the current learner. Sections 4.1–4.2 separate which interaction factors can be controlled from how generation effort is allocated over them. This gives failure diagnosis an explicit connection to subsequent generation decisions, rather than leaving it as a free-form error summary.

Section 4.3 connects this representation to task blueprints, trajectory generation, and environment verification. The resulting path from diagnosis to usable supervision is coherent and practically meaningful. The contribution is a synthesis interface through which feedback changes training experience, with a clear distinction between adapting existing controls and introducing new ones—not merely the addition of several agent roles.

S2. Main experiments support the effectiveness of the complete synthesis pipeline

Table 1 uses the same Qwen3-8B learner and includes synthesis variants with the same Opus 4.5 backend. EvoSyn achieves 77.8% average success versus 64.5% for the strongest variant, improving all four evaluation domains. This is the paper’s clearest empirical result and cannot simply be attributed to a stronger generator or a larger learner. Later-round improvements also occur with both open synthesis backends, supporting usefulness beyond one generator.

Sections 5.3–5.4 add meaningful supporting comparisons. Table 2 shows performance degradation when learner feedback, the Designer, or role specialization is removed. Figure 4 starts from the same first-round data and contrasts static accumulation with feedback-driven subsequent synthesis. Together, these results support the practical value of the complete pipeline and multi-round adaptation, rather than relying solely on an endpoint score.

Critical weaknesses

W1. Performance drops from R1 to R2 are not explained

Figure 5 shows that Communication and Multi-read success rates decrease by 23.8 and 12.8 percentage points from R1 to R2, respectively. Communication’s share of newly synthesized verified data increases by 6.2 percentage points, while Multi-read’s share decreases by 29.8 percentage points. The learner trains on accumulated verified data, but the paper does not explain how these changes in the new data relate to the decline in each task category, particularly Communication. Figure 10 shows later improvement in both categories but does not explain the earlier drops.

W2. Actual training budgets are not clearly reported

The paper does not report how many training tokens EvoSyn and Static actually process in Figure 4. Table 2 controls the number of candidate tasks, and Appendix B.1 specifies a common six-epoch training recipe, but equal epochs do not ensure equal training budgets when dataset sizes or trajectory lengths differ. Appendix B.9 notes that EvoSyn has more accepted examples in the pools used for structural analysis. Figure 12 matches sample counts to compare structural coverage, but does not report task success after training on those size-matched datasets. It is therefore unclear whether the performance comparison controls for the amount of training, and whether the gains come from better data or more training.

Questions for authors

Q1. What explains the R1-to-R2 regressions in Figure 5?

What explains the R1-to-R2 regressions in Figure 5, particularly the decline in Communication despite its increased share of newly synthesized verified data? Please report the numbers and proportions of Communication and Multi-read examples in both the newly added and cumulative training datasets, and explain how changes in this composition relate to the performance drops.

Q2. Does EvoSyn retain its performance advantage under matched training budgets?

For the EvoSyn–Static comparison in Figure 4, could the authors report round-wise and cumulative training tokens processed and test whether the success-rate advantage remains under a matched training-token budget? Please also report downstream task success after training on the size-matched datasets used for Figure 12, which currently compares structural coverage only.

6 comments
AuthorsOpenReview ↗

Response to Reviewer bSA3

We sincerely thank you for the careful review and insightful suggestions. Your constructive comments have helped us identify important areas where the experimental evidence and presentation could be strengthened. We hope the following clarifications, additional analyses, and new experiments adequately address your main concerns.


Reply to Weakness

Reply to W1 -- Explaining the R1-to-R2 Performance Regressions

We sincerely thank you for highlighting these non-monotonic changes. We conducted additional task-level, trajectory-level, and training-data analyses to understand the regressions. The findings reveal three important points.

1. The observed regressions are concentrated in small evaluation slices with substantial statistical uncertainty.

Communication and Multi-read contain only 7 and 13 evaluation tasks, respectively. Paired task-level bootstrap analysis shows that the 95% confidence intervals for both declines include zero. Averaging over available checkpoints further reduces the observed regressions:

SliceFigure 5 Change95% Bootstrap CICheckpoint-Averaged Change
Communication (7 tasks)−23.8 pp[−57.1, +9.5]−13.5 pp
Multi-read (13 tasks)−12.8 pp[−35.9, +7.7]−3.2 pp
Multi-write (14 tasks)+26.2 pp—+24.0 pp
High-write (6 tasks)+38.9 pp—+42.2 pp

Thus, while some small slices exhibit non-monotonic behavior, the improvements in the explicitly targeted write-intensive capabilities remain substantial.

2. Increased marginal data coverage does not imply coverage of the interaction structures required by evaluation.

Figure 5 reports a +6.2 pp increase in Communication within the verified synthesis pool. However, its share in the actual cumulative SFT data increases by only +2.9 pp (15.0% → 17.9%). More importantly:

  • 44 of the 139 new R2 Communication examples are simple direct-dial tasks requiring no contact lookup.
  • 4/7 Communication evaluation tasks require calendar-grounded interactions, but only 3/699 R2 SFT examples cover calendar-grounded Communication.
  • For Multi-read, the verified-pool share decreases from 35.3% to 5.6%, while its GT-aligned share in newly added SFT data decreases from 7.8% to 3.4%.

These findings explain why increasing a task category's overall frequency does not necessarily strengthen its underlying interaction capabilities. R2 primarily expands write-intensive experience, while complex read-dependent interactions remain insufficiently represented.

3. Trajectory inspection identifies specific failure mechanisms rather than broad tool-use degradation.

The Communication regression is primarily driven by two tasks:

  • base_37: In all 14 failing R2 runs, the agent executes the correct tool trajectory. The benchmark rejects the final response solely because the meeting time is expressed as "2 PM" instead of the required "14:00". This task also belongs to Multi-read, contributing to both reported declines.
  • base_61: The agent occasionally generates incorrect contact IDs instead of grounding them in calendar attendees. This specific cross-tool grounding pattern is absent from the SFT data.

Importantly, after R3 introduces num_read and increases coverage of read-intensive interactions, base_37 recovers in 8/10 checkpoints, while policy-only Communication failures decrease from 26% to 4% across the analyzed runs.

The additional evidence indicates that the apparent regressions reflect small-slice variability and specific underrepresented interaction behaviors, rather than a uniform deterioration of tool-use capability. These observations also reinforce EvoSyn's central motivation: learning utility depends on fine-grained interaction coverage, not merely the marginal frequency of task categories.

Reply to W2 -- Training-Budget Transparency and New Budget-Matched Experiments

We sincerely thank you for raising this important concern. We agree that matching candidate generation budgets and SFT epochs does not necessarily guarantee equal training exposure. To address this, we conducted a detailed training-budget audit and an additional Budget-Matched Static Synthesis experiment, directly evaluating downstream performance under comparable data and training-compute budgets.

1. Training-budget audit and controlled comparison.

Our audit confirms that the original Static baseline used fewer SFT examples (639 vs. 1,287) and processed fewer training tokens (106.8M vs. 227.6M) than EvoSyn. We therefore generated additional Static samples using the original fixed synthesis procedure, matching EvoSyn's final SFT dataset size and closely aligning input and supervised token volumes, without introducing learner feedback or synthesis-space evolution.

Importantly, the new Static baseline processes 235.3M training tokens, exceeding EvoSyn's 227.6M by 3.4%, measured across the complete six-epoch training runs.

MethodSFT ExamplesFull-Run Training TokensSuccess RateFactor CoverageTrajectory Diversity
Static (Original)639106.8M61.3%0.7870.750
Static (Budget-Matched)~1,287235.3M64.6%0.8120.784
EvoSyn1,287227.6M72.6%0.8930.938

2. EvoSyn retains a substantial advantage despite the increased Static training budget.

Increasing Static's data and training budget improves its success rate from 61.3% to 64.6% (+3.3 pp). Nevertheless, EvoSyn still outperforms Budget-Matched Static by 8.0 pp (72.6% vs. 64.6%), retaining approximately 71% of its original performance advantage, despite slightly lower aggregate training-token consumption.

Moreover, EvoSyn maintains substantially higher factor coverage (0.893 vs. 0.812) and trajectory diversity (0.938 vs. 0.784), indicating that additional static synthesis does not reproduce the breadth and diversity of interaction structures achieved through feedback-guided synthesis evolution.

These results demonstrate that EvoSyn's improvements cannot be explained solely by larger synthetic datasets or greater aggregate training compute. They provide stronger evidence that what experience is synthesized matters beyond how much experience is generated.

Reply to Question

Reply to Q1 -- Training-Data Composition and R1-to-R2 Regressions

We further audited the actual SFT training datasets, distinguishing newly added examples from cumulative training data. Importantly, Figure 5 reports proportions in the verified synthesis pool, which can differ from the trajectories ultimately used for SFT.

1. Detailed training-data composition.

MetricR1 NewR2 NewR1 CumulativeR2 Cumulative
Total SFT examples4886994881,187
Communication73 (15.0%)139 (19.9%)73 (15.0%)212 (17.9%)
Multi-read (GT-aligned)38 (7.8%)24 (3.4%)38 (7.8%)62 (5.2%)
Multi-read (teacher-executed)57 (11.7%)101 (14.4%)57 (11.7%)158 (13.3%)

The GT-aligned definition measures whether the task's ground-truth trajectory requires multiple reads, consistent with the evaluation slicing. The teacher-executed definition measures the actual tool calls in SFT trajectories. We report both to distinguish task requirements from realized supervision.

2. Why Communication declines despite increased data share.

Although Communication's cumulative SFT share increases from 15.0% to 17.9%, the new examples do not necessarily cover the interactions required by evaluation:

  • 44/139 new Communication examples are simple direct-dial tasks without contact lookup.
  • 4/7 Communication evaluation tasks require calendar-grounded interactions, but only 3/699 R2 SFT examples cover calendar-grounded Communication.
  • Communication's lookup-grounded proportion decreases from 99% in R1 to 68% in R2, indicating a shift toward simpler or different interaction patterns.

Task-level inspection further identifies two specific regressions: base_37 executes the correct tool trajectory but violates the required 24-hour response format, while base_61 incorrectly grounds contact IDs from calendar attendees, a pattern absent from the training data.

3. Why Multi-read declines.

For GT-aligned Multi-read tasks, the newly added SFT proportion decreases from 7.8% to 3.4%, and the cumulative proportion decreases from 7.8% to 5.2%, consistent with R2's synthesis emphasis on write-intensive interactions. However, teacher-executed read counts show a different trend, so reduced read-action exposure alone cannot establish the cause of the regression.

Moreover, both evaluation slices are small (7 Communication and 13 Multi-read tasks), with paired bootstrap 95% confidence intervals including zero for the reported declines. The Communication regression is largely driven by two specific tasks, including base_37, which also contributes to the Multi-read decline.

The additional analysis clarifies why marginal data proportions do not directly predict downstream success: the coverage of specific interaction requirements and the resulting learned behaviors matter more than task-category frequency alone.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.