acceptodds
Under review as a conference paper at ICLR 2027

ANDES: Agent-Native Data Evolving Synthesis via Feedback-Controlled Experience Acquisition

12 views

Abstract

AI agents are increasingly used to automate LLM post-training, yet acquiring targeted, high-quality training data remains a major bottleneck. Open-web curation requires long-horizon search, filtering, and balancing, while static synthesis pipelines cannot revise their acquisition distributions from intermediate outcomes. To address this gap, we formulate agentic data synthesis as feedback-controlled experience acquisition, in which observations from each synthesis round guide where subsequent supervision is acquired. We instantiate this formulation in Andes (Agent-Native Data Evolving Synthesis). Given a trainer-specified capability target, Andes maintains an acquisition distribution over a data-synthesis space modeled by a self-evolving World Tree. Diagnostics of effective quantity, topical allocation, and logical diversity then redirect subsequent acquisition toward under-covered capabilities while preserving broad contextual coverage. This synthesis-level feedback enables lightweight adaptation of the training signal before model-level training and evaluation. Across four base models under strict compute constraints, Andes substantially improves automated post-training, achieves performance comparable to frontier agents on PostTrainBench, and exhibits robust cross-task generalization. Our project is available at https://anonymous.4open.science/status/ANDES-4DAE.

Primary area:
foundation or frontier models, including LLMs
Keywords:
data synthesis, data-centric AI, LLM agents, post-training

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_p9S8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper formulates agentic data synthesis as feedback-controlled experience acquisition through ANDES (Agent-Native Data Evolving Synthesis). Given a trainer-specified capability deficit, ANDES uses an evolving World Tree of topics, themes, and scenarios to select contexts for refined supervision. The paper validates ANDES on PostTrainBench across multiple post-training base models and tasks, demonstrating that ANDES improves the automated post-training capability of GLM-4.7, achieving performance on PostTrainBench comparable to that of GPT-5.5 xHigh with Codex-based automated post-training and Claude Opus-4.8 Max with Claude Code-based automated post-training.

Critical strengths

Strength 1: Organizing data synthesis as a process that continually adapts to feedback has clear methodological value. ANDES connects task requirements, data generation, quality revision, and diagnostic feedback, allowing the training agent to adjust subsequent synthesis goals based on shortcomings in the existing data. This addresses a key need in automated post-training: deciding not only how to generate data, but also what data to generate next.

Strength 2: The world tree offers a concrete way to balance task relevance and data diversity. The Topic–Theme–Scenario hierarchy links broad task requirements to specific settings. Combined with task-aware routing and subtree expansion, it allows the method to focus on target capabilities while exploring varied contexts. Although its advantages over other organizational structures remain to be tested, it provides a clear and practical way to control the coverage of synthetic data.

Strength 3: The comparison under the same training scaffold provides direct evidence of effectiveness. With the same custom training scaffold, adding ANDES raises the overall PostTrainBench score from 21.56% to 34.40%. This comparison is more informative than a comparison with the OpenCode baseline alone, as it suggests that the gains do not simply come from improvements to the training framework. Ablations of the world tree and interactive feedback further support the contributions of these components.

Strength 4: The evaluation covers multiple target models and tasks. The paper evaluates four target base models on tasks including mathematical reasoning, code generation, and tool use, and reports results from repeated runs.

Overall, in the context of the current research trend toward recursive self-improvement, I find the core idea of this paper meaningful. Its essential contribution lies in progressively expanding the model's learning boundary according to its current capability level, enabling the model to acquire knowledge of previously unseen topics and tasks. This direction carries meaningful implications for the advancement of RSI research.

Critical weaknesses

My main concerns are as follows.

1. Unclear construction and generalization of the world tree.

The paper reports the size of the initial world tree, but does not explain how it was constructed, the extent of model or human involvement, or the associated costs. Moreover, subsequent evolution only expands Themes and Scenarios without updating top-level Topics, which may limit adaptability to new domains and cross-domain tasks. Given the apparent complexity of the world tree, would transferring the method to a new domain (such as finance) require rebuilding it from scratch?

2. Insufficient justification for using a tree structure.

The paper does not adequately explain why a tree structure is chosen to manage data evolution. Since different Topics and Scenarios may overlap, could this structure limit the representation of cross-domain tasks? What advantages does it offer over alternatives such as graph structures?

3. Unclear model capability boundaries and data filtering criteria.

The paper does not clearly define how the post-trained model's capability boundary is determined or how challenging but valuable samples are distinguished from invalid ones. Could the current filtering mechanism discard difficult samples that are essential for expanding the model's capabilities, thereby limiting further improvement? I suggest further justifying the filtering criteria and providing relevant ablation studies.

4. Insufficient separation between the benefits of external strong models and the method itself.

ANDES uses GPT-4o for routing, generation, revision, and evolving, while GLM-4.7 handles training decisions, whereas some baselines mainly rely on a single model. This comparison does not adequately control for the additional resources provided by external models, making it difficult to determine whether the gains come from ANDES itself or the capabilities of these stronger models. I suggest conducting fair comparisons under matched model resources and compute budgets, along with relevant ablation studies.

5. Insufficient comparisons with stronger baselines and Agent Harnesses.

The paper uses OpenCode as the primary baseline agent harness but lacks comparisons with newer frameworks, such as Claude Code and Codex, using the same underlying model. Moreover, the experiments do not adequately cover the latest generation of advanced models. I suggest comparing GLM-4.7 with Claude Code and Codex, evaluating ANDES with models such as GLM-5.2, GPT-6 Astra, and Claude Opus 5.5, and comparing the results against systems such as GPT-6 Astra with Codex and Claude Opus 5.5 with Claude Code. These experiments would help establish whether ANDES still provides meaningful gains with stronger models and Harnesses. In addition, recent work such as RSIBench-Data has proposed more general data evolution methods, as discussed in the survey Data-centric recursive improvement for foundation models: A survey. Furthermore, the models used for post-training in their experiments are relatively small-scale (1.7B-8B), and it remains uncertain whether the proposed approach generalizes effectively to larger-scale LLMs. The paper lacks systematic comparisons with these approaches, and I encourage the authors to add relevant experiments.

Questions for authors

See weakness.

7 comments
AuthorsOpenReview ↗

Response to Reviewer p9S8 (Part 1/4)

Thank you for recognizing the design and experimental value of ANDES for autonomous post-training. We clarify the World Tree's structure, reuse, and extensibility, and explain the roles of refinement and different model combinations.

W1: World Tree Construction and Adaptation with Fixed Topics

The initial World Tree is constructed through LLM-assisted top-down domain decomposition: Topics are subdivided into Themes and then into Scenarios for question generation. It is provided as a reusable label resource in the codebase and contains 72 Topics, 394 Themes, and 1,182 Scenarios. During synthesis, sampling weights are adjusted to the target, and scenario support is updated under repeatedly sampled Topics.

Topics provide a stable domain index, while the trainer's task description specifies the capability requirements and Themes/Scenarios supply finer-grained contexts. The initial tree is constructed once and reused across tasks. New tasks within existing domains can update lower-level contexts; a new top-level domain can be added as a Topic in the label configuration without rebuilding unrelated branches. We will distinguish initialization from online updates in the method description.

We compare the initial tree with the 36 named fields of the OECD FOS classification [1], using GPT-6.1 Sol to assess whether existing labels explicitly support each field or a constituent subfield.

Broad FieldNamed FieldsGPT-Confirmed FieldsLabel Coverage
Natural sciences66100.00%
Engineering and technology10770.00%
Medical and health sciences4375.00%
Agricultural sciences4375.00%
Social sciences8787.50%
Humanities44100.00%
Total363083.33%

The initial tree has direct label support in all six broad fields and covers 83.33% of the named fields, providing a broad contextual basis for synthesis.

We further test whether new domain content can be introduced by expanding Themes and Scenarios without changing Topics. The Evolver expands 11 fields; GPT assessment finds target-relevant scenarios for 9 (81.82%), while 2 remain closer to neighboring fields. All updates preserve the top-level Topics and unrelated branches. This shows that domain content can be extended locally without rebuilding the entire tree.

References

[1] OECD. Revised Field of Science and Technology (FOS) Classification in the Frascati Manual. 2007. DSTI/EAS/STP/NESTI(2006)19/FINAL, Annex 1. Official OECD document.

AuthorsOpenReview ↗

Response to Reviewer p9S8 (Part 2/4)

W2: Why a Tree Rather Than a Graph, and How Overlap Supports Cross-Domain Tasks

Graph-based synthesis provides a relevant alternative for organizing training-data generation. GraphGen [2] builds a knowledge graph from source text and uses neighborhood sampling to generate atomic, aggregated, and multi-hop QA pairs. CGMIS [3] traverses a concept graph to construct reasoning paths and synthesize verifiable multi-hop instructions. In these methods, graph relations and paths guide the content and reasoning dependencies of generated examples.

ANDES instead uses the World Tree to organize synthesis contexts and allocate the acquisition budget. The Topic-Theme-Scenario hierarchy directly matches its sampling and update rules: suitability routing adjusts Topic-level sampling weights, Themes and Scenarios are then sampled within the selected Topic, and dynamic node evolution updates the subtree of a repeatedly sampled Topic. The task description specifies the required capability, while the sampled scenario supplies the context.

This correspondence motivates the tree design: each acquisition follows an explicit path, budget allocation operates at the Topic level, and scenario updates remain local to the selected subtree. A graph-based acquisition design could additionally use cross-domain relations, with corresponding rules for sampling and updating connected nodes. The hierarchy provides the operations needed by the current acquisition loop; Condor [4] and WIST [5] likewise use hierarchical structures for data acquisition.

Semantic overlap between branches allows the same context to support different capabilities rather than requiring mutually exclusive domains. A financial scenario can involve probability, constrained optimization, or code implementation; a medical scenario can involve scientific reasoning, numerical analysis, or dialogue. During generation, the task description specifies the reasoning requirement and the scenario supplies its context. Their combination supports cross-domain supervision.

Tree paths therefore provide controllable acquisition entry points, while capability requirements combined with scenario contexts express cross-domain tasks. We will emphasize that the hierarchy does not require domains to be mutually exclusive.

References

[2] Zihong Chen et al. GraphGen: Enhancing Supervised Fine-Tuning for LLMs with Knowledge-Driven Synthetic Data Generation. 2025. arXiv:2505.20416.

[3] Zechen Sun et al. CGMIS: Concept-Graph Based Multi-Hop Instructions Synthesis for Enhancing Long-Context Reasoning. AAAI, 2026. DOI:10.1609/aaai.v40i39.40599.

[4] Maosong Cao et al. Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and Refinement. 2025. arXiv:2501.12273.

[5] Fangyuan Li et al. WIST: Web-Grounded Iterative Self-Play Tree for Domain-Targeted Reasoning Improvement. 2026. arXiv:2603.22352.

AuthorsOpenReview ↗

Response to Reviewer p9S8 (Part 3/4)

W3: Filtering Criterion and Difficulty Coverage

The filtering criterion concerns answer quality and repairability, not question difficulty or whether the student currently solves the question. The Refiner assigns an effort score from 1 to 5 and removes samples with ei>4e_i>4 before rewriting: these answers require complete replacement rather than a usable revision. Difficult questions with reliable or repairable answers remain eligible.

Difficulty diversity is handled during generation. For each sampled Theme, the prompt requests one Easy, one Medium, and one Hard question with distinct reasoning requirements. This encourages coverage across difficulty levels rather than restricting acquisition to questions that the student already solves.

To isolate screening, we disable only ei>4e_i>4 removal while retaining refinement, suitability routing, and report-driven updates, with the generation backend, retained-data budget, training configuration, and evaluation protocol matched.

MethodAIME 2025ArenaHard-WritingBFCLGPQA MainGSM8KHealthBenchHumanEvalWeighted AVG
GPT-6.1 Sol + ANDES/DS6.67 ±\pm 2.7242.54 ±\pm 0.9393.25 ±\pm 0.7632.59 ±\pm 1.6887.33 ±\pm 0.3934.99 ±\pm 0.7274.59 ±\pm 1.1242.16 ±\pm 0.75
w/o high-effort sample filtering6.67 ±\pm 2.2142.10 ±\pm 0.8893.50 ±\pm 0.9532.38 ±\pm 1.3686.96 ±\pm 0.4634.81 ±\pm 0.8174.39 ±\pm 1.2142.00 ±\pm 0.67

This control tests the effect of retaining answers that require complete replacement, separately from the effect of rewriting repairable answers. The screening mechanism targets unreliable supervision, while generation promotes difficulty diversity.

W4: Dependence on Internal Models and Use by Moderate-Capability Trainers

We distinguish the model roles: the trainer issues acquisition requests and executes training, while the internal backend performs routing, generation, refinement, evolution, and diagnostics. The original GLM-4.7 trainer and GPT-4o backend were chosen to examine whether an earlier-generation, moderate-capability combination could obtain competitive results through better data acquisition. The supplementary experiments use GPT-6.1 Sol with a DS backend to examine the same workflow with a stronger trainer.

We replace all internal modules from GPT-4o with Gemini-3.1-Flash on Qwen3-1.7B while keeping the trainer and training configuration fixed.

Internal BackendAIME 2025ArenaHard-WritingBFCLGPQA MainGSM8KHealthBenchHumanEvalWeighted Average
GPT-4o0.0014.2290.1628.4863.1927.0857.5131.41
Gemini-3.1-Flash0.0013.8492.0024.3764.3727.6757.8330.84

The weighted average changes by 0.57 points. BFCL, GSM8K, HealthBench, and HumanEval retain strong performance, showing that the acquisition workflow can operate with different internal backends.

In the original paper, GLM-4.7 + ANDES achieves an average weighted score of 34.40 across four student models, compared with 34.29 for GLM-5.2 (Max) and 34.08 for Opus-4.8 (Max). This shows that an earlier-generation trainer can obtain competitive performance through more effective acquisition. The backend substitution further demonstrates compatibility with different internal models.

Reviewer_Mm1FOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents ANDES, a feedback-driven data synthesis framework for automated LLM post-training. It uses a self-evolving World Tree, targeted routing, and report-based adjustments to generate training data. On PostTrainBench, ANDES improves the average score from 21.56% to 34.40% under the same trainer scaffold, with evaluations across four base models and seven benchmarks.

Critical strengths

1. Well-motivated framework. Adapting data acquisition based on synthesis feedback is a useful idea. The World Tree and report-driven workflow offer a practical implementation.

2. Broad evaluation. The improvements across models and tasks are encouraging. The data-quality, contamination, and held-out transfer analyses further strengthen the results.

Critical weaknesses

1. Novelty could be better established. The paper discusses related dynamic synthesis methods such as Middo and WIST but lacks direct comparisons, making the distinct contribution of ANDES less clear.

2. Component attribution remains somewhat unclear. Removing the World Tree also introduces a Condor-style synthesis pipeline, making its individual contribution difficult to isolate. Table 2 also does not report run-to-run variation for the ablation variants.

3. Limited comparison of synthesis effectiveness and cost. Table B.1 includes comparisons with DataFlow and Infinstruct, but stronger dynamic synthesis baselines are missing. Token usage and API costs are also not reported, making it difficult to assess the efficiency of ANDES relative to its additional synthesis overhead.

Questions for authors

Please refer to the weaknesses.

7 comments
AuthorsOpenReview ↗

Response to Reviewer Mm1F (Part 1/3)

Thank you for recognizing the practical value of treating data acquisition as an adjustable part of autonomous post-training. We clarify how this design differs from existing dynamic-data methods and examine its contribution under matched GPU and time constraints.

W1: Differences from Dynamic Synthesis and Autonomous Training Methods

ANDES makes data acquisition an object of iterative control by the trainer. The trainer specifies capability targets; the synthesizer returns training data and a report of coverage, effective sample count, and redundancy; and the trainer updates its next request. This loop operates before expensive model updates, allowing the agent to decide what the next batch should contain rather than fixing the entire dataset before training.

Unlike Middo's model-signal feedback [1] and WIST's web-grounded self-play [2], ANDES connects the trainer and synthesizer through synthesis-stage diagnostics, allowing requests to change before the next student-model update. Compared with complete autonomous research systems, ANDES is a data-acquisition tool that can be used by different trainers. Its contribution is the report-driven acquisition loop, supported by structured contexts, generation, and refinement.

We compare Middo, WIST, the GPT-6.1 baseline, and ANDES on Qwen3-4B-Base with a single GPU and a 10-hour-per-task limit.

MethodAIME 2025ArenaHard-WritingBFCLGPQA MainGSM8KHealthBenchHumanEvalWeighted Average
Middo [1]0.0020.9286.2726.5680.8912.1043.2928.68
WIST [2]0.0038.1293.3220.0981.0527.2078.6635.86
GPT-6.1 baseline6.6724.4994.2424.5584.0427.1571.9536.77
GLM-4.7 + ANDES (original paper)9.0051.9491.2231.4083.6331.0766.6241.21
GPT-6.1 Sol + ANDES/DS6.6742.5493.2532.5987.3334.9974.5942.16

ANDES achieves a weighted average of 42.16, improving over the baseline with the same trainer by 5.39 points. Data acquisition therefore remains a useful source of improvement with a stronger trainer.

We also compare the published PostTrainBench results of DataMaster [3], AutoTrainess [4], and ANDES with Qwen3-1.7B as the student model.

MethodAIME 2025ArenaHard-WritingBFCLGPQA MainGSM8KHealthBenchHumanEvalWeighted Average
DataMaster (GLM-5) [3]3.3321.9334.2731.0249.4334.9343.2927.91
AutoTrainess (GPT-5.4 + Codex) [4]0.000.3290.0029.9154.7414.9540.8525.67
GLM-4.7 + ANDES0.0014.2290.1628.4863.1927.0857.5131.41

ANDES obtains a weighted average of 31.41, with 63.19 on mathematics and 57.51 on code generation. With the earlier-generation GLM-4.7 trainer, it exceeds DataMaster with GLM-5 and AutoTrainess with GPT-5.4 in weighted average. This highlights the role of data-acquisition design in providing effective supervision for smaller student models.

References

[1] Zinan Tang et al. Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learning. 2025. arXiv:2508.21589.

[2] Fangyuan Li et al. WIST: Web-Grounded Iterative Self-Play Tree for Domain-Targeted Reasoning Improvement. 2026. arXiv:2603.22352.

[3] Yaxin Du et al. DataMaster: Data-Centric Autonomous AI Research. 2026. arXiv:2605.10906v2.

[4] Zhaojian Yu et al. AutoTrainess: Teaching Language Models to Improve Language Models Autonomously. 2026. arXiv:2606.31551v1.

AuthorsOpenReview ↗

Response to Reviewer Mm1F (Part 2/3)

We rename the original w/o World Tree setting w/o Dynamic Tree Evolution to clarify that it retains static World Tree labels but removes suitability-based reweighting and dynamic node evolution (This is a naming error; in reality, data synthesis cannot be performed at all without a "world tree."), yielding static synthesis similar to Condor [5]. w/o Interaction retains the internal dynamic World Tree, fixes requests, and disables report-driven configuration adjustment; its other settings match the full system. The revision will align the names, implementation definitions, and descriptions so that the two comparisons examine the dynamic acquisition space and report-driven request updates, respectively.

We extend the component ablations to GPT-6.1 Sol + Qwen3-4B-Base, keeping the backend, refinement procedure, data amount, and training configuration fixed.

MethodAIME 2025ArenaHard-WritingBFCLGPQA MainGSM8KHealthBenchHumanEvalWeighted Average
Qwen3-4B-Base3.333.420.0013.3941.8513.3836.5914.33
ANDES w/o Dynamic Tree Evolution3.33 ±\pm 3.8533.12 ±\pm 1.2790.25 ±\pm 0.9428.13 ±\pm 1.4282.64 ±\pm 0.8330.78 ±\pm 1.0670.12 ±\pm 1.6137.64 ±\pm 0.98
ANDES w/o Interaction6.67 ±\pm 1.9236.18 ±\pm 0.5890.75 ±\pm 1.0929.91 ±\pm 1.1785.08 ±\pm 0.6431.74 ±\pm 0.8971.95 ±\pm 1.3539.71 ±\pm 0.56
GPT-6.1 Sol + ANDES/DS6.67 ±\pm 2.7242.54 ±\pm 0.9393.25 ±\pm 0.7632.59 ±\pm 1.6887.33 ±\pm 0.3934.99 ±\pm 0.7274.59 ±\pm 1.1242.16 ±\pm 0.75

The full system improves over the two ablations by 4.52 and 2.45 points. Static labels still support synthesis, but removing target-driven reweighting and scenario evolution reduces its benefit. Retaining the dynamic World Tree while fixing requests also prevents the trainer from fully using each round's diagnostics.

We compare matched reports with reports from other previously generated batches on three tasks, holding training conditions fixed.

BenchmarkMatched ReportsMismatched ReportsGain from Matched Reports
GSM8K87.33 ±\pm 0.3985.46 ±\pm 0.33+1.87
HumanEval74.59 ±\pm 1.1271.34 ±\pm 0.86+3.25
BFCL93.25 ±\pm 0.7689.95 ±\pm 0.43+3.30

Each condition is trained independently with the same two seeds; we report the mean and sample standard deviation.

Report mismatch reduces performance on all three tasks. Both conditions receive report text, so the improvement depends on the report's association with the current data state, which lets the trainer adjust the capability focus and subsequent acquisition direction. The comparison distinguishes useful feedback from merely adding a report module.

We also compare Raw answers, Refined answers, and Refined answers with 20% corrupted supervision in mathematics, code, and writing, holding questions and training budgets fixed.

BenchmarkRawRefinedRefined + 20% Corrupted Supervision
GSM8K83.68 ±\pm 0.3787.33 ±\pm 0.3985.31 ±\pm 0.32
HumanEval70.71 ±\pm 2.0074.59 ±\pm 1.1271.73 ±\pm 0.86
ArenaHard-Writing39.30 ±\pm 0.5442.54 ±\pm 0.9342.15 ±\pm 0.45

Each condition is trained independently with the same two seeds; we report the mean and sample standard deviation.

Refinement improves performance by 3.65, 3.88, and 3.24 points, respectively, while corrupted supervision reduces these benefits. These training comparisons and the paired data-quality validation explain the contribution of refinement.

References

[5] Maosong Cao et al. Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and Refinement. 2025. arXiv:2501.12273.

AuthorsOpenReview ↗

Response to Reviewer Mm1F (Part 3/3)

W2 (continued): Repeated Qwen3-1.7B Ablations

We additionally summarize repeated runs of the Qwen3-1.7B ablation settings from Table 2, reporting task-level means and standard deviations.

MethodAIME 2025ArenaHard-WritingBFCLGPQA MainGSM8KHealthBenchHumanEvalWeighted Average
GLM-4.7 OpenCode0.00 ±\pm 0.000.87 ±\pm 0.060.00 ±\pm 0.0013.90 ±\pm 0.2312.82 ±\pm 0.227.40 ±\pm 0.2112.50 ±\pm 0.427.09 ±\pm 0.03
ANDES w/o Dynamic Tree Evolution0.00 ±\pm 0.005.20 ±\pm 0.3971.00 ±\pm 1.4121.96 ±\pm 0.3749.60 ±\pm 0.8426.30 ±\pm 0.6055.10 ±\pm 0.5526.03 ±\pm 0.08
ANDES w/o Interaction0.00 ±\pm 0.007.80 ±\pm 0.3168.50 ±\pm 0.7125.73 ±\pm 0.4559.39 ±\pm 0.6722.94 ±\pm 0.4649.70 ±\pm 0.4426.65 ±\pm 0.10
ANDES w/ Gemini Backend0.00 ±\pm 0.0013.57 ±\pm 0.3991.50 ±\pm 0.7124.63 ±\pm 0.3663.99 ±\pm 0.5427.36 ±\pm 0.4558.19 ±\pm 0.5030.78 ±\pm 0.09
ANDES (GLM-4.7 trainer)0.00 ±\pm 0.0014.35 ±\pm 1.1789.93 ±\pm 1.4828.61 ±\pm 0.7563.38 ±\pm 1.1226.95 ±\pm 1.2557.72 ±\pm 1.7931.45 ±\pm 0.61

The full system obtains a weighted average of 31.45, above 26.03 and 26.65 for the two component ablations, consistent with the main comparison. Repeated statistics on Qwen3-1.7B and the Qwen3-4B comparisons show the gains across student-model sizes, while the paired-answer and report-mismatch experiments explain the contributing mechanisms.

W3: Resource Overhead

ANDES follows a data-synthesis route that lets the agent actively construct and adjust supervision for its target capabilities, reducing dependence on existing web materials and their cleaning and adaptation. Generation, refinement, and diagnostics consequently incur additional API costs.

We report the main-agent and external synthesis API costs separately, averaged across the seven tasks.

MethodMain-Agent Cost per Task (USD)External Synthesis Cost per Task (USD)API Subtotal per Task (USD)
Middo17.940.0017.94
WIST22.200.0022.20
GPT-6.1 baseline17.140.0017.14
GPT-6.1 Sol + ANDES24.8133.8458.65

All methods use a single GPU and a wall-clock limit of at most 10 hours per task. ANDES's additional expenditure mainly comes from external synthesis; its main-agent cost exceeds the baseline by $7.67. Local generation in Middo, WIST, and the baseline still consumes GPU resources. Under the same GPU and time constraints, ANDES improves the weighted average over the baseline by 5.39 points and also exceeds Middo and WIST. The expenditure supports more effective supervision and gives the agent direct control over data acquisition.

Thank you for these suggestions. The additional comparisons clarify ANDES's contribution and provide evidence across student models, component ablations, and controlled training conditions.

Reviewer_fW2JOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Summary: The ANDES (Agent-Native Data Generation for Post-Training) is a novel approach to generating synthetic data for post-training of autonomous language models using feedback-driven experience acquisition. Unlike traditional methods which use fixed data generation pipelines, ANDES uses a dynamic feedback loop to adjust the data acquisition process based upon the desired capability and the feedback received from the generation process. The ANDES architecture includes three main components: a self-organizing "World Tree" which organizes the various contexts used in the generation process into topic/theme/scenario; a routing algorithm which determines where to allocate samples within the World Tree; and a two-step generation/refinement process which generates both training data and diagnostic reports. The diagnostic reports are used to inform future data acquisition decisions in order to maximize relevance to the target, coverage of the context, and logical diversity. The authors evaluated ANDES using four different base models and seven PostTrainBench tasks. They found that ANDES provided significantly better results than a trainer-only scaffold and performed at a level similar to several other state-of-the-art post-training agents. In addition to evaluating ANDES on these tasks, the authors conducted additional experiments to study the effects of each component of the architecture, different types of generation backends, data quality, data contamination, and generalization to unseen tasks.

Contributions: 1. The paper develops a feedback-driven data acquisition framework for synthetic data generation using a diagnostic approach at the synthesis level to inform future data generation requests without needing to update the model in between. 2. ANDES is based on a hierarchical synthesis space with weighted suitability routing and dynamic node expansion. The purpose of this mechanism is to focus generation on target-relevant capabilities while providing a diverse set of contexts. 3. The framework is an automated post-training workflow that integrates question-answer generation, LLM-based quality refinement, diversity diagnostics and report-driven acquisition adjustments. 4. Authors evaluate the framework on four base models and seven benchmarks, along with component ablations, backend replacement, data-quality validation, contamination analysis and held-out generalization experiments.

Validation: The primary experimental evaluations were performed using PostTrainBench, which evaluated four base models and seven downstream benchmarks using a single A100 GPU for 10 hours. All experiments were performed four times per setting. The authors reported an average score of 34.40% for ANDES, while the same trainer scaffold without ANDES resulted in a score of 21.56%. This represents a 12.84 percentage-point improvement. In addition to achieving performance comparable to several state-of-the-art post-training agents such as GLM-5.2 and Claude Opus-4.8, the authors note that these comparisons do not represent equivalent total computational or API costs. Additionally, the component ablations provided further evidence of the effectiveness of the framework. For Qwen3-1.7B, the full system achieved a score of 31.41% compared to 25.97% when the World Tree was removed and 26.58% when report-driven interactions were removed. When the internal synthesis backend was replaced with Gemini-3.1-Flash, a comparable score of 30.84% was achieved, demonstrating some degree of robustness to the selection of generation model. The reported four-run standard deviations were also relatively small. Furthermore, the paper included deterministic checks for selected synthetic data, contamination analysis, and transfer experiments on SVAMP and ASDiv-A. These results support the use of generated supervision beyond the exact target evaluation questions. However, the evidence for generalization is relatively limited, and the multi-target evaluation demonstrated only a modest aggregate improvement over the base model, with regressions on several individual benchmarks.

Critical strengths
  1. The problem addressed in this paper is a major bottleneck in automated LLM post-training; namely, how to obtain relevant and high-quality supervision with limited resources. Data collection before costly training updates is a valuable approach with clear practical motivation.

  2. Significant improvements using a matched trainer scaffold. The 12.84-point improvement over the scaffold-only setting is a strong empirical demonstration of the advantages of adaptive synthesis. The evaluation of multiple base-model families and various downstream tasks demonstrates that the benefits are not limited to a single model or benchmark.

  3. The use of hierarchical routing, node evolution, data refinement, and diagnostic reporting forms a cohesive implementation of adaptive synthesis. The modular architecture also allows for component-level studies and integration into existing training workflows.

  4. The paper includes component ablations, backend replacements, repeated runs, contamination checks, and held-out evaluations. These additional studies provide evidence beyond just reporting aggregate benchmark improvements and demonstrate the reliability of the framework.

Critical weaknesses
  1. The full system includes multiple components (structured routing, LLM-based generation, refinement, diversity diagnostics, and report-driven updates) that are introduced simultaneously. While the ablations are helpful, they are primarily demonstrated on one base model and do not fully isolate each mechanism. A matched static synthesis baseline using the same generation backend, refinement procedure, data budget, and training configuration would better establish the benefit of adaptive feedback itself.

  2. ANDES uses external GPT-4o calls for routing, generation, refinement, evolution, and diversity assessment. The single-GPU training constraint reported in the paper does not by itself establish comparable total inference costs across systems. Without accounting for external API usage, synthesis time, and additional model inference, the claim that ANDES achieves performance comparable to frontier agents is difficult to interpret as a resource-matched comparison.

  3. The paper discusses existing closed-loop and dynamically optimized synthesis methods, including Middo and WIST, but does not directly compare against them under matched settings. Most experimental baselines instead evaluate static synthesis or variants of the proposed system. This makes it difficult to determine whether the main improvements arise from the proposed acquisition framework or from established benefits of adaptive data generation.

  4. The acquisition updates rely heavily on LLM-generated suitability judgments and diversity summaries, while report-conditioned retention uses coarse random-discard decisions. The paper does not sufficiently establish whether these diagnostics accurately identify deficient capabilities or genuinely redundant supervision. Furthermore, increasing the number of World Tree nodes and visualizing embedding distributions does not necessarily demonstrate greater semantic or reasoning diversity in the resulting training data.

  5. The deterministic validation checks only selected subsets and properties of generated examples. For instance, syntax and interface validity do not guarantee functional correctness, and executable validation covers only 166 HumanEval-related samples. The paper would benefit from more systematic semantic correctness assessments, independent verification, and analysis of how residual labeling errors affect downstream training.

Questions for authors
  1. Can the authors provide additional analysis of the individual effects of adaptive routing, node evolution, refinement, and feedback updates in an ablation study of World Tree and interactions, while holding other variables constant?

  2. While Appendix B.2 provides deterministic validation, can the authors determine the extent to which refinement improves semantic accuracy and evaluate the effect of residual labeling errors on downstream performance?

  3. How reliable are the LLM's suitability and diversity assessments? Can the authors validate these assessments' relationship to downstream performance and analyze the consequences of inaccurate feedback?

  4. How does ANDES compare to existing adaptive synthesis frameworks (e.g., Middo and WIST) when using identical generation and training budgets?

7 comments
AuthorsOpenReview ↗

Response to Reviewer fW2J (Part 1/5)

Thank you for recognizing the practical value of adapting data acquisition before expensive model updates. Your comments prompted us to clarify the component comparisons and connect the diagnostic analyses to downstream performance.

W1 and Q1: Matched Baselines and Component Contributions

The central role of ANDES is to let the trainer adjust the next acquisition request based on synthesis outcomes. We examine whether this report-driven adjustment improves performance over fixed requests when the generation backend, refinement procedure, and training conditions are matched.

To clarify the ablation scope, we rename the original w/o World Tree setting w/o Dynamic Tree Evolution (This is a naming error; in reality, data synthesis cannot be performed at all without a "world tree."). This setting retains the static World Tree labels but removes suitability-based reweighting and dynamic node evolution. w/o Interaction retains the internal dynamic World Tree, fixes acquisition requests, and disables report-driven configuration adjustment. Its generation backend, refinement procedure, data amount, and training configuration are otherwise identical to the full system. These comparisons examine the dynamic World Tree and report-driven request updates, respectively. We will update the names and definitions in the revised manuscript.

The four mechanisms act on sampling probabilities, scenario nodes, answers, and subsequent requests:

MechanismObject AdjustedComparison
Suitability routingSampling probabilities of existing Topicsw/o Dynamic Tree Evolution
Dynamic node evolutionTheme/Scenario subtrees of repeatedly sampled Topicsw/o Dynamic Tree Evolution
RefinementAnswer quality for the same questionsRaw/Refined paired validation and training comparison
Report-driven configuration adjustmentThe next acquisition requestw/o Interaction and mismatched reports

We extend the component ablations to Qwen3-4B-Base, holding the backend, refinement procedure, data amount, and training budget fixed, to examine whether the gains observed on Qwen3-1.7B extend to a larger student model.

MethodAIME 2025ArenaHard-WritingBFCLGPQA MainGSM8KHealthBenchHumanEvalWeighted Average
Qwen3-4B-Base3.333.420.0013.3941.8513.3836.5914.33
ANDES w/o Dynamic Tree Evolution3.33 ±\pm 3.8533.12 ±\pm 1.2790.25 ±\pm 0.9428.13 ±\pm 1.4282.64 ±\pm 0.8330.78 ±\pm 1.0670.12 ±\pm 1.6137.64 ±\pm 0.98
ANDES w/o Interaction6.67 ±\pm 1.9236.18 ±\pm 0.5890.75 ±\pm 1.0929.91 ±\pm 1.1785.08 ±\pm 0.6431.74 ±\pm 0.8971.95 ±\pm 1.3539.71 ±\pm 0.56
GPT-6.1 Sol + ANDES/DS6.67 ±\pm 2.7242.54 ±\pm 0.9393.25 ±\pm 0.7632.59 ±\pm 1.6887.33 ±\pm 0.3934.99 ±\pm 0.7274.59 ±\pm 1.1242.16 ±\pm 0.75

Full ANDES improves the weighted average by 2.45 points over w/o Interaction and 4.52 points over w/o Dynamic Tree Evolution. Fixing trainer requests reduces performance even when the internal dynamic World Tree is retained; retaining only static labels and sampling also falls short of the full acquisition process. The comparisons support the contributions of request updates and the dynamic World Tree.

AuthorsOpenReview ↗

Response to Reviewer fW2J (Part 2/5)

W1 and Q1 (continued): Report-Mismatch Comparison

We compare the main-experiment, which uses correctly matched synthesis reports, with the mismatched-report runs on GSM8K, HumanEval, and BFCL.

Mismatched reports are taken from other previously generated batches of the same task and substituted using a fixed permutation. For example, a batch that repeatedly uses proportional reasoning receives another batch's report about insufficient coverage of equation solving. The report text is unchanged; only its association with the current batch is altered.

BenchmarkMatched ReportsMismatched ReportsScore Difference
GSM8K87.33 ±\pm 0.3985.46 ±\pm 0.33+1.87
HumanEval74.59 ±\pm 1.1271.34 ±\pm 0.86+3.25
BFCL93.25 ±\pm 0.7689.95 ±\pm 0.43+3.30

The main-experiment exceeds the mismatched-report runs by 1.87, 3.25, and 3.30 points on the three tasks. Correctly associated feedback directs acquisition toward the current batch's reported coverage gaps, whereas mismatched feedback refers to issues from other batches. These score differences are consistent with the value of the report's correspondence to the current data.

Suitability routing reallocates sampling probabilities over existing regions, while dynamic node evolution updates their scenario support. We therefore examine them jointly in the dynamic World Tree ablation. Report-driven updates are examined through w/o Interaction and report mismatch; the paired quality and training comparisons for refinement appear in our Q2 response (Part 5).

W2: The Cost of External Synthesis APIs

We report main-agent and external synthesis costs separately, averaged across seven tasks. All four methods use GPT-6.1 Sol as the main agent; ANDES additionally uses DS for external synthesis.

The rates used for accounting are in USD per million tokens (M):

Model / RoleUncached InputCache ReadsOutput
GPT-6.1 Sol / Main agent2.000.1010.00
DS / External synthesis0.300.0061.20

The token columns below cover only the main agent and use GPT-6.1 Sol rates, not DS rates.

MethodUncached Input (M)Cached Input (M)Output (M)Main-Agent Cost (USD)External Synthesis Cost (USD)API Subtotal (USD)
Middo2.1297.480.4017.940.0017.94
WIST2.72124.970.4322.200.0022.20
GPT-6.1 baseline3.0373.870.3717.140.0017.14
GPT-6.1 Sol + ANDES3.08157.150.2924.8133.8458.65

For uncached input U, cache reads C, and output O in M, main-agent cost is 2.00*U + 0.10*C + 10.00*O; DS cost is 0.30*U + 0.006*C + 1.20*O. Cached input is excluded from U, and reasoning tokens are already included in O. We compute from raw usage and round after averaging, rather than from the rounded token columns. DS costs use a fixed conversion of 1 CNY = 0.15 USD.

Although API costs are higher, ANDES adopts a plug-and-play data-synthesis approach that gives the trainer direct control over generating, refining, and adaptively acquiring supervision for its target capabilities. It is worth noting that the additional API expenditure is primarily directed toward external synthesis rather than the orchestration of the primary agent; in practice, the refined training data generated through this process can be reused in subsequent, related model training.

AuthorsOpenReview ↗

Response to Reviewer fW2J (Part 3/5)

W3 and Q4: Comparisons with Middo, WIST, and Autonomous Training Methods

Middo [1] and WIST [2] differ from ANDES in their feedback sources and workflows: Middo uses model signals for dynamic data optimization, while WIST performs iterative self-play grounded in web materials. Their original evaluations use their own iterative training protocols rather than PostTrainBench's single-GPU, fixed-wall-clock, end-to-end agent protocol. This motivated adapting their mechanisms to a common budget rather than directly mixing their original scores with ours.

We adapt Middo and WIST to PostTrainBench v1 with one GPU and at most 10 hours per task, and compare them with the GPT-6.1 baseline and ANDES on Qwen3-4B-Base.

MethodAIME 2025ArenaHard-WritingBFCLGPQA MainGSM8KHealthBenchHumanEvalWeighted Average
Middo [1]0.0020.9286.2726.5680.8912.1043.2928.68
WIST [2]0.0038.1293.3220.0981.0527.2078.6635.86
GPT-6.1 baseline6.6724.4994.2424.5584.0427.1571.9536.77
GLM-4.7 + ANDES (original paper)9.0051.9491.2231.4083.6331.0766.6241.21
GPT-6.1 Sol + ANDES/DS6.6742.5493.2532.5987.3334.9974.5942.16

GPT-6.1 Sol + ANDES achieves a weighted average of 42.16, exceeding the GPT-6.1 baseline, WIST, and Middo by 5.39, 6.30, and 13.48 points, respectively. Relative to the baseline with the same GPT-6.1 trainer, ANDES improves writing, scientific QA, mathematics, medical dialogue, and code generation.

We also compare the published PostTrainBench results of DataMaster [3], AutoTrainess [4], and ANDES on Qwen3-1.7B-Base, recomputing their weighted averages with the same task weights.

MethodAIME 2025ArenaHard-WritingBFCLGPQA MainGSM8KHealthBenchHumanEvalWeighted Average
DataMaster (GLM-5) [3]3.3321.9334.2731.0249.4334.9343.2927.91
AutoTrainess (GPT-5.4 + Codex) [4]0.000.3290.0029.9154.7414.9540.8525.67
GLM-4.7 + ANDES0.0014.2290.1628.4863.1927.0857.5131.41

All three use the same 1.7B student model. ANDES uses GLM-4.7 as the trainer, while DataMaster and AutoTrainess use GLM-5 and GPT-5.4, respectively. Despite using an earlier-generation trainer, ANDES obtains a higher weighted average by 3.50 and 5.74 points. These results highlight the value of the data-acquisition workflow alongside improvements in trainer capability.

References

[1] Zinan Tang et al. Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learning. 2025. arXiv:2508.21589.

[2] Fangyuan Li et al. WIST: Web-Grounded Iterative Self-Play Tree for Domain-Targeted Reasoning Improvement. 2026. arXiv:2603.22352.

[3] Yaxin Du et al. DataMaster: Data-Centric Autonomous AI Research. 2026. arXiv:2605.10906v2.

[4] Zhaojian Yu et al. AutoTrainess: Teaching Language Models to Improve Language Models Autonomously. 2026. arXiv:2606.31551v1.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Final Summary of Contributions and Discussion

Dear Area Chair,

Thank you for coordinating the discussion. We briefly summarize our contribution and the additional evidence below.

Our paper presents ANDES, an agent-native, plug-and-play system that makes adaptive data acquisition a first-class capability of autonomous post-training agents. Rather than treating synthesis as fixed preprocessing, ANDES enables agents to actively steer supervision toward their training objectives across capability domains. It offers a practical approach to improving automated post-training through targeted, high-quality, and contextually diverse synthetic data.

Core contribution and reviewer recognition. Reviewers recognized targeted data acquisition as a major bottleneck and a key need in automated post-training: deciding not only how to generate data, but what data to generate next. ANDES addresses this through an agent-interaction-driven, plug-and-play synthesizer, combining a self-evolving World Tree with synthesis reports that guide subsequent acquisition requests—without intervening student-model updates. Across four student models and seven tasks, it improves the same-scaffold score from 21.56 to 34.40.

Evidence added during rebuttal. Related methods and stronger trainer: On Qwen3-4B under the same single-GPU, 10-hour protocol, ANDES scores 42.16, versus Middo 28.68, WIST 35.86, and the same-model, same-harness GPT baseline 36.77. Component contributions: Controlled ablations show gains of 4.52 points over static-tree synthesis and 2.45 points over fixed acquisition requests. Report-mismatch and Raw/Refined comparisons further support feedback and refinement. Reliability and transparency: We added diagnostic validation, clarified tree reuse and filtering, and reported token usage and API costs.

We appreciate the constructive reviews and will incorporate the additional evidence and clarifications into the manuscript.

The authors of submission #37716

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.