acceptodds
Under review as a conference paper at ICLR 2027

LLM Agents for Automated Post-Training via Structured Recipe Optimization

Abstract

Automated post-training transforms pre-trained large language models (LLMs) into models capable of handling diverse domains by automatically constructing and optimizing a recipe of coupled decisions over data sources, data processing, cross-domain composition, and training configuration under a limited experimental budget. Existing automation either searches predefined recipe spaces that enable systematic optimization but remain fixed rather than adapting their available choices to the target tasks, or uses open-ended agents that discover task-specific choices but leave them implicit in trial-level edits, limiting their systematic reuse and recombination across trials. To fill this gap, we introduce , a novel paradigm that can adapt the recipe space itself to the target tasks while representing its choices explicitly for systematic reuse, recombination, and joint optimization. We instantiate this paradigm with , comprising an LLM-based Recipe-Space Designer that constructs the task-adapted recipe space and a Multi-Domain Recipe Optimizer that uses structured trial evidence to guide LLM candidate construction and multi-objective Bayesian trial selection. Extensive experiments show that SRO achieves the highest average score in all evaluated target-model and agent-LLM combinations. In joint post-training across three capability domains and six benchmarks, SRO reaches for the strongest baseline on Qwen3-1.7B with GPT-5.4, and on Qwen3-4B with DeepSeek-V4-Flash.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Post-Training, Automated Post-Training, AI for AI (AI4AI), Long-Horizon LLM Agents

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_RLj5OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies automated post-training for large language models under fixed compute budgets without human-curated training data. The authors argue that predefined recipe spaces (e.g., AutoPipe) cannot discover new data, while open-ended code-editing agents (e.g., PostTrainBench, AutoTrainess) suffer from low sample efficiency and execution crashes. The authors propose Structured Recipe Optimization (SRO), which operates over a task-adapted recipe space. The Recipe-Space Designer guides an agent to define discrete choices across data sources, synthesis templates, mixtures, and hyperparameters. The Multi-Domain Recipe Optimizer uses domain-structured trial memory and Random Forest surrogates with Expected Hypervolume Improvement (EHVI) to select candidate recipes. The method is evaluated on Qwen3-1.7B-Base and Qwen3-4B-Base across Math, Code, and Science benchmarks under fixed wall-clock time.

Critical strengths

The paper proposes a well-engineered system architecture with several practical strengths:

  1. Practical system workflow: Constraining agent search to discrete recipe choices with an execution workflow prevents the script syntax errors and runtime crashes common in open-ended coding agents (Section 3.1, Figure 2).

  2. Transparent reporting of failure cases: The appendix documents optimization trajectories and acknowledges notable failure modes, including severe performance drops on AIME and BFCL regressions (Appendix D.3, Appendix D.4).

Critical weaknesses

However, several experimental and methodological aspects require further clarification:

  1. Absence of statistical replication: All results (Tables 1–3) are single runs without random seeds, standard deviations, or significance tests. On small test sets like AIME 2025 (30 problems, 3.3% per problem), performance deltas fall within sampling noise.

  2. Search-time evaluation overlap: Each trial evaluates on 50% of the target benchmark test samples to select the best checkpoint (Appendix B.1), which is then reported on the full test set. This creates data overlap between search-time selection and final reporting.

  3. Severe surrogate sparsity and unstable BO selection: Multi-domain runs complete only 8 to 17 total trials (Table 10). After 5 warm-up trials, the Random Forest surrogate fits on just 3 to 12 data points. Removing EHVI in Table 3 actually improves scores on AIME (10.0 vs 3.3) and GPQA (36.4 vs 30.8).

  4. Confounded ablations: The ablation "w/o structured recipe search" removes both the recipe space and the execution workflow (Table 3), attributing general execution stability to the search algorithm.

  5. Severe negative transfer in the Science domain: On Qwen3-1.7B (Table 2), multi-domain GPQA drops from 34.2 (single-domain, Table 1) to 16.1 (barely above the 14.1 base model), lagging far behind AutoTrainess (30.8).

Questions for authors

Please refer to the detailed concerns and clarification points listed in the Critical Weaknesses section above.

7 comments

We sincerely thank the reviewer for the thorough examination of our methodology and results. We appreciate your suggestions on experimental reliability, search design, and cross-domain trade-offs, which have guided additional validation and clearer explanations. We address each point below.

W1: Statistical replication and small-test-set variability. Thank you for highlighting statistical replication and variability on small test sets. Each experiment in the submission was evaluated with one independent run, given the GPU time and API costs of complete searches. We have now conducted additional independent full search runs for every method and ablation configuration listed below, reporting mean ± sample standard deviation over three independent runs. Given the rebuttal time constraints and computational budget, these experiments cover both model sizes, all three methods, and all four ablation variants using DeepSeek-V4-Flash.

These repeated runs follow the search and final-evaluation protocols used in the submission and are reported separately from the additional 50/50 split experiments. All scores are percentages. Avg. is the unweighted mean across the six benchmarks.

Repeated-run main results. SRO achieves the highest mean aggregate score at both model sizes, with lower aggregate run-to-run variability than both baselines. It exceeds the strongest baseline at each model size by 6.61 percentage points on 1.7B and 6.98 points on 4B. Its aggregate standard deviations are 1.20 and 2.02, compared with 6.78–6.93 and 3.39–4.99 for the baselines.

Repeated Runs — Main Results

ModelMethodAIMEGSM8KBFCLHumanEvalGPQAHealthBenchAvg.
Qwen3-1.7BPostTrainBench0.00 ± 0.0056.33 ± 17.7389.00 ± 7.0046.53 ± 15.7924.40 ± 5.4713.33 ± 9.1738.27 ± 6.78
Qwen3-1.7BAutoTrainess0.00 ± 0.0060.33 ± 11.7988.00 ± 4.5836.77 ± 18.8426.70 ± 3.8411.07 ± 4.0337.14 ± 6.93
Qwen3-1.7BSRO (Ours)2.23 ± 3.8768.03 ± 10.2692.33 ± 1.1558.30 ± 0.3524.87 ± 7.6223.53 ± 1.2744.88 ± 1.20
Qwen3-4BPostTrainBench2.23 ± 3.8770.47 ± 7.4891.67 ± 3.2152.83 ± 13.9432.20 ± 1.2517.33 ± 3.6544.46 ± 4.99
Qwen3-4BAutoTrainess0.00 ± 0.0065.07 ± 8.7092.00 ± 3.4648.60 ± 5.6730.67 ± 3.4215.57 ± 0.2141.98 ± 3.39
Qwen3-4BSRO (Ours)1.10 ± 1.9183.17 ± 3.8793.67 ± 1.5373.37 ± 9.2432.87 ± 2.0024.50 ± 4.0751.44 ± 2.02

Repeated-run ablation results. Full SRO achieves the highest mean aggregate score at both model sizes, exceeding the four ablated variants by 2.36–10.66 percentage points on 1.7B and 2.06–7.73 points on 4B. The table below reports means and standard deviations for every benchmark, showing both average gains and run-to-run variability.

Repeated Runs — Ablation Results

ModelVariantAIMEGSM8KBFCLHumanEvalGPQAHealthBenchAvg.
Qwen3-1.7BFull SRO2.23 ± 3.8768.03 ± 10.2692.33 ± 1.1558.30 ± 0.3524.87 ± 7.6223.53 ± 1.2744.88 ± 1.20
Qwen3-1.7Bw/o structured recipe search0.00 ± 0.0052.84 ± 4.9885.33 ± 4.5131.69 ± 7.6125.66 ± 2.279.88 ± 2.4434.22 ± 2.61
Qwen3-1.7Bw/o LLM proposer3.34 ± 3.3570.96 ± 2.9594.00 ± 1.0051.43 ± 3.9723.28 ± 6.5612.14 ± 1.5842.52 ± 1.98
Qwen3-1.7Bw/o EHVI selection0.00 ± 0.0058.44 ± 2.9493.00 ± 1.0043.69 ± 6.8430.81 ± 1.0216.07 ± 2.2540.35 ± 0.72
Qwen3-1.7Bw/o structured memory1.10 ± 1.9164.08 ± 10.1491.67 ± 4.0450.22 ± 8.4423.36 ± 7.0518.67 ± 5.2841.53 ± 1.62
Qwen3-4BFull SRO1.10 ± 1.9183.17 ± 3.8793.67 ± 1.5373.37 ± 9.2432.87 ± 2.0024.50 ± 4.0751.44 ± 2.02
Qwen3-4Bw/o structured recipe search3.34 ± 3.3569.02 ± 7.6989.00 ± 4.5856.50 ± 9.8931.64 ± 1.4712.81 ± 5.8143.71 ± 3.75
Qwen3-4Bw/o LLM proposer2.21 ± 1.9185.48 ± 0.9894.33 ± 1.1563.43 ± 8.8830.87 ± 7.4918.46 ± 3.7249.12 ± 1.64
Qwen3-4Bw/o EHVI selection4.44 ± 5.0981.96 ± 5.6493.00 ± 0.0059.75 ± 8.5335.87 ± 0.4716.38 ± 2.2148.56 ± 2.93
Qwen3-4Bw/o structured memory1.11 ± 1.9279.56 ± 2.1493.33 ± 1.5366.48 ± 1.0433.47 ± 2.7022.40 ± 6.4849.38 ± 1.05

Regarding AIME 2025’s small test set and sampling noise: with 30 problems, each problem corresponds to approximately 3.3 percentage points in a single evaluation. We therefore do not treat changes of one or two solved problems as evidence of statistically significant improvement. The repeated experiments measure variability across independent search runs; our aggregate conclusion rests on the six-benchmark average at both model sizes, rather than small differences on AIME alone.

W2: Search-time evaluation overlap. Thank you for highlighting search-time evaluation overlap. We conducted additional strict 50/50 split experiments that separate search-time selection from final reporting: all methods use the same first 50% of examples, fixed across trials, for search and checkpoint selection, while the remaining 50% is reserved exclusively for final evaluation.

The original experiments follow PostTrainBench’s protocol, which permits querying the target benchmark for evaluation during iteration but prohibits training on its test questions or answers. SRO uses the fixed first 50% during search (50 examples for HealthBench-Easy), receiving only scores without inspecting individual evaluation examples or their errors; the submitted results were evaluated on the full benchmarks. The additional experiments assess performance when final evaluation examples are entirely excluded from search and checkpoint selection.

Given the rebuttal time constraints and computational budget, these experiments cover both model sizes and all three methods using DeepSeek-V4-Flash. The two tables below report each method’s same final selected checkpoint on the search and held-out halves, respectively.

50/50 Split — Search-Set Results

ModelMethodAIMEGSM8KBFCLHumanEvalGPQAHealthBenchMean
Qwen3-1.7BPostTrainBench0.00000.43250.44000.42680.26340.09670.2766
Qwen3-1.7BAutoTrainess0.00000.54480.94000.65850.31700.10440.4274
Qwen3-1.7BSRO (Ours)0.00000.71020.94000.62200.28570.13650.4491
Qwen3-4BPostTrainBench0.06670.55990.92000.69510.29910.18610.4545
Qwen3-4BAutoTrainess0.13330.62520.96000.62200.32590.18480.4752
Qwen3-4BSRO (Ours)0.00000.81340.98000.67070.35270.27200.5148

50/50 Split — Held-Out Results

ModelMethodAIMEGSM8KBFCLHumanEvalGPQAHealthBenchMean
Qwen3-1.7BPostTrainBench0.00000.38940.40000.20730.33040.06990.2328
Qwen3-1.7BAutoTrainess0.00000.52730.90000.24390.25890.09130.3369
Qwen3-1.7BSRO (Ours)0.00000.57730.90000.29270.26790.11670.3591
Qwen3-4BPostTrainBench0.00000.66520.78000.39020.32140.15150.3847
Qwen3-4BAutoTrainess0.06670.65910.86000.21950.29020.14540.3735
Qwen3-4BSRO (Ours)0.00000.83180.92000.40240.30800.20940.4453

With no example overlap between search-time selection and final held-out evaluation, SRO achieves the highest mean score at both model sizes, exceeding the strongest baseline by 2.22 percentage points on 1.7B and 6.06 points on 4B. These results show that SRO’s aggregate advantage persists in this setting when search-feedback examples are excluded from final scoring.

W3: Surrogate sparsity and BO selection stability. Thank you for highlighting surrogate sparsity and unstable BO selection. We first clarify that the five warm-up trials are included in surrogate training, and each subsequent fit uses all valid observations available at that point. Thus, 3–12 refers to the number of post-warm-up trials, not the surrogate’s training sample count.

SRO does not rely solely on BO to search the entire recipe space. It separates candidate construction from trial selection: the LLM proposes candidates using the available choices, task knowledge, and structured trial history; EHVI then selects among them using predicted cross-domain outcomes and uncertainty. These components are complementary: the LLM supplies prior knowledge and contextual reasoning, while EHVI uses accumulated observations to balance exploration and exploitation.

Regarding the higher AIME and GPQA scores after removing EHVI: in the same 4B ablation, AIME increases from 3.3 to 10.0 and GPQA from 30.8 to 36.4, but HealthBench decreases from 26.9 to 14.2 and the six-benchmark average decreases from 53.8 to 51.9. This reflects cross-domain trade-offs; EHVI does not guarantee improvement over the ablated variant on every benchmark.

To further address run-to-run variability in BO selection, we conducted two additional independent full search runs. Across three runs, full SRO retains a higher mean aggregate score than w/o EHVI:

ModelFull SROw/o EHVI selection
Qwen3-1.7B44.88 ± 1.2040.35 ± 0.72
Qwen3-4B51.44 ± 2.0248.56 ± 2.93

Results are the mean ± sample standard deviation of the six-benchmark average (%) using DeepSeek-V4-Flash. Full SRO improves the mean aggregate score by 4.53 percentage points on 1.7B and 2.88 points on 4B, showing that its aggregate advantage persists across independent repeats rather than appearing only in the original single run.

W4: Separating structured search from execution support. Thank you for suggesting a clearer separation between the contributions of the execution workflow and the search algorithm. The original w/o structured recipe search ablation compares two end-to-end optimization approaches: SRO integrates recipe selection with its executable workflow, whereas an open-ended agent organizes experiments and execution itself. Accordingly, that ablation removes both the structured recipe space and its associated workflow. To further distinguish their contributions, we added w/o structured recipe search (shared workflow), retaining the same executable workflow, data validation, data-generation tools, and 20-hour runtime limit.

The control reuses SRO’s discovered data sources, augmentation operators, and mixture and training choices, together with the same validation, caching, training, and evaluation pipeline. The distinction is how the methods search over these same options: SRO constructs, recombines, and selects candidate recipes using domain-specific evidence; the control receives resource descriptions and directly chooses its next configuration based on its own trial history, without SRO’s structured candidate construction, candidate batches/pool, explicit cross-domain recombination, or EHVI selection. Both methods select the final checkpoint by the unweighted six-benchmark average.

Given the rebuttal time constraints and computational budget, the currently completed shared-workflow controls comprise one run per model size using DeepSeek-V4-Flash. We compare these with SRO’s mean ± sample standard deviation over three independent full search runs, including the original run.

Shared-Workflow Control — Results

Scores are percentages. Control denotes w/o structured recipe search (shared workflow). SRO entries report three-run means ± sample standard deviations; Control entries report single-run scores.

ModelMethodAIMEGSM8KBFCLHumanEvalGPQAHealthBenchAvg.
Qwen3-1.7BControl0.0057.9290.0045.7330.1318.3540.36
Qwen3-1.7BSRO (Ours)2.23 ± 3.8768.03 ± 10.2692.33 ± 1.1558.30 ± 0.3524.87 ± 7.6223.53 ± 1.2744.88 ± 1.20
Qwen3-4BControl3.3380.8295.0062.8031.0324.9949.66
Qwen3-4BSRO (Ours)1.10 ± 1.9183.17 ± 3.8793.67 ± 1.5373.37 ± 9.2432.87 ± 2.0024.50 ± 4.0751.44 ± 2.02

SRO’s three-run mean exceeds the currently available single-run control score by 4.52 percentage points on 1.7B and 1.78 points on 4B. This comparison supports the additional value of SRO’s structured search mechanism beyond the executable workflow, with the proposer, EHVI, and memory ablations further examining its individual components. The higher six-benchmark averages align with our objective of joint multi-domain optimization. Additional independent runs of this control are underway, and we will report the completed results in the final version.

W5: GPQA degradation and multi-domain trade-offs. Thank you for highlighting the GPQA degradation. SRO’s multi-domain objective is to improve overall performance across all six benchmarks. Across three independent runs, SRO achieves the highest mean six-benchmark average among the evaluated methods at both model sizes. On Qwen3-1.7B, it achieves 44.88 ± 1.20, compared with 37.14 ± 6.93 for AutoTrainess, an improvement of 7.74 percentage points (mean ± sample standard deviation, %).

Regarding GPQA specifically, the three-run mean scores are 24.87 for SRO and 26.70 for AutoTrainess, a difference of 1.83 percentage points. SRO therefore remains below AutoTrainess on this benchmark while achieving a higher aggregate score.

The single-domain experiment specifically optimizes GPQA, whereas multi-domain optimization jointly targets six benchmarks and selects the final checkpoint by their unweighted average, without a non-regression constraint on each benchmark. This permits cross-domain trade-offs: our repeated-run results support improved aggregate multi-domain performance, rather than improvement on every individual benchmark.

Reviewer_yjUgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates automated LLM post-training under limited experimental budgets, addressing the trade-off between predefined recipe spaces, which support systematic optimization but may lack task adaptability, and open-ended LLM agents, which can discover task-specific interventions but may not systematically reuse them. The authors propose Structured Recipe Optimization (SRO), which combines an LLM-based Recipe-Space Designer with a Multi-Domain Recipe Optimizer. The former constructs explicit choices for data sources, data processing, domain composition, and training configurations. The latter uses domain-structured trial memory, LLM candidate generation, and multi-objective Bayesian optimization with expected hypervolume improvement (EHVI) to select recipes for execution. Experiments cover two Qwen3 base models, two agent LLMs, and single- and multi-domain post-training. SRO achieves the highest aggregate score in all evaluated configurations. In multi-domain experiments, it reaches 41.1 versus 15.4 for the strongest baseline on Qwen3-1.7B with GPT-5.4, and 53.8 versus 40.8 on Qwen3-4B with DeepSeek-V4-Flash. Ablations provide evidence for the overall framework and individual optimization mechanisms. However, these results primarily establish empirical effectiveness within the evaluated settings, rather than independently validating every proposed mechanism. The reported multi-domain experiments are SFT-only, whereas an optional GRPO phase is included in the single-domain setup.

Critical strengths
  1. The paper identifies an important systems-level problem: how to incorporate task-specific data and training decisions into a reusable optimization space. Sections 3.1–3.2 provide a concrete formulation that separates recipe-space construction from optimization over executable combinations. The ability to register newly generated datasets as reusable choices is particularly relevant. This formulation offers a practical way to integrate agent-driven exploration with systematic experimental search, rather than relying entirely on trial-level edits.

  2. Tables 1–2 evaluate two base-model sizes and two agent LLMs, with shared agent scaffolds, evaluation interfaces, hardware, and wall-clock budgets. The improvements are substantial in several configurations, particularly the 25.7-point multi-domain advantage on Qwen3-1.7B with GPT-5.4. These results provide meaningful evidence that the integrated system can discover better post-training recipes than the evaluated agent-based baselines under the specified constraints. The comparison does not depend exclusively on achieving state-of-the-art model performance.

  3. Table 3 examines structured recipe search, LLM proposals, EHVI selection, and structured memory. Figure 3 illustrates how Rewrite and Correction datasets become reusable choices during HumanEval optimization, with best-so-far performance progressing from 52.4 to 64.6 across data and training interventions. Appendices C, D.2, and E further document prompts, constructed data sources, verification examples, and contamination safeguards. Together, these provide useful evidence of the framework's operational feasibility and improve the interpretability of its search process, although the trajectory alone does not isolate causal contributions.

Critical weaknesses
  1. Appendix B.1 states that SRO repeatedly evaluates recipes on 50% of each target benchmark, except HealthBench-Easy, which uses 50 samples. These results guide optimization and checkpoint selection, after which the selected model is evaluated on the complete benchmark sets. Consequently, the final evaluation includes examples already used for adaptive selection. Although Appendix E describes safeguards against training-data contamination, these do not eliminate potential selection overfitting to evaluation examples. The baselines also use benchmark feedback, but their search-time sample sizes are agent-selected, introducing additional protocol differences. No direct evidence is found in the manuscript for performance on a benchmark holdout entirely excluded from optimization. The reported improvements remain informative, but their interpretation as independent generalization evidence is limited.

  2. The main ablation removes both structured recipe-space construction and its associated executable workflow. The performance decreases in Table 3—from 45.3 to 34.5 and from 53.8 to 40.3—therefore support the combined design rather than uniquely identifying the benefit of explicit, reusable choices. Furthermore, the predefined-space comparison in Appendix D.1 uses a fixed 5,000-example GSM8K dataset and a restricted configuration space, while SRO can discover additional data and generate supervision. This comparison cannot establish whether the improvement primarily arises from structured representation, better training data, or broader search coverage. These distinctions are central to the proposed paradigm's claimed advantage.

  3. Tables 1–3 report individual experimental outcomes without estimates of variability across independent training and agent-search runs. No direct evidence is found in the manuscript for repeated-run confidence intervals or seed-level variability of the principal comparisons. This matters because some margins are small: on Qwen3-1.7B with DeepSeek-V4-Flash, SRO achieves 45.3 versus 45.1 for AutoTrainess, while removing the LLM proposer reduces the score from 45.3 to 44.0. In addition, AIME 2025 contains only 30 evaluation problems, making its accuracy estimates relatively coarse. The larger reported improvements remain noteworthy, but the paper does not establish consistency across stochastic searches or the reliability of smaller component-level gains.

  4. Section 3.1 defines the final objective as the unnormalized average across domains. However, Appendix B.4 constructs Bayesian optimization objectives using benchmark normalization relative to base and instruction-tuned models, followed by Pareto-aware EHVI selection. These objectives are not necessarily aligned with maximizing the reported aggregate score at every iteration. Moreover, the “w/o EHVI selection” ablation replaces Bayesian selection with LLM-reported confidence rather than another surrogate-based acquisition strategy. Thus, Table 3 supports the use of the proposed selection mechanism over confidence-based selection, but does not establish that EHVI specifically provides an advantage over scalar Bayesian optimization targeting the final metric.

Questions for authors
  1. Were the search-time subsets fixed or resampled across trials, and how did the baselines choose their evaluation subsets? Can the authors provide results on examples never used during recipe selection, or an existing analysis separating previously evaluated from unseen examples? This would help determine how much of the reported advantage generalizes beyond the feedback used during optimization.

  2. Could the authors compare structured and unstructured search while holding the candidate data sources, generation methods, executable workflow, training configurations, and budget fixed? Alternatively, could existing trial records clarify which factors differ in the current ablation? Such evidence would help isolate the contribution of recipe-space structure rather than the complete system.

  3. Can the authors provide repeated-run results, particularly for the close multi-domain baseline comparison on Qwen3-1.7B with DeepSeek-V4-Flash and the smaller optimizer-ablation differences? Means, dispersion estimates, and confidence intervals across independent runs would help distinguish consistent improvements from stochastic variation. If reruns are infeasible, clarifying the available uncertainty evidence would still be useful.

  4. Can the authors explain how the normalization in Appendix B.4 affects candidate rankings and cross-domain trade-offs? A controlled comparison against scalar Bayesian optimization, using the same candidate sets, surrogate features, and compute budget, would help determine whether the multi-objective acquisition itself adds value beyond surrogate-guided selection.

7 comments

We sincerely thank the reviewer for the detailed and constructive feedback. Your questions have helped us sharpen the experimental comparisons and clarify the relationship between our search objectives and reported results. Below, we respond to each weakness and question in turn.

Response to Weaknesses

W1: Evaluation independence and search-time feedback. Thank you for highlighting evaluation independence. Our original experiments follow PostTrainBench’s protocol, which permits querying the target benchmark for evaluation during iteration but prohibits training on its test questions or answers. AutoTrainess’s original evaluation workflow additionally requires sample-level diagnostics containing inputs, targets, and model outputs to guide subsequent iteration; SRO uses only score feedback. To directly assess performance on examples excluded from recipe selection, we added strict 50/50 split experiments, standardizing the examples available for search and checkpoint selection across methods and reserving the remaining half exclusively for final evaluation. SRO achieves the highest held-out average at both model sizes. The original subset settings and full additional results are discussed in Q1.

W2: Contributions of structured search, execution support, and available resources. The original ablation evaluates the overall benefit of structured recipe design and its associated executable workflow; Appendix D.1 examines a setting with a fixed training dataset and search restricted to training configurations. To further distinguish search contributions from resources and execution support, we added a control sharing SRO’s discovery outputs, data-generation methods, mixture and training options, and executable workflow. Sharing these options deliberately holds available resources fixed, allowing comparison between SRO’s structured search mechanism and search organized directly by the agent. Details are provided in Q2. SRO’s contribution relative to predefined-space approaches also includes autonomously discovering task-relevant data and processing choices, reducing the need for task-specific manual recipe-space design.

W3: Independent replication and the reliability of small margins. We conducted additional independent full search runs for all three methods and four ablation variants at both model sizes using DeepSeek-V4-Flash, reporting means and sample standard deviations over three runs. These results cover the specifically highlighted 1.7B baseline comparison and LLM-proposer ablation, with benchmark-level run-to-run variability reported in Q3. Given AIME’s coarse accuracy increments, our conclusions do not rely on changes of one or two solved problems; we assess performance across the six benchmarks and both model sizes.

W4: Search objectives, the final metric, and multi-objective acquisition. We distinguish the normalized search objectives from the raw average used for final checkpoint selection and reporting. Our additional Scalar-EI control keeps the normalization, candidate sets, surrogate features and configuration, and compute budget unchanged to compare multi-objective EHVI with scalar EI. It therefore targets the mean of the normalized domain objectives, rather than directly targeting the final raw average. This design isolates the acquisition-strategy comparison under the same normalization. The results support the additional value of retaining multiple domain objectives in this setting; the normalization’s effect on candidate evaluation and the complete comparison are discussed in Q4.

Q1: Search-time subsets and held-out generalization. Thank you for asking about search-time subsets and performance on unseen examples. In the original experiments, SRO’s search-time subsets were fixed across trials, not resampled: the first 50% of each benchmark, with 50 examples for HealthBench-Easy. The baseline agents chose how many examples to evaluate through the shared evaluation interface, which selected the corresponding prefix of the evaluator’s fixed dataset order. All methods were prohibited from training on test questions or answers. SRO received only evaluation scores, whereas AutoTrainess’s original workflow also required sample-level diagnostics containing inputs, targets, and model outputs to guide subsequent iteration. The submitted final results were evaluated on the full benchmarks.

To assess whether the advantage generalizes to examples never used during recipe selection, we conducted strict 50/50 split experiments. All methods use the same first 50%, fixed across trials, for search and checkpoint selection; the remaining 50% provides no search feedback and is used exclusively for final evaluation. Given the rebuttal time constraints and computational budget, these experiments cover both model sizes and all three methods using DeepSeek-V4-Flash.

The following tables report each method’s same final selected checkpoint separately on search-time examples and unseen held-out examples.

50/50 Split — Search-Set Results

ModelMethodAIMEGSM8KBFCLHumanEvalGPQAHealthBenchMean
Qwen3-1.7BPostTrainBench0.00000.43250.44000.42680.26340.09670.2766
Qwen3-1.7BAutoTrainess0.00000.54480.94000.65850.31700.10440.4274
Qwen3-1.7BSRO (Ours)0.00000.71020.94000.62200.28570.13650.4491
Qwen3-4BPostTrainBench0.06670.55990.92000.69510.29910.18610.4545
Qwen3-4BAutoTrainess0.13330.62520.96000.62200.32590.18480.4752
Qwen3-4BSRO (Ours)0.00000.81340.98000.67070.35270.27200.5148

50/50 Split — Held-Out Results

ModelMethodAIMEGSM8KBFCLHumanEvalGPQAHealthBenchMean
Qwen3-1.7BPostTrainBench0.00000.38940.40000.20730.33040.06990.2328
Qwen3-1.7BAutoTrainess0.00000.52730.90000.24390.25890.09130.3369
Qwen3-1.7BSRO (Ours)0.00000.57730.90000.29270.26790.11670.3591
Qwen3-4BPostTrainBench0.00000.66520.78000.39020.32140.15150.3847
Qwen3-4BAutoTrainess0.06670.65910.86000.21950.29020.14540.3735
Qwen3-4BSRO (Ours)0.00000.83180.92000.40240.30800.20940.4453

SRO achieves the highest mean score on both splits at both model sizes. On the held-out examples, it exceeds the strongest baseline by 2.22 percentage points on 1.7B and 6.06 points on 4B, showing that its aggregate advantage in these experiments extends beyond the feedback used during optimization.

Q2: Structured search under shared resources and workflow. Thank you for this suggestion. The original w/o structured recipe search ablation removed both the structured recipe space and its associated executable workflow. To further compare structured and unstructured search, we added w/o structured recipe search (shared workflow), retaining the same candidate data sources, generation methods, executable workflow, available training configurations, and 20-hour runtime limit.

The control reuses SRO’s discovered resources and options, together with the same data validation, caching, training, and evaluation pipeline. Both methods can choose from the same data and training combinations; the difference is how they organize search. SRO constructs, recombines, and selects candidate recipes using domain-specific evidence. The control instead receives flat resource descriptions and directly chooses its next configuration based on its own trial history, without SRO’s structured candidate construction, candidate batches/pool, explicit cross-domain recombination, or EHVI selection. Both methods select the final checkpoint by the unweighted six-benchmark average.

Given the rebuttal time constraints and computational budget, the currently completed shared-workflow controls comprise one run per model size using DeepSeek-V4-Flash. We compare these with SRO’s mean ± sample standard deviation over three independent full search runs, including the original run.

Shared-Workflow Control — Results

Scores are percentages. Control denotes w/o structured recipe search (shared workflow). SRO entries report three-run means ± sample standard deviations; Control entries report single-run scores.

ModelMethodAIMEGSM8KBFCLHumanEvalGPQAHealthBenchAvg.
Qwen3-1.7BControl0.0057.9290.0045.7330.1318.3540.36
Qwen3-1.7BSRO (Ours)2.23 ± 3.8768.03 ± 10.2692.33 ± 1.1558.30 ± 0.3524.87 ± 7.6223.53 ± 1.2744.88 ± 1.20
Qwen3-4BControl3.3380.8295.0062.8031.0324.9949.66
Qwen3-4BSRO (Ours)1.10 ± 1.9183.17 ± 3.8793.67 ± 1.5373.37 ± 9.2432.87 ± 2.0024.50 ± 4.0751.44 ± 2.02

SRO’s three-run mean exceeds the currently available single-run control score by 4.52 percentage points on 1.7B and 1.78 points on 4B. This comparison supports the additional value of SRO’s structured search mechanism beyond the executable workflow, with the proposer, EHVI, and memory ablations further examining its individual components. The higher six-benchmark averages align with our objective of joint multi-domain optimization. Additional independent runs of this control are underway, and we will report the completed results in the final version.

Reviewer_ZnS4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Structured Recipe Optimization (SRO) for automated LLM post-training. An LLM-based designer constructs an explicit, task-adapted search space covering data sources, data processing, mixtures, and training configurations. A recipe optimizer combines domain-structured trial memory, LLM candidate proposals, and expected hypervolume improvement for multi-objective Bayesian selection.

Experiments use Qwen3-1.7B-Base and Qwen3-4B-Base with DeepSeek-V4-Flash or GPT-5.4 as agents. Single-domain experiments target GSM8K, GPQA Main, and HumanEval under a 10-hour, single-GPU budget. Multi-domain experiments jointly target AIME 2025, GSM8K, BFCL v3 exec simple, HumanEval, GPQA Main, and HealthBench-Easy under a 20-hour budget. SRO achieves the highest aggregate score in every evaluated model–agent configuration. For example, its six-benchmark average reaches 41.1 versus 15.4 for the strongest baseline with Qwen3-1.7B/GPT-5.4, and 53.8 versus 40.8 with Qwen3-4B/DeepSeek-V4-Flash. However, the corresponding margin for Qwen3-1.7B/DeepSeek-V4-Flash is only 45.3 versus 45.1, and gains are not uniform across benchmarks. Ablations support the usefulness of structured search, candidate proposal, acquisition-based selection, and memory.

Critical strengths

A practical integration of task adaptation and systematic search. SRO makes agent-discovered data and processing choices explicit, executable, and reusable across trials. Combining these choices with a validated execution workflow and evidence-guided search addresses a concrete weakness of unrestricted trial-by-trial agent editing. This is consequential because expensive training experiments benefit from reliable execution and systematic reuse of prior discoveries.

Substantial improvements in several matched-budget settings. The evaluation covers two target-model sizes, two agent LLMs, and complementary capability domains, with common hardware and wall-clock limits. Several gains over both automated baselines are large, and the optimizer ablations reuse the same validated initial recipe space. These results provide meaningful evidence that the complete system can improve automated post-training within a limited experimental budget.

Critical weaknesses

The final evaluation is not independent of recipe selection, and run-to-run uncertainty is unreported. Appendix B.1 states that SRO repeatedly evaluates on 50% of each target benchmark, or 50 HealthBench-Easy examples, and subsequently reports performance on the complete benchmark sets, which include those search-time examples. Preventing benchmark examples from entering training does not eliminate adaptive selection overfitting. The baselines also choose their own search-time evaluation sizes, further complicating comparison. Without scores on untouched examples and repeated independent runs, it is difficult to assess generalization and the reliability of small margins, particularly 45.3 versus 45.1 in one multi-domain configuration.

The evidence does not sufficiently isolate the central recipe-space contribution from execution support. The “w/o structured recipe search” ablation removes both explicit recipe-space construction and its associated executable workflow. Its substantial degradation therefore cannot distinguish the value of structured recombination from the benefits of a reliable training and data-processing scaffold. The supplementary predefined-space baseline searches only eight SFT configurations over a fixed GSM8K dataset, with discovery, mixtures, and generation disabled. It consequently offers limited evidence against a competitive predefined recipe space with comparable capabilities. This attribution gap weakens the claimed methodological advance beyond the overall engineered system.

Questions for authors

Evaluation clarification: Can you report results separately on search-time examples and the untouched remainder of each benchmark? Were the search subsets fixed across trials, and can the main comparisons be evaluated under a common search-time evaluation allocation?

Reliability clarification: How many independent agent/search runs underlie each reported configuration? What variability is observed in the main results and ablations, particularly where improvements are only a few points or less?

Contribution clarification: Is there a comparison that retains the same executable workflow, validation, data-generation tools, and compute budget while removing only explicit recipe-space construction and recombination? Such a control would directly affect my assessment of the paper’s central contribution.

6 comments

We sincerely thank the reviewer for the thoughtful assessment and for recognizing the value of combining task-adapted recipe design with systematic search. Your constructive suggestions have helped us strengthen the evaluation and clarify the contributions of structured search. We address each concern below, with additional experiments and supporting evidence.

Response to Weaknesses

W1: Evaluation independence and run-to-run uncertainty. Thank you for highlighting these concerns. We added a common 50/50 split experiment that strictly separates examples used for search and checkpoint selection from those used for final evaluation, reporting results on both halves. We also conducted independent repeats for all main methods and four ablation variants at both model sizes using DeepSeek-V4-Flash, reporting means and standard deviations across three runs. Full protocols and results are provided in our responses to Q1 and Q2, respectively.

W2: Separating recipe-space contributions from execution support. The original ablation evaluates the overall benefit of structured recipe design and its associated executable workflow. We added a control sharing the same discovery outputs, data-generation tools, training options, and executable workflow. Sharing the discovered options holds available resources fixed, allowing us to compare SRO’s structured search mechanism with search organized directly by the agent under the same execution support. SRO retains higher average scores at both model sizes, supporting the additional value of its structured search mechanism. Details are provided in our response to Q3.

W3: Scope of the predefined-space baseline and the value of autonomous construction. Optimization within a predefined configuration space is an established approach represented by work such as AutoPipe [1]. Our Appendix D.1 control searches SFT configurations and subsequently explores GRPO using a fixed GSM8K training dataset, assessing performance when task-specific data are supplied in advance and optimization is restricted to training configurations. SRO’s contribution extends beyond performance gains: the agent autonomously discovers task-relevant data sources and processing methods and organizes them into explicit, reusable, and composable choices, reducing the need for task-specific manual recipe-space design. The additional shared-workflow control shares these discoveries, together with mixture, generation, and training options. It therefore extends beyond the original predefined control’s configuration scope and tests the benefit of structured search given these shared capabilities.

[1] Chwa, Channe, Xinle Wu, and Yao Lu. "Automatic Configuration of LLM Post-Training Pipelines." arXiv preprint arXiv:2603.18773 (2026).

Response to Questions

Q1: Evaluation clarification. Thank you for this suggestion. We conducted additional experiments with a common search-time evaluation allocation and report results separately on search-time examples and the untouched remainder. All methods use the same first 50% of each benchmark for search and checkpoint selection, fixed across trials, with the remaining 50% reserved exclusively for final evaluation. Given the rebuttal time constraints and computational costs, these additional experiments cover both model sizes and all three methods using DeepSeek-V4-Flash.

The original experiments follow PostTrainBench’s protocol, which permits querying the target benchmark for evaluation during iteration but prohibits using its test questions or answers for training. SRO uses the fixed first 50% during search (50 examples for HealthBench-Easy), receiving only scores without inspecting individual evaluation examples or their errors; the submitted results were evaluated on the full benchmark. The additional experiments instead strictly exclude the held-out half from search and checkpoint selection.

The two tables below report each method’s same final selected checkpoint on the search and held-out halves, respectively.

50/50 Split — Search-Set Results

ModelMethodAIMEGSM8KBFCLHumanEvalGPQAHealthBenchMean
Qwen3-1.7BPostTrainBench0.00000.43250.44000.42680.26340.09670.2766
Qwen3-1.7BAutoTrainess0.00000.54480.94000.65850.31700.10440.4274
Qwen3-1.7BSRO (Ours)0.00000.71020.94000.62200.28570.13650.4491
Qwen3-4BPostTrainBench0.06670.55990.92000.69510.29910.18610.4545
Qwen3-4BAutoTrainess0.13330.62520.96000.62200.32590.18480.4752
Qwen3-4BSRO (Ours)0.00000.81340.98000.67070.35270.27200.5148

50/50 Split — Held-Out Results

ModelMethodAIMEGSM8KBFCLHumanEvalGPQAHealthBenchMean
Qwen3-1.7BPostTrainBench0.00000.38940.40000.20730.33040.06990.2328
Qwen3-1.7BAutoTrainess0.00000.52730.90000.24390.25890.09130.3369
Qwen3-1.7BSRO (Ours)0.00000.57730.90000.29270.26790.11670.3591
Qwen3-4BPostTrainBench0.00000.66520.78000.39020.32140.15150.3847
Qwen3-4BAutoTrainess0.06670.65910.86000.21950.29020.14540.3735
Qwen3-4BSRO (Ours)0.00000.83180.92000.40240.30800.20940.4453

SRO achieves the highest mean score on both splits at both model sizes. On the held-out half, it outperforms the strongest baseline by 2.22 percentage points on 1.7B and 6.06 points on 4B, retaining its aggregate advantage under a common search-time example allocation with final evaluation examples excluded from search and checkpoint selection.

Q2: Reliability clarification. Thank you for highlighting the number of independent agent/search runs and the variability of the results. Given the GPU time and API costs of complete searches, each configuration in the submission was evaluated with one independent run. We have now conducted two additional independent full search runs for every method and ablation configuration listed below, reporting mean ± sample standard deviation over three runs, including the original run. Given the rebuttal time constraints and computational budget, these repeated experiments cover both model sizes, all three methods, and all four ablation variants using DeepSeek-V4-Flash.

These repeated runs follow the search and final-evaluation protocols used in the submission and are reported separately from the additional 50/50 split experiments above. All scores are percentages. Avg. is the unweighted mean across the six benchmarks; its standard deviation is calculated from the per-run averages.

Variability in the main results. SRO achieves the highest mean aggregate score at both model sizes, with lower aggregate run-to-run variability than both baselines. It exceeds the strongest baseline by 6.61 and 6.98 percentage points on 1.7B and 4B, respectively. Its aggregate standard deviations are 1.20 and 2.02, compared with 6.78–6.93 and 3.39–4.99 for the baselines.

Repeated Runs — Main Results

ModelMethodAIMEGSM8KBFCLHumanEvalGPQAHealthBenchAvg.
Qwen3-1.7BPostTrainBench0.00 ± 0.0056.33 ± 17.7389.00 ± 7.0046.53 ± 15.7924.40 ± 5.4713.33 ± 9.1738.27 ± 6.78
Qwen3-1.7BAutoTrainess0.00 ± 0.0060.33 ± 11.7988.00 ± 4.5836.77 ± 18.8426.70 ± 3.8411.07 ± 4.0337.14 ± 6.93
Qwen3-1.7BSRO (Ours)2.23 ± 3.8768.03 ± 10.2692.33 ± 1.1558.30 ± 0.3524.87 ± 7.6223.53 ± 1.2744.88 ± 1.20
Qwen3-4BPostTrainBench2.23 ± 3.8770.47 ± 7.4891.67 ± 3.2152.83 ± 13.9432.20 ± 1.2517.33 ± 3.6544.46 ± 4.99
Qwen3-4BAutoTrainess0.00 ± 0.0065.07 ± 8.7092.00 ± 3.4648.60 ± 5.6730.67 ± 3.4215.57 ± 0.2141.98 ± 3.39
Qwen3-4BSRO (Ours)1.10 ± 1.9183.17 ± 3.8793.67 ± 1.5373.37 ± 9.2432.87 ± 2.0024.50 ± 4.0751.44 ± 2.02

Variability in ablations. Full SRO achieves the highest mean aggregate score at both model sizes. Compared with the closest ablated variant, the results are 44.88 ± 1.20 vs. 42.52 ± 1.98 on 1.7B (w/o LLM proposer) and 51.44 ± 2.02 vs. 49.38 ± 1.05 on 4B (w/o structured memory), corresponding to improvements of 2.36 and 2.06 percentage points. We report the mean and sample standard deviation for every benchmark below.

Repeated Runs — Ablation Results

ModelVariantAIMEGSM8KBFCLHumanEvalGPQAHealthBenchAvg.
Qwen3-1.7BFull SRO2.23 ± 3.8768.03 ± 10.2692.33 ± 1.1558.30 ± 0.3524.87 ± 7.6223.53 ± 1.2744.88 ± 1.20
Qwen3-1.7Bw/o structured recipe search0.00 ± 0.0052.84 ± 4.9885.33 ± 4.5131.69 ± 7.6125.66 ± 2.279.88 ± 2.4434.22 ± 2.61
Qwen3-1.7Bw/o LLM proposer3.34 ± 3.3570.96 ± 2.9594.00 ± 1.0051.43 ± 3.9723.28 ± 6.5612.14 ± 1.5842.52 ± 1.98
Qwen3-1.7Bw/o EHVI selection0.00 ± 0.0058.44 ± 2.9493.00 ± 1.0043.69 ± 6.8430.81 ± 1.0216.07 ± 2.2540.35 ± 0.72
Qwen3-1.7Bw/o structured memory1.10 ± 1.9164.08 ± 10.1491.67 ± 4.0450.22 ± 8.4423.36 ± 7.0518.67 ± 5.2841.53 ± 1.62
Qwen3-4BFull SRO1.10 ± 1.9183.17 ± 3.8793.67 ± 1.5373.37 ± 9.2432.87 ± 2.0024.50 ± 4.0751.44 ± 2.02
Qwen3-4Bw/o structured recipe search3.34 ± 3.3569.02 ± 7.6989.00 ± 4.5856.50 ± 9.8931.64 ± 1.4712.81 ± 5.8143.71 ± 3.75
Qwen3-4Bw/o LLM proposer2.21 ± 1.9185.48 ± 0.9894.33 ± 1.1563.43 ± 8.8830.87 ± 7.4918.46 ± 3.7249.12 ± 1.64
Qwen3-4Bw/o EHVI selection4.44 ± 5.0981.96 ± 5.6493.00 ± 0.0059.75 ± 8.5335.87 ± 0.4716.38 ± 2.2148.56 ± 2.93
Qwen3-4Bw/o structured memory1.11 ± 1.9279.56 ± 2.1493.33 ± 1.5366.48 ± 1.0433.47 ± 2.7022.40 ± 6.4849.38 ± 1.05

Comments on the paper

1 comment

Dear reviewers,

The author-reviewer discussion period is approaching the deadline. Please read through the author's rebuttals and update your reviews. Thank you for your time and work.

Best Regards, AC.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.