acceptodds
Under review as a conference paper at ICLR 2027

HeuriTrain: Search Organization and Verification in Autonomous Post-Training

Abstract

Autonomous post-training requires an LLM researcher to choose data and training interventions under costly and variable feedback. Evaluating such systems requires understanding what they discover, how they reach their configurations, and whether observed gains recur. We present HeuriTrain, a framework for budgeted experimental search over complete data and training-recipe configurations. Explicit task contracts specify editable components, execution and verification rules, and budgets, while experimental records connect configuration inheritance, referenced results, executed changes, and outcomes. Within this framework, we study global search policies and Factorized Experimental State (FES), which organizes parent selection, visible history, and edits by intervention surface. Experiments on IFEval and Countdown examine checkpoint discovery and researcher decisions, complemented by fixed-configuration retraining and repeated policy comparisons on a deterministic control task. We observe that researchers can draw on results beyond their selected parent, that permitted and executed edit scopes can differ, and that high discovery scores need not persist under retraining. The control task provides evidence of a short-horizon policy effect, with much of the FES–Greedy separation present at the first proposal. Together, these findings motivate evaluating autonomous post-training through its discovered checkpoints, decision process, and repeatability.

Primary area:
foundation or frontier models, including LLMs
Keywords:
autonomous post-training, LLM agents, automated machine learning, experimental search

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_yQX7OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces HeuriTrain, a framework for autonomous exploration of LLM post-training configurations. An AI agent proposes changes to training data and recipes, while a harness validates the changes, executes training and evaluation, and records results under a fixed task contract and budget. The paper compares Global Greedy, Global Archive, and Factorized Experimental State (FES), which separates data and recipe exploration. Experiments on IFEval, Countdown, and a deterministic Splay task examine search performance, interactions between edits, and the reliability of results under re-evaluation and re-execution.

Critical strengths
  1. The framework records proposed edits, executed changes, evaluation results, and computational costs, providing useful traceability for inspecting and comparing agent-driven training experiments.
Critical weaknesses
  1. The framework largely follows the standard process of proposing data or hyperparameter changes, running training experiments, and using evaluation feedback to guide further trials. This is a common workflow for human researchers, and this paper only validates that modern agents can do the same thing. The paper claims they find that data and recipes should not be seperated. This is a common sense for doing research, not a finding or contribution of this paper.
  2. The central FES policy receives only a brief definition in the main text, and its contribution is not clearly separated from that of the HeuriTrain harness. FES changes parent eligibility, visible history, edit permissions, and opportunity allocation together, making it unclear which design choice drives the reported gains.
Questions for authors
  1. What specific limitation of AI auto-research does HeuriTrain address beyond the standard experiment–feedback loop, and what evidence demonstrates that its proposed components are necessary to address this limitation?
  2. Can you provide ablations that independently vary parent selection, visible history, edit scope, and opportunity allocation, so that the contribution of factorized experimental state can be distinguished from the effects of restricting the search space?
2 comments
AuthorsOpenReview ↗

Response to Reviewer yQX7

Thank you for the review, for recognizing the value of the experimental records, and for the specific request to separate the effects of FES's design choices. We address the contribution, the interpretation of composition, and the additional ablations below.

W1 and Q1 — Research question and contribution. We agree that the experiment–feedback loop is a standard research workflow. The paper studies choices within that loop: which measured configuration the agent inherits, which earlier results it can consult, what it may change, and how its remaining opportunities are allocated. HeuriTrain makes these choices explicit so that different search organizations can be compared under a common task contract.

The experimental records connect those choices to actual interventions and outcomes. This enables analyses of inherited versus consulted evidence, permitted versus executed edits, interactions between separately discovered changes, and the persistence of discovery results under verification. The additional ablation examines which parts of the tested policy account for its observed behavior; it does not establish that every component is necessary.

W1 — Separate exploration and composition. We would like to clarify a distinction central to the paper. FES separates Data and Recipe exploration by construction (Sec. 2.3). The composition experiments ask whether changes that were beneficial separately remain beneficial when combined (Secs. 3.3–3.4).

The executed IFEval crossing has a −26-prompt interaction contrast. In the deterministic Splay composition study, 10 of 12 combinations score below the stronger constituent, and 4 score below the root. These measurements show that individually favorable changes need not compose favorably in the tested configurations. They do not imply that Data and Recipe exploration should never be separated.

W2 and Q2 — Attribution to individual design choices. Thank you for this suggestion. We evaluated three intermediate policies alongside Greedy and FES on the same 18 Splay blocks with six proposals per search. Greedy + scope introduces alternating single-surface edits while retaining Greedy's parent rule and history. Adding the FES parent rule or history separately, and then jointly in full FES, gives a 2×2 comparison among the four constrained variants (global response, item 2).

All four constrained variants have higher observed means than Greedy; Greedy + scope has the highest mean (675.5 rotations of gain, versus 467.1 for Greedy and 569.0 for FES). The individual confidence intervals against Greedy include zero, and the results do not establish an additional benefit from separate parent pools or histories in this six-proposal setting. Opportunity allocation remains coupled to the alternation schedule, so the available ablation does not independently identify all four factors.

For clarity, FES maintains two surface-specific states with their own eligible parents and visible histories. The fixed alternation schedule determines the active edit surface, while the inactive surface remains at its root configuration. The harness enforces the task contract; FES specifies how the search context and edit permissions are organized within that contract.

We hope these clarifications and additional comparisons help make the empirical contribution and its scope easier to assess. We appreciate your consideration of this evidence.

Thanks for the response from the authors. I'm still concern about the position and contribution of this paper. The data exploration, recipe exploration, config inheriting, are all common practice for human researcher. The core of this paper seems to just define some contracts to ensure the agents follow a common research workflow. The model improvement brought by such workflow is expected. In my view, this paper seems to not identify a specific critical problem about AI auto-research and solve it, or bring some new insights to this field. It is more like an engineering practice. Besides, there's a lof of room for improving the writing of this paper. For example, the paper names their framework as HeuriTrain, while in the experiment, FES is refered as their method, which is only described in a short paragraph (line 187-192) and confusing.

Reviewer_rbEmOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces HeuriTrain, an autoresearch framework for LLM post-training in which a research agent iteratively modifies training data and training recipes, runs experiments, and uses evaluation feedback to guide next steps. The main methodological contribution is Factorized Experimental State (FES), which separates Data and Recipe exploration into two search branches, with distinct parent pools, experimental histories, etc.

The authors evaluate FES against Global Greedy (similar to AutoResearch-style search policy) and Global Archive on two post-training tasks, IFEval and Countdown, as well as a deterministic task, i.e., Splay control.

While the paper addresses an interesting problem and provides useful experimental analyses, my major concerns are its limited methodological novelty and the insufficient empirical evidence supporting the effectiveness of FES.

Critical strengths
  1. The paper studies an important problem: how autonomous research agents should organize past discoveries under limited computational budgets.

  2. The authors investigate experimental variability and interactions between Data and Recipe changes for autoresearch tasks. The verification analyses provide useful insights into the reliability of autonomous post-training experiments.

Critical weaknesses
  1. Limited methodological novelty

The overall proposed HeuriTrain framework largely follows the established AutoResearch framework: an LLM researcher iteratively proposes modifications, runs experiments, evaluates performance, and uses historical feedback to guide subsequent proposals. The paper's Global Greedy baseline already implements this general workflow.

The main methodological contribution beyond this established approach seems to be Factorized Experimental State (FES), which separates Data and Recipe exploration into independent branches with different states.

In fact, my belief is that FES imposes a potentially restrictive search space. As described in Section 2.3, the inactive component remains fixed at its root configuration. Consequently, standard FES exploration considers configurations of the form ((D,R_0)) or ((D_0,R)), but does not jointly optimize both components away from their root values. This may exclude configurations that require coordinated improvements in Data and Recipe (correct me if I am wrong). So the upper bound of the this method is much lower than the un-restricted search, and also requires a strong initialization configuration, i.e., D_0 and R_0.

Overall, I find the methodological contribution relatively incremental compared with existing autonomous research frameworks.

  1. Insufficient empirical evidence supporting FES

Although FES is the paper's central methodological/technical contribution, the experiments do not convincingly demonstrate that it improves over simpler search strategies, particularly Global Greedy(AutoResearch). In Table 1, FES substantially underperforms Global Greedy on IFEval (43.07 vs. 49.35), while outperforming it on Countdown (57.10 vs. 49.15). These mixed results do not establish a consistent advantage for FES on actual autoresearch post-training tasks.

The authors also provide eight paired IFEval searches in Table 2, but these experiments do not include Global Greedy. Moreover, the reported mean advantage of FES over Archive is relatively modest (+1.73) with high variance. Therefore, the repeated experiments do not provide conclusive evidence that FES improves average search performance even over Global Archive.

The deterministic Splay experiments provide stronger evidence favoring FES. However, as discussed in Section 3.4, nearly the entire advantage over Global Greedy emerges after the first proposal.

Given these findings, it remains unclear when or why practitioners should prefer FES over the simpler Global Greedy/autoresearch-style approach.

Questions for authors
  1. What is the core distinction between HeuriTrain and existing AutoResearch-style frameworks beyond factorizing Data and Recipe exploration?
  2. Understanding the first-proposal advantage of FES. In Section 3.4, the authors report that almost the entire FES advantage over Global Greedy in the deterministic Splay experiment shows after the first proposal. Since no prior experimental feedback is available at this stage, could the authors clarify what causes this substantial initial advantage?
2 comments
AuthorsOpenReview ↗

Response to Reviewer rbEm

Thank you for the thoughtful discussion of the search space and the evidence needed to prefer FES over a simpler policy. We appreciate the opportunity to clarify both the intended contribution and the interpretation of the comparisons.

W1 — Search-space restriction. Your description of standard FES exploration is correct: proposals have the form (D,R0)(D,R_0) or (D0,R)(D_0,R), with the inactive surface fixed at its root value. This can exclude useful joint configurations. A smaller reachable set, however, does not by itself determine which policy performs better under a fixed short budget; that question requires an empirical comparison.

The new ablation helps separate single-surface edits from keeping the inactive surface at its root. Greedy + scope changes one surface at a time while extending the global incumbent, so successive proposals can reach configurations in which both surfaces differ from the root. Its observed mean gain is 675.5 rotations, compared with 569.0 for full FES. The six-proposal results therefore do not establish an additional benefit from the FES parent rule (global response, item 2).

The composition results address a related but distinct issue: the benefit of an edit can depend on the configuration in which it is applied. They motivate evaluating complete configurations when combining edits; they do not establish that keeping the inactive surface at its root is generally preferable.

Novelty and Q1 — Contribution beyond the experiment–feedback loop. We agree that the basic propose–execute–evaluate loop is established, and Global Greedy implements that familiar baseline in our setting. HeuriTrain provides a common task contract and explicit policy state for comparing how parent eligibility, visible evidence, edit permissions, and budget shape the next experiment. The ledger distinguishes the inherited parent from other consulted evidence and from the changes actually executed; verification then distinguishes checkpoint re-evaluation from configuration re-execution.

The contribution is the resulting experimental study of search organization, decisions, edit interactions, and repeatability. The new component ablation makes the attribution question more directly testable within that framework.

W2 — Strength of the empirical evidence. We agree that the post-training results do not establish a consistent FES advantage over Greedy. Table 1 reports single searches: FES scores 43.07 versus 49.35 on IFEval and 57.10 versus 49.15 on Countdown. The IFEval searches also use different roots, so those endpoints are not a matched policy comparison.

The repeated IFEval study compares FES with Archive only. Its mean difference is +1.73 percentage points (95% CI [−0.25, 3.10]); it does not resolve a positive average effect at that confidence level. On Splay, the new FES–Greedy replication has a smaller positive mean difference, and the pooled result is reported alongside both cohorts in the global response.

Q2 — Origin of the first-proposal gap. Thank you for identifying this diagnostic. At proposal 1, feedback from previous proposals is unavailable, so the separation cannot be attributed to accumulated experimental history. The candidate-level analysis shows lower mean first-proposal scores for unrestricted edits under the six-proposal request, while the constrained proposals have positive mean changes. Larger edits are also associated with worse score changes (global response, item 3). These are scores of proposed candidates before incumbent retention, not evidence that Greedy accepts a worse incumbent.

The results are consistent with the initial proposal constraints affecting candidate quality. The scope-only ablation further shows how that constraint can be used while retaining access to joint configurations. Its higher observed mean makes it a useful comparison for understanding the short-budget Splay result; the available post-training evidence does not yet establish a general preference for FES over Greedy.

Thank you for the reply. While I appreciate the added empirical analysis, my concerns regarding novelty and the effectiveness of FES's distinctive design choices remain. I will hence keep my original rating

Reviewer_YVsXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how autonomous post-training agents should organize experiments under a fixed budget. HeuriTrain records task constraints, parent configurations, visible history, edits, outcomes, and costs. Its Factorized Experimental State (FES) searches data and recipe separately, keeping the other component fixed at the root configuration. Experiments compare FES with Global Greedy and Global Archive on IFEval, Countdown, and a deterministic Splay task. Additional analyses examine edit interactions, checkpoint re-evaluation, and configuration re-execution.

Critical strengths
  1. The paper reports unfavorable results, initialization losses, uncertain policy differences, and changes under re-evaluation. The paired analyses use complete searches or matched blocks as statistical units.

  2. The harness provides a useful experimental framework that links each proposal to its available evidence, executed changes, and measured outcome. Evaluating complete configurations and separating inherited parents from consulted evidence makes the search easier to audit.

  3. The 18 matched Splay blocks support an advantage for FES in that setting. The composition experiments also show that individually useful edits can interact negatively without training execution noise.

Critical weaknesses
  1. The incremental contribution and mechanism remain unclear. Prior works already studied structured feedback and search organization. FES offers a specific combination of constraints, but the experiments do not isolate its added value. In Splay, the FES vs Greedy gap is 278.9 rotations after the first proposal and 283.8 at the end (Section 3.4). Thus, almost all separation appears before feedback from earlier proposals exists. This supports an effect of the initial proposal constraints, but provides limited evidence for the benefit of separate histories.

2.Practical gains and generalization are not sufficiently established. In the paired IFEval study, the trained root scores 34.57%, compared with 47.32% for the unadapted model, while FES reaches 47.25% on average (in Tables 2 and 5). The shared root supports a conditional policy comparison, but much of the gain recovers the loss caused by initialization. Results from better-calibrated roots would clarify practical value.

  1. Verification identifies instability without establishing whether the main policy results remain valid. The paper does not show whether the main policy rankings persist after consistent checkpoint re-evaluation and controlled retraining.

  2. Root definitions, proposal units, and stopping rules are scattered across the paper; unresolved references further reduce clarity.

Questions for authors
  1. See Weakness 2: Can you include Greedy under the exact paired IFEval protocol and assess whether the comparison holds from a calibrated root?
  2. See Weakness 3: Do the main policy rankings hold after re-evaluating the selected checkpoints with the same evaluation setup and repeating training under controlled conditions?
2 comments
AuthorsOpenReview ↗

Response to Reviewer YVsX

Thank you for the careful review and for recognizing the value of the matched comparisons, composition experiments, and experimental records. Your comments help distinguish the evidence for a search-policy effect from the evidence needed to establish practical post-training gains and their repeatability.

W1 — Component attribution and contribution. We agree that a gap already present at the first proposal cannot demonstrate a benefit from accumulated history. The new six-proposal ablation directly examines this issue: Greedy + scope has a mean gain of 675.5 rotations, compared with 467.1 for Greedy and 569.0 for FES. The individual contrasts with Greedy remain uncertain, and the additional parent-pool and history components do not show a resolved incremental benefit in this setting (global response, item 2).

HeuriTrain makes parent eligibility, visible evidence, edit permissions, and budget explicit within a common experimental framework. Its records connect the inherited configuration, consulted results, executed changes, and measured outcome. This supports the paper's empirical study of proposal behavior, interactions between edits, and the relationship between discovery scores and subsequent verification. The new ablation adds a component-level comparison to that study.

W2 and Q1 — Initialization and the paired IFEval comparison. Thank you for highlighting this distinction. The shared trained root scores 34.57% (187/541), compared with 47.32% for the unadapted model. FES reaches 47.25% on average, so the gain over this root largely recovers the initialization loss.

The shared root controls the initial condition for the paired FES–Archive comparison: +1.73 percentage points on average, with a 95% CI of [−0.25, 3.10]. This is a conditional policy comparison and does not establish an improvement over the unadapted model. We have not completed the requested matched Greedy comparison or a replication from a better-calibrated root. The available evidence therefore leaves those practical comparisons open.

W3 and Q2 — Stability under verification. We appreciate the distinction between checking selected results and verifying a policy ranking. The re-evaluation of 32 adapters and the three re-execution pairs examine selected checkpoints and configurations. They do not provide a complete, consistently re-evaluated and retrained comparison of all search policies, so post-training policy-ranking stability remains unestablished.

The additional Splay replication addresses repeatability of the search comparison under deterministic scoring. Its smaller mean difference and confidence interval are reported in item 1 of the global response. This is a separate control-task result and does not settle the requested post-training verification.

W4 — Protocol clarity. We agree that the root definitions, proposal units, and stopping rules should be easier to locate. For the paired IFEval comparison discussed here, the shared trained root is the configuration scoring 187/541; it is distinct from the unadapted model. A proposal opportunity is a scheduled attempt, whereas an evaluated candidate has a measured outcome. The registered proposal and compute limits determine when a campaign stops. Keeping these distinctions explicit is important when interpreting the reported counts and comparisons. We also appreciate the note on cross-references and will check their consistency when incorporating these clarifications.

Thank you again for the specific suggestions. They help sharpen both the interpretation of the existing results and the questions that remain open.

Thank you for the response and additional experiments. They help clarify the role of the FES components. I also appreciate the authors’ clear acknowledgment of the limitations. However, my concerns about practical gains and results stability remain, so I will keep my rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global response

We thank the reviewers for their careful reading and constructive suggestions. The reviews raise important questions about component attribution, initialization, and verification. We address the post-training questions in the individual responses below and report the additional Splay experiments completed for this rebuttal (972 researcher calls).

1. Replication of the short-budget comparison. We repeated Global Greedy and FES on the same 18 blocks under the six-proposal protocol. The FES–Greedy difference was +101.9 rotations (95% CI [−142.7, 354.2]; 10 wins, 8 losses), compared with +283.8 [77.7, 493.8] in the original cohort. Pooling the two cohorts gives +192.9 [28.7, 359.2], with 24 wins and 12 losses across 36 matched observations. The new cohort has a smaller positive mean difference, with uncertainty that includes zero. The pooled estimate provides a fuller account of the variation across these repeated searches.

2. Component ablation. To address the attribution question, we evaluated three intermediate policies alongside the repeated Greedy and FES baselines on the same 18 blocks, with six proposals per search.

PolicyParent ruleHistoryEdit scopeMean gain over rootDifference from Greedy [95% CI]
Global GreedyGreedyGreedyunrestricted467.1—
Greedy + scopeGreedyGreedyalternating single surface675.5+208.4 [−75.3, 462.2]
Greedy + scope + parentFESGreedyalternating single surface606.9+139.8 [−87.9, 380.8]
Greedy + scope + historyGreedyFESalternating single surface653.1+186.0 [−94.7, 452.8]
FESFESFESalternating single surface569.0+101.9 [−142.7, 354.2]

All four constrained variants have higher observed means than Greedy, with the highest mean attained by Greedy + scope. The individual confidence intervals include zero. Within the four constrained variants, the estimated main effects of the FES parent rule and history are −76 and −30 rotations, respectively. These results do not establish an additional benefit from separate parent pools or histories in this six-proposal setting. Scope restriction and the alternation schedule are introduced together, so their individual contributions are not separated by this ablation.

3. First-proposal behavior. We examined candidate scores before incumbent retention. Under the six-proposal request, unrestricted first proposals score below the root on average in both cohorts (−251.1 and −519.4 rotations). The corresponding constrained proposals have positive mean changes: +142.4 in the original cohort and +48.3 across 72 new samples of the identical constrained request. Larger edits are associated with lower score changes in the two cohorts (Spearman correlations −0.32 and −0.51).

These observations are consistent with proposal constraints affecting initial candidate quality, before feedback from earlier proposals is available. Generation remains variable despite deterministic scoring: three groups with identical first-proposal requests have 18-block means of −3, +299, and −165 rotations. We therefore interpret the first-proposal analysis together with the repeated endpoint comparisons.

4. Longer search horizon. Under the 12-proposal request, the FES–Greedy gap is +212.4 rotations at proposal 1, +224.6 at proposal 6, and +129.9 at proposal 12 (95% CI [−173.4, 428.0]). The mean score change of an unrestricted first candidate is −5.1 rotations under this request, showing that its behavior also varies with the requested horizon. The observed endpoint gap narrows by proposal 12; this experiment does not establish a growing advantage from accumulated history.

Together, these analyses clarify the short-budget Splay result and distinguish observed proposal effects from unresolved benefits of maintaining separate search states.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.