acceptodds
Withdrawn from ICLR 2027

SPOT: Stateful Post-Training Search by LLM Researchers and Actionable Training Insights

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Post-training requires coupled decisions about data, model updates, and checkpoint initialization; because each experiment changes the evidence and checkpoints available next, these decisions form a sequential joint search. LLM researchers can close this loop, but existing systems either expose a narrow action space or permit pipeline changes that make experiments difficult to compare. We present SPOT (Stateful POst-Training Search), which converts hypotheses into structured joint interventions and executes them through a shared interface. SPOT records both the proposed recipe and the operation actually consumed by training, then returns its telemetry, evaluation outcomes, and resulting checkpoint to guide subsequent pro- posals. Across six search campaigns, SPOT evaluates 141 candidate checkpoints; every LLM researcher improves over the shared base, and the best search-time composite score increases from 0.4092 to 0.5152 on a 0–1 scale. Interface ab- lations further show that restricting proposal scope changes both update choice and checkpoint inheritance, rather than merely reducing the number of editable fields. Beyond endpoint scores, the aligned records reveal a diagnostic hierarchy for choosing the next experiment. Does the sampled data provide usable learning contrast, or do different-quality outputs receive indistinguishable rewards? Does the surviving signal reward the intended behavior, or does the proxy improve while the target behavior deteriorates? Can the current search space express the required repair, or does persistence across explored controls indicate a missing intervention? These questions distinguish when to continue training, change sampling, redesign the objective, or expand the intervention space.

Primary area:
foundation or frontier models, including LLMs
Keywords:
post-training, LLM, recursive self-improvement

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_8rzxOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents SPOT, a framework in which an LLM-based researcher repeatedly proposes a joint post-training recipe. Each proposal is validated, executed, and evaluated through a shared pipeline, and the outcomes and new checkpoint are returned to the researcher's history for the next round. The authors instantiate SPOT on tool-use post-training of Qwen3-8B with LoRA, using four LLM researchers (GLM-5.3, Qwen3.8-27B, Kimi-K2.6, and Nemotron-3-Ultra) plus TPE and random search as classical references. The best search-time composite score rises from 0.4092 to 0.5152. They further run interface ablations (one-component proposals and no auxiliary annotations) and use the recorded training trajectories to propose a three-question diagnostic hierarchy.

Critical strengths
  • The paper studies a timely and important question, namely, automated LLM post-training with LLM-based researchers (a form of AutoML/auto-research).
Critical weaknesses
  • Limited novelty and unclear positioning. The core loop (i.e., LLMs proposes a configuration, observes the outcome, and revises the next proposal) already exists in LLM-guided hyperparameter optimization and autonomous post-training systems [1-5]. SPOT mainly enlarges the action space and adds detailed logging, which is an engineering improvement rather than methodological innovation. The diagnostic hierarchy, which the paper presents as more important, consists of retrospective case analyses of known phenomena (zero within-group reward variance and reward over-optimization [6,7]) and is not validated prospectively.

  • The central stateful claim is not supported by the evidence. 24 of the 34 parent-relative updates regress, three of the four best LLM-found checkpoints (including the overall best, 0.5152) were trained from the base model, and no controlled ablation isolates the effect of checkpoint inheritance. In addition, each campaign is run only once on one student model, and the selection utility is computed on the same benchmarks used to report results, without a held-out test. The headline improvement can therefore neither be attributed to stateful search nor shown to be statistically reliable.

[1] Large Language Models to Enhance Bayesian Optimization. ICLR, 2024.

[2] AgentHPO: Large Language Model Agent for Hyper-Parameter Optimization. CPAL, 2025.

[3] Lamdagent: An autonomous framework for post-training pipeline optimization via llm agents. EMNLP, 2025.

[4] PostTrainBench: Can LLM Agents Automate LLM Post-Training? ICML, 2026.

[5] TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Exploration. arXiv, 2026.

[6] DAPO: An Open-Source LLM Reinforcement Learning System at Scale. NeurIPS, 2026.

[7] Scaling Laws for Reward Model Overoptimization. ICML, 2023.

Questions for authors
  1. What is methodologically new in SPOT compared with LLAMBO, AgentHPO, LaMDAgent, PostTrainBench, and TREX, beyond a larger action space? Have the authors compared SPOT with any of them under the same budget?
  2. Under the same budget, how does the Full interface perform when checkpoint inheritance is disabled? Among the top-k candidates, how many come from inherited runs and how many from restarted runs?
  3. How do the selected checkpoints perform on a held-out benchmark that was never used for selection, and how large is the run-to-run variance of a campaign?
  4. Have the code, proposer prompts, ledgers, and evaluation scripts referred to in data/appendix_20260925/ be released through an anonymous repository? If not, why are these paths placed in the paper?
  5. Please explain or make the candidate counts consistent in the main text. The caption of Figure 4 mentions 97 LLM candidates, but the data in Figure 4(c) add up to 99. Additionally, Section 3.1 reports a total of 141 fully evaluated candidates, while Appendix F lists 150.
  6. Suggestion: Extend the study to other student model families and sizes, full-parameter training, and other post-training domains (e.g., math reasoning).
Reviewer_BViQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SPOT, a new framework that enables LLM researchers to iteratively propose, run, and evaluate recipes. The proposed framework allows LLM researchers to jointly select multiple changes to difference aspects of the training, e.g., data, learning algorithm.

The authors evaluate four LLM researchers for post-training Qwen3-8B. Across 141 evaluated checkpoints, the best composite evaluation score improves from 0.4092 to 0.5152. The paper also analyzes researcher behavior and training diagnostics, including reward-signal quality and reward–behavior misalignment.

While the paper is trying to solve an interesting and practical problem, I have concerns regarding the work's novelty, the weakness of its chosen baselines, and the experimental evidence supporting its proposed design choices.

Critical strengths
  1. The paper works on a practical problem of automating LLM post-training. It provides a unified interface covering different aspects of the LLM training.

  2. The experiments include four different LLM researchers and a relatively large collection of training experiments, offering useful observations about how different models conduct autonomous research.

Critical weaknesses

1. Missing strong LLM-based autonomous research baselines

My major concern is the absence of comparisons against existing LLM-based autonomous experimentation frameworks. The proposed SPOT workflow is simply an LLM researcher proposing changes to the configurations, and iteratively refining subsequent proposals has been used in many existing frameworks. For example, Karpathy's AutoResearch (https://github.com/karpathy/autoresearch) already shows an LLM agent autonomously modifying training configurations, running experiments, evaluating outcomes, and retaining or discarding changes based on previous results. Additional baselines include AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench [1]. However, the paper mainly compares against random search and Optuna TPE, which are classical optimization methods rather than LLM-based autonomous research agents. These baselines cannot evaluate whether SPOT provides meaningful improvements over existing agentic experimentation frameworks.

I would strongly encourage the authors to compare against a simpler AutoResearch-style LLM agent operating with the other conditions.

2. Insufficient justification and inconsistent empirical benefits of the proposed components

The paper emphasizes two design choices: allowing joint intervention proposals across multiple training components and providing auxiliary reviewer annotations. However, the ablation results in Section 4 do not consistently support their effectiveness. For joint proposal scope, it works for GLm and Qwen while the opposite occurs for Kimi-K2.6 and Nemotron-3-Ultra. Thus, the experiments do not establish that joint interventions are consistently more effective than simpler single-component modifications. For auxiliary annotations, similarly, the No-annotation setting achieves comparable or better performance at least for two of the four researchers, most notably Kimi-K2.6, whose best score increases after removing external annotations. These results raise questions about the practical benefits of introducing the additional reviewer and reward-hacking auditor.

Put it simply, I find the design choices insufficiently justified. The current ablations results simply show that different choices lead to different behaviors, but they do not convincingly establish that the proposed added components improve autonomous post-training.

3. Limited novelty

I find that the overall designs are very much standard practice used in auto research agents. The proposed work's main distinction appears to be its structured search space within a unified post-training interface. However, the paper does not sufficiently explain why this particular formulation shows a substantive methodological advance over existing approaches, like autoresearch, [1], and [2].

[1] Toledo, Edan, et al. "Ai research agents for machine learning: Search, exploration, and generalization in mle-bench." Advances in Neural Information Processing Systems 38 (2026): 35309-35348.

[2] Li, Junbo, et al. "The Optimizer Is the Agent: Reasoning-Driven Search across Prompts, Programs, and ML Workflows." COLM 2026.

Questions for authors
  1. How does SPOT differ fundamentally from AutoResearch, and other LLM-based auto research baselines? Which specific component constitutes the principal methodological novelty beyond the baselines?

  2. Can the authors provide a straightforward LLM-based autonomous research agent experiment with access to the same post-training tools and checkpoints?

  3. Given that "One-component'' outperforms "Full" for Kimi and Nemotron, and "No-annotation" outperforms "Full" for Kimi, what evidence supports the practical necessity of the proposed joint intervention and annotation mechanisms?

  4. The paper reports improvements on a composite evaluation metric consisting of some benchmarks. However, these same benchmarks are repeatedly evaluated during the auto-search process, and their results are provided as feedback to LLMs. Therefore, the reported improvements reflect search-time optimization rather than independent held-out generalization. Would it be possible for the authors to include an independent held-out test performed only after checkpoint selection, to establish whether the discovered post-training recipes generalize?

Reviewer_VnrpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces an LLM researcher/AI scientist which focuses on LLM post training. The levers for the scientist include data, training route (e.g., DPO, SFT), checkpoint choices and hyperparameters, and the scientist adaptively decides next steps based on historical results. Overall, the results from experiments are positive and validate the proposed approach. In terms of areas of improvement - the paper's scope is relatively narrow (just focusing on post training), and modeling novelty/contributions can improve. The paper's writing can also improve significantly - in most sections, the narratives lack a logical flow, making it hard for readers to follow/digest the details.

Critical strengths

The paper introduces an AI scientist focusing on post training. Experiments share results from different base LLMs, providing insights on how they evolve/search differently. Overall, this is a useful contribution which can make experimentation/post-training less human-intensive and faster.

Critical weaknesses

The paper's scope is limited to post-training, and there is limited discussion on the possible failure modes (e.g., through reward hacking). The focus on the three diagnostic questions, while being a reasonable choice, does not have enough justification around completeness/optimality. For example, how would the approach handle conflicting observations.

Another major concern is the paper's writing quality - its lacks a logical flow in most sections, making it difficult to read.

Questions for authors
  1. Can the authors share summarize insights around reward hacking that they saw in their experiments? the proposed approach includes an auditor for reward hacking but its hard to glean learnings around it from the paper's writing/presentation.

  2. Can the authors provide any learnings around when the three question framework failed/was not sufficient? E.g., did they see any conflicting patterns in their experiments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.