acceptodds
Withdrawn from ICLR 2027

OPTScientist: Toward Recursive Self-Improving Agents via Automated Optimizer Discovery

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Designing optimizers for modern deep learning remains a challenging scientific problem, requiring the joint consideration of optimization geometry, state dynamics, numerical stability, implementation constraints, and empirical generalization. Existing automated discovery methods typically search either over unconstrained code spaces or within narrowly parameterized optimizer families, trading flexibility for validity and interpretability. We introduce OPTScientist, a theory-guided multi-agent framework that casts optimizer discovery as a collaborative scientific process. Specialized agents propose hypotheses, instantiate candidate optimizers in an extensible typed DSL, and evaluate them through transformer pretraining, while a shared knowledge system accumulates candidate lineages, experimental outcomes, failure modes, and reusable lessons to guide subsequent search. The DSL provides compiler-checked validity while supporting conservative extensions when existing primitives limit promising hypotheses. Combining multi-fidelity evaluation, cross-scale validation, and controlled interventions, OPTScientist identifies proxy hacking and translates diagnosed failures into executable constraints. The system discovers eighteen optimizer designs that achieve lower target-scale validation bpb than all baselines in our main benchmark, with selected designs also improving downstream performance and transferring to vision after learning-rate retuning. These results position OPTScientist as a concrete step toward recursive self-improvement, where experimental evidence refines the knowledge, search space, and constraints that guide subsequent discovery.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Self-Improving, Multi-agent, Optimizer Design, Evolutionary Search

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_oeacOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper builds a multi-agent system that searches for Transformer pretraining optimizers. Candidate update rules are written in a typed DSL, screened for scale risk and learning-rate bounds before any training, then evaluated in a three-stage proxy cascade from 50M to 200M parameters; selected candidates are trained at 1.1B parameters against Muon. The motivating observation is that candidates which look better on the small proxies often lose that advantage at 1.1B, which the authors call proxy hacking. The paper attributes these reversals to parameter-norm-dependent amplitude scaling, supports that attribution with controlled edits and a dose-response sweep, and turns the diagnoses into admission rules. It reports eighteen designs that beat Muon on validation bpb at 1.1B under a single training seed, one of which (AnnealSink) is validated over five seeds, and shows transfer to a CIFAR-100 ViT and two other language-model configurations. The framing is recursive self-improvement: the research process is said to improve the knowledge, search space and constraints that guide the next round.

Critical strengths
  1. The failure analysis is causal rather than correlational, and that is the part of the paper I find convincing.

Figure 4 is the strongest evidence in the submission. Replacing AnnealSink's amplitude-correction numerator with the parameter norm degrades it severely (off-scale). Removing the additional parameter term or lowering the learning rate from 0.045 to 0.020 improves the affected candidates. Sweeping the parameter-term coefficient over {0, 0.5, 1, 2} at a fixed rate produces a monotone degradation. This connects a specific program dependency to a specific failure, and it is the kind of evidence that a correlation between program features and proxy scores could never supply. If the paper were only this section plus the 1.1B confirmation, I would argue for acceptance.

  1. The full-scale evaluation is real work and is reported with appropriate caution.

Training at 1.1B parameters for 21k steps against matched-seed Muon, rather than stopping at the proxy, is more than the surrounding literature usually does. The paper also keeps a rejected candidate that succeeds at the target scale (AnnealSink-S, rejected for declaring a rate of 0.025, yet improving by 1.0e-3), and reports that AnnealSink is slightly worse on a faithful GPT-2 configuration. Both are negative results that the authors could have omitted, and the limitations section is honest about the gate's conservatism.

Critical weaknesses
  1. The phenomenon the paper is organized around is never measured, and the one experiment that could test its thesis measures something else.

The paper claims that proxy-to-target reversal is the central obstacle in optimizer autoresearch. The evidence offered is four representative candidates in Figure 2, three of which reverse at 1.1B, plus qualitative discussion of candidate histories. No base rate is reported: among all candidates that passed the proxy cascade, how many lost their advantage at the target scale? What is that rate for Generation 1 and Generation 2 separately? Without a denominator, the paper establishes that reversals occur and can be diagnosed, not that they are the dominant failure mode that motivates the architecture.

The only cohort with target-scale labels runs against the framing. Of 28 fully evaluated designs, 17 of the 18 admitted designs beat Muon and 1 of the 10 rejected designs does, which is a 93% accurate gate and a 5.6% reversal rate among admitted candidates. The authors describe this cohort as a catalog of successful designs rather than an evaluation cohort, which is precisely the point: the denominator has been removed by curation.

The ablation is misaligned with the thesis in the same way. Table 2 reports three quantities, all of them proxy measurements (best proxy score, number of proxy wins, gate pass fraction). But the claim under test is that shared experience amplifies proxy-specific bias. To test that, one should ask whether removing shared knowledge changes the target-scale transfer rate of the candidates it produces. As reported, the ablation shows that knowledge raises the proxy score, which is the direction the paper itself identifies as suspicious. The differences involved (1.6274, 1.6262, 1.6255) also sit inside the run-to-run standard deviation of 0.0011 to 0.0023, with no test reported.

  1. The headline result does not survive the statistics the paper itself supplies, and the search baselines are never evaluated where it matters.

All eighteen designs in Table 4 are single-seed: the paper states that every result there uses training seed 0. The improvements are 1.0e-3 to 2.7e-3 in validation bpb. From Table 9 (paired difference 0.00260, t = 3.44, n = 5) the standard deviation of the paired difference is about 1.7e-3, which is close to the between-seed standard deviation of Muon (1.99e-3) reported in the same table. Pairing on the training seed therefore buys little variance reduction here, and a single-seed comparison at this effect size is roughly a 1.5-sigma test. The eighteen designs were then selected as winners from a much larger pool, with no multiple-comparison control, so a substantial fraction of the individual claims are expected to be noise. Only four designs have repeated-seed support at all: AnnealSink over five seeds (paired t-test, p = 0.026) and three others over three seeds each.

The one design with proper support is also the one that does not improve downstream. AnnealSink reaches a lower validation bpb than Muon but a CORE score of 26.186 against 27.349, and an identical Avg(9) of 46.09, while its BoolQ result drops by 14.8 points relative to a 9.6-point gain on CSQA. The abstract's hedge ("selected designs also improving downstream performance") is fair, but the paper's own best-supported optimizer is a validation-loss result rather than a capability result.

Separately, the count of eighteen is inflated by near-duplicate variants. E11 and E12 differ only in the weight-decay coefficient (0.25 against 0.30). E6 and E8 differ only in the constants of the momentum schedule b(t; a, h), with a = 300 against a = 400. The KronGate family contributes four entries that differ in coefficient and schedule. E1 to E4 are four decompositions of one skeleton: momentum annealing plus alternating row/column normalization. The number of independent mechanism families is closer to seven than to eighteen.

Finally, the search baselines are compared only on proxy score. Table 2 gives 1.6273 for the full system against 1.6207 and 1.6215 for OpenEvolve over the DSL and over code, a difference of about 0.4%. The most informative experiment in the paper is missing: take OpenEvolve's best candidate to 1.1B. If it reverses, the gate is justified by direct evidence; if it does not, the motivation weakens considerably. Either outcome is worth more than the comparison that is reported, and the comparison that is reported uses the metric the paper spends its introduction arguing against.

Questions for authors

1.Among all candidates that were trained at 1.1B, what fraction passed the proxy cascade and then failed to beat Muon? Please give the count and the rate by generation, including candidates that are not in the Table 4 catalog. If the target-scale labels do not exist for Generations 1 and 2, please say so explicitly.

2.How many of the eighteen designs have been evaluated on five or more seeds? For those that have, what is the result after correcting for the eighteen comparisons?

3.Which component of the admission rule carries the discriminative power? The learning-rate bound is (0, 0.02], the default declared rate is 0.02, and Muon's own best grid point is 0.014 (Table 8), so I would like to know whether the rule does more than enforce the calibrated default. Can it be tested prospectively on a pool of candidates that was not used to derive it?

4.In the ablation without shared knowledge, what is the target-scale transfer rate of the candidates produced? The current ablation reports only proxy metrics.

5.Appendix B.2 notes that the proxy and target differ in model size, training horizon, effective batch size and corpus. Can a single-axis comparison isolate scale, for instance 1.1B at a short horizon, or the largest proxy at the full 21k steps, so that the scale attribution in Section 3.3 is not confounded with the corpus and batch changes?

6.Which parts of the loop are agent-authored and which are author-authored? In particular, are the admission rules in Eq. 7 written by the specialists from diagnosed failures, or written by the authors and then versioned, and who decided the transitions from Generation 1 to 2 and from 2 to 3? I ask because the recursive self-improvement framing depends on the answer, and Section 4.3 and Appendix B.5 read differently on this point.

Reviewer_SxvKOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

Summary: This paper considers the well studied optimizer learning problem, and proposes a multi-agentic solution to optimizer search. There is a rather elaborate set of specialist agents that share knowledge and propose optimizers, which are checked by a DSL, and successful proposed optimizers go through a ladder of validation from 1900 steps on a 49.8M LM => 21K steps on a 1.1B LM. The results show it discovers some new optimisers that show some improvement on muon.

Critical strengths

S1: Optimizer learning is a tough and important problem.

S2: Agentic solutions to AI Scientist for optimizer learning is promising.

S3: Some heuristics to mitigate the so called proxy gap (Godohart’s law for optimizer learning) where the meta-optimizer hacks the surrogate objective and produces one that gets good score on a proxy metric, but doesn’t do well on the real metric.

S4: Some improved results on LM training. And some demonstrations of transfer

S5: Some results on transfer of the optimizers to new architectures.

Critical weaknesses

Weaknesses:

W1: Overclaim. RSI is the central claim but it’s not demonstrated. The paper mentions three generations of a progressively more sophisticated multi-agent harness for optimizer research. However, as far as I can tell, this appears to be the result of successive improvements by the authors based on lessons learned; not the result of an autonomous self improvement procedure.

W2: Reproducibility. The optimizer discovery procedure and and cross-generation evaluation are insufficiently specified. In general, Sec3+4 are far from specifying the methodology clearly enough for reader to have an intuition, let alone enough for reproduction. To mention a few: (1) What is the lifecycle of parent candiate selection, and child candidate proposal? (2) What is the history and lessons exposed to each agent? (3) how are candidates selected for expensive full evalution after the three stage proxy ladder? (4) What are the complete per-generation inputs, outputs, and search budgets?

Parts of the evalution are almost as mixed up: (1) The final system is compared to OpenEvolve and some ablations, but no controlled comparison of Gen1, Gen2, Gen3. So then why introduce them at all? (2) The 18 successful optimizers are pooled from multiple generations and ablations, making the final claim about a certain system’s discovery effectiveness impossible to assess.

W3: Novelty and validation. Proxy-target mismatch (imperfect surrogate objectives) is widely known in communities around AutoML, HPO, learned optimizers. The mitigation proposed here seems to be to identify some specific failure families, and encode them as static rejection rules. Which is neither very novel nor general. The empirical validation of this part is 17/18 admitted candidates outperforming Muon vs 1/10 rejected candidates. But this seems to be post-hoc with the actual failures having been used to define the rules. So it does not establish generalization.

W4: Limited and inconsistent evidence of optimizer generalisation across problems. Optimizer generalization (architectures, datasets, scales, recipes) is a major long term challenge for optimizer learning research. The evaluation here is dominated by a single 1.1B nanochat/fineweb setup. Evidence of optimizer generalization beyond this is limited: (i) AnnealSink slightly improves val loss on a 600M llama style model. But underperforms Muon on a 1.04B GPT2 model across 3 seeds. (ii) Vision transfer is limited to one VIT/CIFAR config with optimizer-specific LR selection. (iii) Only a subset of the 18 discovered optimizers receive cross-problem evaluation. Moreover, the static rule checkers that mitigate the proxy mismatch do not particularly address the source of cross-problem generalization failures.

W5: Optimizer quality is incompletely characterized. Optimizer learning is a multi-objective problem that is very tricky to specify and measure. The meta-objective here is fixed-horizon validation bpb. But this cannot distinguish several related optimizer learning meta-objectives of interest: (1) Faster training loss convergence vs lower final training loss (2) Better empirical optimization vs better validation generalization. (3) Short vs long horizon convergence. (4) Better quality per training step vs better quality per unit of wall clock compute.

The paper’s proxy and target settings change scale, training horizon, data, etc simultaneously. So it’s hard to say that results uniquely and causally isolate scale-dependent update mechanisms.

W6: Efficiency. For expensive optimizer learning, one cares very much about the achieved meta-objective (val-bpb in this case) as a function of wall clock time spent on optimizer search. One can usually further advance the meta-objective by throwing more compute at the optimizer search, distinctly from methodological improvements in optimizer search. There is no evaluation here to show in compute matched comparison to baselines.

W7: Writing. (See also W2). Section 1+2 read OK. However, Section 3+4 are almost incomprehensible. Lots of jargon is introduced without definition, context, explanation or intuition. Topics are introduced in random order rather than a logical scientific sequence. The narrative interleaves chronological author experience, the flow of the optimizer search, the flow of the multiple generations of optimizer search algorithm, and early anecdotes from the results. As one example, the last paragraph on Pg 4 is completely incomprehensible.

Questions for authors

Re: W1:Please distinguish autonomous agent actions from human intervention, if any, and specify the mechanism if any by which generation n produces generation n+1.

Reviewer_3VMVOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

OPTScientist searches a typed optimizer DSL with multiple agents, scores candidates on a three-stage proxy cascade (50M to 201M parameters), screens them with a static "scale-risk" gate, and validates survivors on a 1.1B nanochat run. The headline result is 18 designs that beat Muon on target-scale validation bpb, the best by 0.0027 (Table 4). The proxy-hacking diagnosis is useful, but the system claims lack support, so I recommend rejection.

Critical strengths
  • The 28-design cohort with full 1.1B evaluations (Figure 3) measures proxy-to-target reversal directly, and Figure 4 isolates three failure causes.
  • Tables 9 and 10 give five-seed paired results for AnnealSink with a t-test (p = 0.026 on bpb).
  • Appendix A.1 writes out every discovered update rule.
Critical weaknesses
  • The "eighteen designs" claim rests on seed 0. Muon's five-seed SD is 0.0020 (Table 9); 13 of the 18 gains in Table 4 fall below 0.0026, the only gain that five seeds confirm, and four fall below one SD. Fix: three seeds per design.
  • The flagship AnnealSink comes from Generation 1 (p. 14: single agent, no DSL, no gate). The only design attributed to a gated arm is E14 (gate v1, gain 0.0021); none is attributed to the v2-gated full system. Fix: a provenance column in Table 4 and the best target-scale result per configuration.
  • Table 2 compares search systems by the proxy score the paper calls hackable; no OpenEvolve candidate is reported at 1.1B, and the ablation gaps (0.0012 and 0.0019, n = 3, SD up to 0.0023) do not support "reduces search performance" (p. 8). Fix: full-fidelity runs of each arm's best candidate.
  • The gate is validated on the failures that produced its rules (p. 9); the named rejections fail on learning rate (0.035, 0.045) while 17 of the 18 discovered designs keep the default 0.02. Fix: report how many of the 10 rejections fail on the LR rule alone.
  • Downstream claims are unsupported: AnnealSink trails Muon on CORE in Table 1 (26.2 vs 27.3), the tuned five-seed gap has p = 0.145 (Table 9), and Muon's default five-seed CORE is never reported. Fix: add Muon's default CORE to Table 9.
  • The abstract and conclusion omit the faithful GPT-2 transfer, where AnnealSink loses on 3 of 3 seeds (Table 11), and Table 8's Lion, Sophia, SGD, and RMSProp baselines have no results.
Questions for authors

CliffSearch (Mroueh et al., arXiv 2604.01210, April 2026) already evolves Muon variants with LLM agents on nanoGPT under reviewer gates for correctness and originality. ASI-Arch (Xu et al., COLM 2026) pairs the same proxy-then-verify design with a distilled knowledge base. This paper adds the typed DSL with scaling tags, the static gate, and the reversal cohort. Neither work is cited, nor are NorMuon (Li et al., arXiv 2510.05491), the Table 1 baseline, FISMO (Xu et al., arXiv 2601.21750), whose name "FISMO-lite" (Figure 3) borrows without a definition, OpenEvolve, or DCLM for CORE.

The arithmetic of Tables 4, 5, 7, 9 and 10 holds. Muon's seed-1 run is an outlier at both learning rates (0.7411), which leaves the baseline six times noisier than AnnealSink, unexplained. Section 4.3 does not say whether the agents or the authors wrote the gate rules between generations, and the interventions in Figure 4 read as author-run; say who did it, or retitle.

AI-written prose: "position OPTScientist as a concrete step toward recursive self-improvement" (p. 1); "useful discoveries are not confined to a single update rule or benchmark metric" (p. 9).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.