acceptodds
Under review as a conference paper at ICLR 2027

An important Bottleneck in LLM Optimizers: Principle-Level Local Optima

Abstract

Feedback-driven LLM optimizers are increasingly used for evolutionary search, Bayesian optimization, heuristic design, and scientific discovery. Yet rapid initial gains often give way to prolonged plateaus despite substantial remaining evaluation budgets. We investigate **principle-level search collapse**, where an optimizer’s search becomes confined to a narrow set of design principles, even though it may continue to generate distinct candidates and achieve small improvements. We assemble a diagnostic benchmark spanning molecular optimization, network sentinel-node selection, and GPU-kernel optimization, and introduce a decision-tree protocol combining principle narrowing, prolonged performance plateaus, and gaps to attainable reference solutions. Across 161 assessable runs on the three problem domains, 78.3% receive support for collapse, including 55.3% with strong support. Candidate-level variation can nevertheless conceal restricted search at the principle level. We then test diversity injection, a common response to search stagnation, such as higher t, multiple LLMs, and web search, and show that it provides limited and inconsistent benefits: among 65 molecular runs, still 73.8% and 53.8% receive support and strong support respectively. We finally test reference-principle interventions and find that they improve some runs, but their benefits vary across models and tasks: access to a better principle does not guarantee its effective use. Our benchmark and diagnostic protocol provide a basis for evaluating future optimizers on principle-level collapse. Our findings highlight the need for online collapse detection and better mechanisms for generating and maintaining alternative principles during optimization.

Primary area:
optimization
Keywords:
LLM-based optimization, Principle-level search collapse, Scientific discovery

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_KrHeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates principle-level search collapse in feedback-driven LLM optimizers, where candidate solutions continue to vary while the search concentrates on a restricted set of design principles. It introduces a diagnostic protocol combining structural concentration, prolonged performance plateaus, and gaps to attainable reference solutions. The protocol is evaluated across molecular optimization, sentinel-node selection, and GPU-kernel optimization using multiple optimizers and LLM backbones. The paper also examines diversity injection and reference-principle interventions, finding that their benefits vary across models and tasks.

Critical strengths

1.Broad empirical investigation. The study examines multiple optimization domains, algorithms, and LLM backbones, providing evidence beyond an isolated example of search stagnation.

2.Useful diagnostic controls. The experiments include repeated runs, negative controls, and kernel-specific threshold and diagnostic-window sensitivity analyses. These help distinguish concentration near competitive solutions from concentration accompanied by substantial performance gaps.

3.Informative intervention results. The reference-principle experiments illustrate that receiving a promising direction and successfully developing a competitive implementation are distinct challenges. The paper reports both improvements and unsuccessful interventions, making the empirical picture more informative.

Critical weaknesses

The incremental contribution over existing work is insufficiently established. The distinction between candidate diversity and principle diversity closely relates to established ideas concerning behavioural diversity, premature convergence, and deceptive search, as the paper itself acknowledges in Section 2. The additional insight provided by the proposed diagnostic framework therefore needs stronger justification. In particular, the experiments do not include direct comparisons with relevant route-preserving approaches such as K-Search or a principle-level archive. Such comparisons would help establish what remains unresolved beyond the observation that increasing candidate variation does not consistently prevent stagnation.

The diagnostic measurements do not sufficiently establish the formally described failure mode. Equation (1) describes the principle distribution governing candidate generation, whereas Equation (3) measures label frequencies among historical top-performing candidates. Although the authors acknowledge these measurements as proxies, concentration after selection does not necessarily imply concentration in the generation process: an optimizer may continue exploring multiple principles while only one enters the retained set. Furthermore, low final entropy in the structural diagnostic branch does not establish that the same principle remained dominant throughout the plateau. A reference gap can also reflect insufficient optimization within an otherwise promising principle. The recent-proposal analysis for kernels partially addresses these concerns, but the connection between the diagnostic labels and principle-level locking remains insufficiently validated across domains.

The plateau criterion requires clarification when observed and planned budgets differ. The appendices define the remaining-budget fraction as ((B-b_{\mathrm{plat}})/B), where (B) is the planned budget, while plateau onset is calculated relative to the observed endpoint (T_{\mathrm{obs}}). If a run terminates early, this criterion can classify it as having a long plateau even when performance continues improving until its final observed evaluation, because unused budget contributes to the fraction. The main-text kernel discussion also refers to observed attempts, creating ambiguity about the implemented criterion. The authors should clarify whether all assessed runs exhaust their planned budgets and, otherwise, separate observed stagnation duration from unused budget and report whether this changes the classifications.

Questions for authors

See Weaknesses

7 comments
AuthorsOpenReview ↗

Response to KrHe [1/2]

We thank the reviewer for recognizing the cross-domain evidence and diagnostic controls. We clarify the principle-aware baseline, add within-plateau identity tracking, and report budget coverage below.


The incremental contribution over existing work is insufficiently established. Direct comparisons with route-preserving approaches such as K-Search or a principle-level archive would help.

Response:

We already include PiEvo as a principle-level archive baseline across all three domains in paper. PiEvo [1] explicitly retains design principles and their associated records, maintains beliefs over the principles, and selects and revises them during optimization. The PiEvo results examine why retaining principles may still be insufficient: a direction can remain available without receiving sustained productive development, and predictable feedback can leave an unproductive direction unrevised. This is an important distinction between having an archive of principles and effectively using it to sustain exploration.


Concentration in historical top-performing candidates does not establish concentration in generation, or that the same principle remained dominant throughout the plateau.

Response:

We now examine both actual proposals and structural identities throughout the observed plateau. The proposal-versus-archive analysis covers 65 original ExLLM runs, with five seeds per setting. Early/late windows contain 1,000 saved attempts, retaining repetitions among valid candidates. The contemporaneous selected-50 uses the original mixed single-/multi-objective selection rule. Support overlap C=∑j:qj>0pjC=\sum_{j:q_j>0}p_j measures proposal mass within this selected structural support; distribution overlap O=∑jmin⁡(pj,qj)O=\sum_j\min(p_j,q_j) also compares frequencies.

Table 1. Molecular proposal concentration, repeated here for clarity.

BackboneEarly proposal N_effLate proposal N_eff
GPT-5.460.117 ± 61.4973.213 ± 0.803
Claude Sonnet 520.467 ± 12.4593.275 ± 2.075
DeepSeek V4 Flash22.301 ± 7.1031.515 ± 0.860
Qwen3.5 35B58.901 ± 40.69624.397 ± 11.294
GPT-4o108.451 ± 23.7543.014 ± 1.002

Table 2. Molecular proposal overlap with the contemporaneous mixed-selection selected-50.

BackboneEarly CLate CEarly OLate O
GPT-5.40.566 ± 0.1220.820 ± 0.0870.494 ± 0.1010.755 ± 0.037
Claude Sonnet 50.645 ± 0.1250.950 ± 0.0330.622 ± 0.1310.869 ± 0.094
DeepSeek V4 Flash0.527 ± 0.1100.974 ± 0.0420.397 ± 0.0990.963 ± 0.047
Qwen3.5 35B0.383 ± 0.1400.449 ± 0.1380.333 ± 0.0700.394 ± 0.101
GPT-4o0.379 ± 0.0450.801 ± 0.0650.373 ± 0.0360.801 ± 0.065

Generation itself narrows for GPT-5.4, Sonnet, DeepSeek, and GPT-4o, and substantially overlaps the selected scaffold region. Thus, the finding is not confined to the historical selected set.

Table 3. Residence Hall proposal concentration and overlap.

BackboneEarly proposal N_effLate proposal N_effLate CLate O
GPT-5.461.277 ± 10.78424.350 ± 5.4960.985 ± 0.0030.830 ± 0.059
Claude Sonnet 522.011 ± 3.89210.020 ± 1.1990.992 ± 0.0070.629 ± 0.041
DeepSeek V4 Flash50.456 ± 12.74212.975 ± 2.3420.805 ± 0.0430.443 ± 0.115
Qwen3.5 35B66.235 ± 14.11080.492 ± 33.8710.738 ± 0.1090.450 ± 0.075

GPT-5.4, Sonnet, and DeepSeek narrow their generated node pools, with substantial overlap in elite node support. Node-support overlap does not require identical node frequencies.

Does the same direction remain dominant during the plateau? We divide the proposals made after plateau onset into consecutive windows of 1,000 attempts and check what is most frequently generated in each window.

AuthorsOpenReview ↗

Response to KrHe [2/2]

Table 4. Does the most common molecular scaffold change during the plateau?

BackboneRuns with the same scaffold leading every windowOther assessed runsProposals using the plateau's most common scaffold
GPT-5.44/5Two scaffolds take turns leading in 1 run60.1% ± 14.4%
Claude Sonnet 53/5Two scaffolds take turns leading in 2 runs65.1% ± 17.4%
DeepSeek V4 Flash3/5Two scaffolds take turns leading in 2 runs79.8% ± 21.3%
Qwen3.5 35B3/4Two scaffolds take turns leading in 1 run38.4% ± 17.6%
GPT-4o2/2None81.5% ± 2.5%

For example, GPT-5.4 keeps the same scaffold as the most common one in every window in four of five runs. In the remaining run, two different scaffolds lead at different times. The last column is the fraction of valid proposals using the single most common scaffold over the whole plateau. One Qwen run and three GPT-4o runs have plateaus too short for this comparison. Thus, most assessed runs repeatedly favour the same scaffold, rather than continually switching to new directions. The exceptions alternate between two window-leading scaffolds and provide weaker evidence.

Table 5. Do dynamic-system proposals keep using the same main nodes during the plateau?

TaskBackboneNode-pool overlap: neighbouring windowsNode-pool overlap: later windows vs first window
Residence HallGPT-5.478.5% ± 9.1%71.6% ± 6.2%
Residence HallClaude Sonnet 589.4% ± 6.1%70.4% ± 15.6%
Residence HallDeepSeek V4 Flash88.1% ± 6.3%46.5% ± 13.1%
Residence HallQwen3.5 35B85.8% ± 5.4%86.2% ± 7.2%
DolphinGPT-5.474.1% ± 1.8%56.6% ± 6.8%
DolphinClaude Sonnet 573.0% ± 6.0%61.4% ± 4.3%
DolphinDeepSeek V4 Flash83.0% ± 12.9%74.0% ± 17.3%
DolphinQwen3.5 35B82.3% ± 7.1%80.2% ± 5.4%

For each window, we take the ten most frequently proposed nodes. Overlap is the number of shared nodes divided by the number of nodes in either pool: 100% means identical pools. The main node pools overlap substantially throughout the plateau: overlap between neighbouring windows averages 73.0%–89.4% across settings. Later windows also retain substantial overlap with the first window (46.5%–86.2%), with some changes accumulating over time. Overall, the proposals repeatedly draw on largely overlapping node pools rather than continually replacing them with entirely different ones.

GPU kernels: generation during the plateau. For kernels, the diagnostic set is top 10. In all 49 kernel runs diagnosed as collapsed, we looked at every valid kernel generated after the plateau began. In the median run, 96% of these newly generated kernels use the same route that dominates the final ten fastest kernels (25th percentile: 88%). When we split each plateau into consecutive windows of 20 newly generated kernels, the dominant route falls below half of a window in only 4 of the 49 runs, and the most common route changes between windows in only 5 runs. About 30% of the attempts after the plateau produce invalid kernels (they fail to compile, crash, or give wrong results); these have no route label and are not counted.

GPU kernels: is the gap due to under-optimization within a promising route? For kernels we can separate the two explanations. For each of the 49 collapsed runs, we compare the run's own best latency with the best latency that any of our 455 kernel runs achieved on the same route. In 46 of the 49 runs, even that best latency is at least 2× slower than the expert kernel (median 23×), so further optimization within the route, as far as any of our runs got, would not close the gap. The run itself is a median 2.25× slower than that best latency, so some within-route under-optimization is also present. On a logarithmic scale, the route's limit accounts for a median 78% of a run's gap to the expert, and within-route under-optimization for the remaining 22%.


Please clarify whether all runs exhaust their planned budgets, separate observed stagnation from unused budget, and report whether this changes the classifications.

Response: Almost all of our runs completed their planned budgets. A small number of runs on molecular optimization tasks were stopped early because their uniqueness dropped to 10% or lower, with no improvement in the top-10 scores for 50 consecutive generations (equivalent to 2,500 generations of candidates). Continuing these runs would incur extreme time and API costs with almost no benefit. Therefore, we assume that their top-10 scores would remain unchanged until the full budget was exhausted. This only happens in molecular optimization task; all node and kernel optimization runs completed their full budgets. We will clearly state this in the paper.

Reference: [1] Pu, et.al . "Principle-evolvable scientific discovery via uncertainty minimization. ICML 2026

Dear Reviewer,

We are nearing the end of the discussion phase. Please read the authors' responses, reply to acknowledge their feedback, and indicate whether your initial concerns have been addressed (along with any score adjustments, if warranted).

Thank you for your dedicated efforts and valuable insights!

Best, Your AC

Reviewer_1sccOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates principle-level search collapse in LLM optimizers and establishes a diagnostic framework. To validate the proposed diagnostic framework, a cross-domain diagnostic benchmark is assembled. After that, the authors examine evidence on how restricted search persists. Also, the authors evaluate current remedies such as diversity injection and reference-principle interventions.

Critical strengths
  1. The paper identifies a real and systematically overlooked level of failure, and provides an operational distinction.
  2. The use of negative controls to combat the "diversity = good" fallacy shows better methodological awareness than comparable work.
  3. The evidence base is genuinely broad and honest about unfavorable results.
  4. The PiEvo case study demonstrates the right way to do multi-method triangulation.
  5. The reproducibility investment is solid.
Critical weaknesses
  1. The Narrow condition does not separate success from failure, so the verdict reduces to a performance threshold. Table 15 in this paper shows that DeepSeek + L2 reaches the reference solution (Top-1 = 4.699 ± 0.000) in all five seeds with Neff = 2.15–2.39; Sonnet reaches 4.617 with Neff = 2.16. The molecular Narrow criterion is Neff ≤ 3. The best runs in the paper therefore satisfy Narrow and escape a collapse verdict only because Headroom fails (Top-1 ≥ 4.3). In Headroom ∧ Plateau ∧ Narrow, the only condition doing discriminating work is then a performance cutoff, and the diagnostic becomes operationally indistinguishable from "this optimizer performs poorly and stopped improving early". A success control — the Neff distribution of runs that reach the reference — is missing, and that is the central construct-validity gap.

  2. The diagnostic is anchored on an attainable reference solution that real optimization problems do not have — and in the molecular experiments that anchor is already misaligned. Only Headroom requires f(x_ref). On real problems (a new kernel, a new scheduling heuristic, a new target) no reference exists — if one did, there would be nothing to optimize. Dropping Headroom leaves Narrow ∧ Plateau, and Plateau is near-vacuously true for any converged evolutionary process, as the 200-evaluation kernel setting illustrates (dominant-route share 0.96–0.99). Worse, the paper does not hold this anchor clean either: baseline Top-1 is 3.97–4.19, whereas the 4.699 reference is the best-init figure in the original ExLLM study (whose worst-init result is 4.229 ± 0.050, and where MOLLEO also reports 4.699 ± 0.000 under best-init, suggesting a retained initial elite rather than a search-discovered solution). If the baselines start from a weaker initialization, Headroom is partly an initialization artifact rather than evidence of a missed principle. As it stands, the framework is a post-hoc attribution tool for benchmarks with expert implementations and known optima; it cannot be applied to real problems, and it cannot deliver the "online collapse detection" the paper calls for.

  3. The prescription is not supported by the paper's own experiments, and the most natural baseline is absent. From the start, a content-free L1 nudge yields a geometric-mean gain of 1.169, better than the oracle L2 (0.833, worse in 21/46 runs); at onset, L3 (0.986) also trails L1 (1.110). If insufficient principle coverage were the bottleneck, naming the correct principle should not hurt. The effective ingredient looks more like "break the current mode" plus implementation capability than "which principle is missing". In addition, there are no paired tests or confidence intervals anywhere (n = 5, SD 0.1–0.37, and every §4.2 difference sits within one SD), so "limited and inconsistent benefits" rests on differences the design cannot detect. Most importantly, §2 argues that a QD archive is precisely the missing selection-side mechanism, yet §4 tests only temperature, multiple LLMs, web search, and hints — a principle-level archive is never tested, although it is the most direct and most necessary test of the proposed remedy.

Questions for authors

Q1. Please report the Neff distribution for runs that reach the reference versus runs that plateau far below it, together with the separation achieved by the decision rule (e.g. AUC). If the two overlap, as Table 15 suggests, how do you establish that narrowing is a cause rather than a corollary of convergence?

Q2. How is this framework meant to be used on a real problem with no reference solution? Please define and evaluate a reference-free variant: what verdict does Narrow + Plateau alone give on your data? Can Headroom be replaced by an attainability proxy, such as the coverage ceiling of a QD archive or the best member of a held-out principle family? And since b_plat is defined retrospectively against the endpoint, it can be computed offline but not online — how would you turn it into a detector that runs during optimization?

Q3. Which ExLLM initial population did the molecular experiments use (best / random / worst)? Is the 4.699 reference simply the best-init elite (MOLLEO reports the identical 4.699 ± 0.000 under that setting)? Please also explain why your reproduction lands 0.2–0.5 below the published numbers.

Q4. L2 is defined as principle-level and candidate-free, yet DeepSeek hits 4.699 ± 0.000 in all five seeds from both injection times — that looks closer to a pointer to the answer. Please give a semantic-distance measure between hint and reference, plus a control with a partially correct or deliberately wrong hint. And why does the oracle L2 underperform the content-free L1?

Reviewer_YECBOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3(+2)
Read review
Paper summary

This paper proposes that a bottleneck in LLM-based optimizers is that they get stuck proposing designs based on the same principles, and show empirically that a variety of sensible strategies aren't very effective.

Critical strengths

The paper's main strength is qualitatively suggesting a hypothesis for why LLM-based optimizers struggle.

Critical weaknesses

Construct validity: the paper's claims rest on the definition of a "principle", which the authors don't really define in any way. To be fair, this is basically an impossible task (I certainly can't give a good definition). However, it makes a "principle-level collapse" basically impossible to actually confirm or deny. One could always look at the things the LLM is not changing, call those "principles" and diagnose principle-level collapse. Or do the opposite: look at things which don't change and call those the "principles", so no collapse has occurred. So it isn't clear how one can apply this paper's framework in a meaningful way to new problems.

Obviousness: if one strips aside the the definition of "principles", the diagnostic framework seems to essentially be "is the optimizer doing a bunch of similar things and not improving", which in my opinion is obvious (i.e. not a significant contribution for a conference paper).

Possible flaw in the diagnostic set definition: the diagnostic sets are all essentially top-N elements seen so far. In optimization problems with broad local optima, there may be N very similar (and similarly good) points clustered around the local optimum. An optimizer may find these points, correctly move away from the local optimum, but still have the top-N element list not change as it takes a while for its exploration of other parts of the space to move on.

Explore-exploit: the thesis of the paper seems to essentially be that if the LLM fails to consider broadly different solutions that is a problem we should correct. I see this as basically an instance of the explore-exploit trade-off problem, which has been intensely studied and doesn't have easy solutions. The right balance between explore-exploit depends on the regret metric, remaining budget, risk tolerance, and beliefs about the relative payoffs of the different options. Behaviour that looks like "continually applying the same principles" could be a rational response to the problem structure.

Questions for authors

As you can tell from the previous section, my main issues with the work are the concepts it defines. I presume you won't be able to precisely define a "principle", so instead perhaps it would be useful to hear your strongest arguments for why this paper would be of interest to the ICLR community without this being precisely defined?

Comments on the paper

1 comment

Dear Reviewer YECB, 1scc, KrHe,

Thank you for your valuable feedback! We have addressed your comments and streamlined our rebuttal to make the responses easier to follow.

We would greatly appreciate it if you could take a moment to review our responses and share any further thoughts before the discussion deadline.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.