acceptodds
Under review as a conference paper at ICLR 2027

MESH: Recursive Self-Improvement through Evolving Search Control

Abstract

Language-model agents increasingly automate discovery by iteratively proposing, implementing, and evaluating candidate solutions. Each execution consumes part of a limited budget and reveals information about the task, and using this information to choose subsequent executions enables recursive self-improvement. An evaluation outcome not only measures the quality of a candidate but also provides evidence about the hypothesis that motivated it. However, existing tree-search agents rely primarily on scores and overlook how outcomes support or challenge the underlying hypotheses. To address this limitation, we propose MESH, a framework in which candidate solutions co-evolve with the explicit beliefs that direct them. At the high level, an agent maintains a global belief state of working hypotheses, each specifying a target candidate, a mechanism grounded in prior evidence, an intervention, and an expected outcome. At the low level, a search process develops candidate solutions. Beliefs compete for the execution budget according to the improvements they have produced. Each selected belief directs a bounded sequence of executions that iteratively apply and refine its intervention, allowing a promising idea to develop beyond a weak initial implementation. The high-level agent periodically revises all beliefs jointly based on the shared search history, so that even failed executions can refine, retarget, or replace hypotheses across the search. The resulting belief-guided recursive loop operates within a single discovery run and is grounded in realized progress rather than the agent’s self-assessment. Experiments on 19 tasks from MLGym and MLE-bench show that MESH attains the best score on 17 of them, including all eight MLE-bench competitions, when compared with linear, tree-structured, and graph-structured search controllers under the same worker models and execution budgets. Further analyses show that belief-directed executions improve the incumbent three times as often as score-driven ones, with the gap widening as score-driven search stalls, and that freezing the beliefs forfeits up to 59% of the gain.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Belief Update, Autonomous Discovery, Tree Search, Self-Improving

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Q1piOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper primarily explores and investigates the feasibility of guiding agents to reflect on and adjust their beliefs in real-time based on exploration and testing results, within the context of recursive self-optimization. I think the main contribution of this paper is MESH, a unified framework where beliefs guide candidate generation and search outcomes recursively revise beliefs. MESH is evaluated on 11 MLGym tasks and 8 MLE-bench competitions against baselines (Linear, MCTS, AIRA, MLEvolve, AB-MCTS-A). It outperforms them on most tasks, demonstrating its superiority.

Critical strengths

The proposed MESH framework unifies high-level beliefs with low-level search into a single paradigm, offering a novel optimization approach for recursive self-optimization in agents and opening a new avenue for this research direction. It moves beyond traditional static prompt engineering and isolated search methods, providing a principled and dynamic foundation for agent self-improvement.

Critical weaknesses
  1. The evaluation is restricted to 19 tasks from MLGym and MLE-bench, which are all fundamentally confined to machine learning engineering, code generation, and simple game-theoretic simulations. While the paper claims to propose a general recursive self-optimization paradigm, the framework is not tested on broader agentic domains such as web navigation (e.g., WebArena), operating system interaction, complex tool orchestration, or multi-agent collaboration.

  2. It remains unclear whether the belief revision mechanism can generalize to domains with sparse, subjective, or multimodal feedback (e.g., user satisfaction or physical world interactions), which are fundamentally different from the numeric metrics (accuracy, R², reward) used in these 19 tasks.

Questions for authors
  1. Regarding the initialization and prompt design of beliefs: Section 4.3 mentions that slots start empty and the first revision forms the initial beliefs from the outcomes of the first ν executions. Could the authors provide details on the exact prompt template used by the high-level agent U to generate these beliefs? How is the LLM guided to distinguish between mechanism (m), directive (δ), and expected outcome (e) without any prior examples?
  2. Regarding hyper-parameter sensitivity: The framework introduces several critical hyper-parameters: the number of belief slots K, the revision interval v, and the bounded commitment length L. The paper does not specify how these were chosen. How sensitive is MESH's performance to these hyperparameters? For instance, what happens if v is too small (frequent revisions without enough evidence) or too large (stale beliefs)?
Reviewer_TbQ3OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MESH, a framework for automated discovery that jointly evolves candidate solutions and explicit working hypotheses. Each belief specifies a target candidate, a proposed mechanism, an intervention, and an expected outcome. A high-level agent periodically revises these beliefs using execution feedback, while Thompson sampling allocates search effort between belief-directed exploration and an existing low-level search controller. Bounded multi-step commitments allow interventions to be refined beyond their initial implementations. Experiments on 19 tasks from MLGym and MLE-bench report improvements over linear, tree-based, and graph-based search controllers.

Critical strengths

The paper addresses an important limitation of score-driven search by treating experimental outcomes as evidence for revising the hypotheses that guide subsequent exploration. The integration of explicit beliefs, adaptive budget allocation, and bounded multi-step refinement provides a coherent framework for exploiting both successful and unsuccessful experiments. Another strength is the broad evaluation across MLGym and MLE-bench, with competitive baselines and shared worker models that help isolate differences in search control. The analysis of delayed improvements and cross-branch belief revision offers useful insights beyond final task performance.

Critical weaknesses

The main methodological concern is that the belief mechanism lacks sufficient validation. The paper does not systematically assess whether expected outcomes are supported by execution feedback or whether belief revisions improve predictive reliability. This makes it difficult to isolate the contribution of explicit beliefs from general adaptive planning. Moreover, retaining historical success statistics after substantial belief revisions may misrepresent the current hypothesis's productivity. The retirement criterion in Algorithm 1 also requires clarification, as previously successful slots may never satisfy it.

The empirical results are promising, but some conclusions need stronger support. The main comparisons use only three runs, with several marginal improvements. Ablations cover only three tasks and report best-of-three rather than mean performance. Furthermore, belief revision introduces additional LLM calls that are not reflected in the matched execution budgets.

Questions for authors
  1. How are the mechanisms and expected outcomes of individual beliefs assessed against execution feedback? Can the authors quantify whether successive revisions improve the reliability of these predictions?

  2. If (SkS_k) is cumulative, how can a previously successful slot satisfy the (SkS_k=0) retirement condition after subsequent failures?

  3. To what extent do the improvements arise from explicit mechanism-level hypotheses rather than adaptive planning and multi-step refinement? Can the authors extend the representation ablation to representative MLE-bench tasks?

  4. Since the main results average three runs while the ablations report the best of three, can the authors provide consistent mean performance and uncertainty estimates? Are the smaller improvements over AB-MCTS-A statistically reliable?

  5. What additional LLM calls, tokens, and runtime are required for belief revision? Does MESH retain its advantage under matched total computational budgets?

Reviewer_rbe3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MESH addresses how discovery agents should use experimental feedback to allocate a limited execution budget. It maintains explicit hypotheses specifying a target candidate, mechanism, intervention, and expected outcome. A reward-based allocator chooses between pursuing a hypothesis for several executions and delegating to AB-MCTS-A; periodic global revision updates hypotheses from shared outcomes. With fixed workers and matched execution counts, MESH reports the highest mean score on 17 of 19 MLGym and MLE-bench tasks. Component ablations and execution traces examine the roles of hypothesis representation, continued refinement, and revision.

Critical strengths
  1. MESH makes a useful distinction between a weak candidate and an idea worth further implementation effort. A hypothesis can target an earlier candidate, survive an unsuccessful first execution, and be revised using evidence from another branch. Charging its slot for every execution while rewarding actual incumbent improvements provides a concrete way to balance persistence against wasted trials. Retaining ordinary search alongside this mechanism is a sensible design choice.

  2. The evaluation is substantial for a controller paper: 19 task configurations, five competing controllers, and three-seed main results with shared workers. MESH leads on 17 tasks, including all eight MLE-bench competitions, where final submissions are selected by validation performance. The gain on aptos2019, from 0.192 for AB-MCTS-A to 0.297, is substantial. The targeted ablations and verbatim examples in Appendix C provide useful evidence beyond final scores, including recovery from failed implementations and information shared across hypotheses.

Critical weaknesses
  1. The reported robustness and component contributions are difficult to assess. Tables 1–2 average three seeds without uncertainty, while Table 4 uses the best of three seeds (Appendix A.5). Ratios of these selected maxima do not establish typical losses of 43–59%. There is also an unresolved discrepancy: CIFAR-10-L1 accuracy is not rescaled, yet Table 4’s best-seed MESH and AB-MCTS-A scores, 0.8292 and 0.7693, are below their Table 1 means, 0.831 and 0.789. If these are separate runs or settings, that must be stated. Paired seed-level results would also distinguish consistent improvements from near ties such as 0.819 versus 0.818 on spaceship-titanic.

  2. The mechanism analysis supports useful behavior more strongly than the claimed causal explanation. Figure 3’s threefold success-rate comparison covers four individual MLE-bench runs with adaptively selected targets and different histories; sharing a seed does not isolate the effect of beliefs. Delayed branch successes likewise do not establish what score-driven search would have achieved with those executions. The representation and revision ablations are relevant controls, but use selected maxima on three MLGym tasks, with no corresponding MLE-bench ablation. A matched, repeated comparison on a representative MLE-bench task, or against an equally budgeted reflective planner, would help establish the added value of explicit hypotheses beyond ordinary feedback-guided refinement.

Questions for authors
  1. Can you reconcile the CIFAR-10-L1 entries in Tables 1 and 4 and report paired per-seed main and ablation results?

  2. Can you provide controller parameters, revision prompts/model, context rules, baseline settings, and measured token/runtime costs?

  3. Can you repeat the representation/revision ablations on a representative MLE-bench task and clarify whether Figure 3’s success counts use validation scores?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.