acceptodds
Desk rejected at ICLR 2027

BeliefOPT: Recursive Self-Evolving Agents for Autonomous Research

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Research is the disciplined conversion of experience into belief. Autonomous research agents accumulate experience through iterative experimentation, creating an opportunity to continually improve their capacity for discovery. Realizing this potential requires turning experimental outcomes into reliable knowledge and revising that knowledge as new evidence emerges. Yet lessons inferred from past success may reflect contingent advantages or unreliable measurements, allowing plausible but unsupported claims to shape subsequent research. We introduce BeliefOPT, a framework for autonomous research that organizes experience into falsifiable, revisable beliefs about when and why one choice should outperform an alternative. A research scientist distills experimental evidence into beliefs, while an engineering squad uses them to explore candidate solutions and returns evidence that informs their subsequent selection and revision. This interaction couples the evolution of the belief store with the research process it guides, supported by an evaluator that expands test coverage as experimentation proceeds. We use GPU kernel optimization as a representative setting for studying autonomous research. On 26 GPU kernels from SOL-ExecBench on NVIDIA B200, BeliefOPT operates from an empty belief store without human intervention, achieving a geometric-mean speedup of 23.9x, exceeding OpenEvolve by 24% and CodeEvolve by 6%. Exploratory ablations suggest that organizing relationships among beliefs improves search at larger budgets, while accumulating additional experience or seeding expert advice does not consistently help. These findings support an approach to autonomous research in which accumulated knowledge remains open to testing and revision, allowing experimental evidence to continually reshape both what an agent believes and how it investigates.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
LLM Agents, Self-Evolving Agents, Agent for research, Recursive Self-Improvement

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_fY7pOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces BeliefOPT, a framework for autonomous research agents that converts experimental experience into structured, revisable beliefs and uses these beliefs to guide subsequent exploration. The authors instantiate the framework in GPU kernel optimization and evaluate it on 26 SOL-ExecBench kernels on NVIDIA B200, reporting improvements over several evolutionary coding agents under matched token budgets.

Critical strengths

1. Carefully controlled optimization evaluation.

The paper provides a relatively detailed experimental setup with matched backbone models, per-task token budgets, and full-workload validation, making the reported GPU kernel optimization results reasonably reproducible.

2. Detailed analysis of the optimization process.

The paper reports additional analyses beyond the final benchmark score, including belief lifecycle statistics and budget-dependent experiments.

Critical weaknesses

1. The effectiveness of the belief mechanism is not sufficiently validated.

The only direct ablation of the belief store is conducted in the small-budget setting (six tasks, 2.4M tokens), where BeliefOPT achieves 24.1 ± 3.1× compared with 24.5 ± 1.5× without the belief store (p ≈ 0.8), showing no statistically significant benefit. The larger-budget experiments compare different belief structures but do not include a corresponding no-belief baseline.

2. The experimental evidence is insufficient to support the broader claims.

The evaluation is limited to a single benchmark (SOL-ExecBench) and one domain (GPU kernel optimization), without demonstrating generalization to other research settings. The overall improvement over CodeEvolve is also modest (5.7%) and sensitive to specific workloads: excluding the two MLA tasks reduces the advantage to approximately 0.9% (calculated from Table 4). Moreover, no belief reaches the "resident" state during the evaluation, leaving the benefits of persistent, validated belief reuse unverified.

Questions for authors
  1. The paper reports that none of the 92 beliefs reaches the "resident" state during the entire evaluation. Although the residency criteria are described, could the authors explain why no belief satisfies them and what this implies for the effectiveness of the proposed persistent belief mechanism?

  2. Appendix B.1 states that the reference implementation initializes the incumbent and that promotion requires at least a 0.5% improvement. However, Table 4 reports only 0.80× for BeliefOPT on Task 005. How can this be explained? Furthermore, several baselines retain solutions with speedups below 1.00×. Although Appendix A attributes some OpenEvolve regressions to subset screening, why are reference fallback and candidate selection handled differently across methods? Are the comparison protocols consistent and fair?

  3. Why does Table 2 omit the high-budget ShinkaEvolve result, especially given that ShinkaEvolve achieves the best performance in the small-budget setting?

1 comment

Since I have not received the rebuttal, I will retain my original score for the time being.

Reviewer_ASXGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces BeliefOPT, a training-free framework for autonomous research that converts experimental experience into explicit, falsifiable, and revisable beliefs while keeping the underlying language model fixed. A research scientist formulates conditional claims from experimental outcomes, an engineering squad translates selected beliefs into candidate solutions, and an evolving evaluator provides correctness and performance feedback. The framework combines Thompson sampling for belief selection, graph-based relationships for organizing exploration, and evaluation-aware utility updates to prevent invalid improvements from reinforcing misleading guidance. Experiments on 26 GPU kernel optimization tasks from SOL-ExecBench using NVIDIA B200 hardware show that BeliefOPT achieves a geometric-mean speedup of 23.94× over reference implementations, outperforming OpenEvolve, ShinkaEvolve, and CodeEvolve under matched per-task token budgets. Additional ablations suggest that structured beliefs may benefit longer searches, while belief accumulation at smaller budgets, expanded experience archives, and expert-provided hints do not consistently improve performance.

Critical strengths
  1. Explicit and revisable representation of research knowledge. BeliefOPT represents experience as conditional, comparative claims with supporting evidence and proposed mechanisms, while distinguishing operational usefulness from mechanistic validity. This provides a principled way to revisit potentially misleading lessons rather than treating accumulated advice as established knowledge, strengthening the paper’s conceptual contribution to autonomous research.

  2. Careful integration of evaluation and belief updates. The framework separates inexpensive screening from full-workload validation and conditions utility attribution on the candidate’s final evaluation status, excluding infrastructure failures from learning signals. This design addresses an important failure mode in self-improving agents: unreliable measurements can otherwise reinforce unproductive research directions. The explicit evidence-accounting rules therefore strengthen the methodological credibility of the approach.

  3. Competitive system-level results under substantially controlled comparisons. On 26 GPU kernel tasks, BeliefOPT outperforms three evolutionary optimization baselines using the same backbone model and matched per-task token budgets, with final solutions remeasured under a common protocol. Its 17 wins, 7 ties, and 2 losses against the strongest external baseline, CodeEvolve, indicate that the aggregate advantage is supported by improvements across multiple tasks. These results provide meaningful empirical support for the complete system within the evaluated domain.

Critical weaknesses

The proposed notion of “beliefs” appears very similar to skills or experience, and the paper does not clearly distinguish beliefs from guideline-style skills.

Although the concept of beliefs is promising, the overall evolutionary-style procedure—selecting candidate directions, evaluating the repair status, and constructing a new best solution—has already been adopted in many studies, including Recursive Self-Improvement of AI Research Agents, SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent Harness, and RRSI: Regularized Recursive Self-Improvement of Agent Harnesses. In terms of the core workflow, it is unclear what advantages BELIEFOPT offers over these prior approaches.

The experiments are limited to tasks from a single domain and a single model. The paper does not examine whether BELIEFOPT remains effective with other models or in other task domains.

Questions for authors

The paper states, “With probability 0.2, an engineer instead tests a previously untried belief from a graph-ordered queue.” How was the value 0.2 determined?

The paper states, “A candidate must satisfy correctness requirements and exceed the incumbent by 0.5%.” How was the 0.5% threshold determined?

What harness is used at each stage of the complete BELIEFOPT pipeline? The paper does not provide sufficient implementation details.

Reviewer_M3HQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the risk that self-evolving research agents distill unreliable lessons from experience, since gains from past experiments may reflect contingent advantages or flawed measurements and then bias later search. The authors propose BeliefOpt, which stores experience as falsifiable, revisable beliefs. A research-scientist agent forms and revises them, an engineering squad samples them via Thompson sampling against an unguided null arm to generate candidates, and an evolving evaluator requires full-workload validation. On 26 GPU kernels from SOL-ExecBench, BeliefOpt achieves a 23.94× geometric-mean speedup, higher than the baselines OpenEvolve, CodeEvolve, ShinkaEvolve, and the human-assisted predecessor. However, ablations are mixed no belief reached residency, and most final solutions beating the reference came from the null arm, which weakens the evidence that accumulated beliefs drive the gains.

Critical strengths
  • The main methodological contribution is a closed-loop framework that turns experimental outcomes into explicit, falsifiable beliefs and uses subsequent experiments to revise both those beliefs and their influence on future search, allowing accumulated advice to guide research while preserving opportunities to discover alternatives.
  • In the learning/development process, directions with poor utility or ones identified as measurement exploits can be kept as warnings rather than discarded, so failures remain useful knowledge.
  • All systems share the same backbone model and per-task token budgets, and BeliefOpt starts from an empty belief store with no human intervention. Their results show an overall empirical performance improvement.
Critical weaknesses
  • There's a weak evidence that the beliefs are driving the performance improvements. Of the 20 final solutions beating the reference, only 4 came from belief-guided draws versus 14 from the null arm. The paper never runs the full system without the belief store on all 26 tasks, so the headline improvement could come from other components.
  • Several ambitious aspects of the framework remain un demonstrated. No belief reaches resident status, and all but one of the 92 generated beliefs are task-specific, leaving limited evidence for durable knowledge accumulation or transfer across tasks.
  • The paper frames its contribution as autonomous research in general, yet evaluates only GPU kernel optimization with a single backbone model, so generalizability is not shown.
  • The 6% advantage over CodeEvolve relies heavily on a few attention kernels, while BELIEFOPT clearly loses on the MoE kernel and performs essentially at the reference level .
Questions for authors
  • Can you measure the effect of the beliefs and compare it with the full system on all 26 tasks at the full budget? This would allow a good understanding of the method behavior.
  • Why did no beliefs reach resident status, and what would be needed for them to do so?
  • Do beliefs learned on one task improve performance on new tasks?
  • How do you distinguish an unhelpful belief from a poor implementation of a useful belief?
  • There are certain broken references ('??') in page 6

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.