acceptodds
Under review as a conference paper at ICLR 2027

Reliable Self-Evolution with Imperfect Proxy Rewards

Abstract

Large language model (LLM)-based self-evolving search is a promising approach to scientific discovery. However, high-fidelity evaluation of every candidate is prohibitively expensive in some domains. Self-evolving systems in such settings therefore rely on low-cost but imperfect proxy rewards, which may assign high scores to infeasible candidates. These false positives may contaminate both the final output and the feedback used to guide subsequent generations. This motivates statistically calibrated reward intervals for more reliable self-evolving search. We propose Conformal Interval-Driven Self-Evolution (CISE), which constructs candidate-specific reward intervals using conditional conformal inference and iteration-wise online density-ratio estimation. CISE uses conservative interval-based rewards for evolutionary feedback and returns candidates only when all required property intervals lie entirely within their respective feasible regions. We derive fixed-iteration coverage results under explicit assumptions of independence and covariate shift. We evaluate CISE on three self-evolving search tasks in materials science. In our experiments, all candidates returned by CISE are true positives under high-fidelity evaluation, whereas the baselines return more candidates but include false positives. These results highlight the value of a smaller, more precise shortlist when downstream validation budgets are limited. Our anonymized repository is available at https://github.com/anonoymouscise/CISE-Reliable-Self-Evolution.git.

Primary area:
reinforcement learning
Keywords:
Self-evolving agents, materials discovery, conformal inference

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_TUc6OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2(+1)
Read review
Paper summary

Large language models (LLMs) can optimize the solution recursively based on the feedback provided by the domain-specific verifier, forming a self-evolution loop. However, some domains cannot obtain high-fidelity verifiers and instead rely on low-cost but imperfect proxy rewards. To improve the quality of the verifier and therefore the performance of the self-evolution outcome, this paper proposes Conformal Interval-Driven Self-Evolution (CISE). Specifically, CISE replaces single-point proxy rewards with calibrated property intervals and guides material search using conservative interval feedback and full-interval compliance rules. Three material tasks show that DFT false positives among the returned candidates are significantly reduced. The method has clear value in addressing the issue of self-evolution under biased proxy feedback.

Critical strengths
  • The paper focuses on an important problem: that scientific agents may repeatedly reinforce wrong candidates when chasing high proxy scores.
  • Extensive experiments have been conducted to provide a detailed discussion of the components of the proposed method.
Critical weaknesses
  • The conference seems not to allow directly showing the GitHub link in the submission.
  • The paper proposes a better reward for the evolving tasks. However, it remains unclear about the significance of the tasks. For example, how could an imperfect proxy reward infer the evolution process and final results?
  • Theorem 2 requires the calibration samples to remain i.i.d. given the generated state, but the states come from previous intervals and archive updates using the same calibration set.
  • The experiment measures the feasibility of the final candidate and has not yet verified the coverage per round and the width of the online interval.
Questions for authors
  1. Please discuss or provide results on how an imperfect proxy reward could infer the evolution process and final results.
  2. Please explain how the theorem applies to actual closed loops.
8 comments
AuthorsOpenReview ↗

[W1] Anonymous code repositories are permitted by the conference policy

Thank you for raising this policy question. The ICLR 2027 Author Guideline* explicitly permit anonymous repository links. In its FAQ on anonymous code sharing, the guidelines state:

"Make an anonymous repository and put the link in your paper."

Thus, an anonymous repository link is an allowed way to share code; the relevant requirement is preserving anonymity, rather than a blanket prohibition on GitHub links. Our inclusion of the repository link was intended to support reproducibility under this option.

  • ICLR 2027 Author Guidelines, https://iclr.cc/Conferences/2027/AuthorGuidelines.
AuthorsOpenReview ↗

[W2, Q1] How imperfect proxy rewards affect evolution and final outcomes

The issue is that a proxy error can affect both the current selection decision and the evidence used to generate later candidates. A candidate may satisfy r^(x)∈R\hat{r}(x) \in \mathcal{R} even though r(x)∉Rr(x) \notin \mathcal{R}. Reusing its favorable point prediction as evidence of success can then guide subsequent proposals toward the same misleading feature combinations. The motivation is therefore not simply to change the final filter, but to improve the reward information used throughout search.

The revised Figure 1, Algorithm 1, and Section 3.3 show this feedback path explicit. CISE uses historical inspection data Dins\mathcal{D}_{\mathrm{ins}} to initialize a hypothesis about feature combinations associated with successful rewards and short intervals. The hypothesis hth_t guides candidate generation and determines the calibration space Gt\mathcal{G}_t. After evaluation, both accepted and rejected candidates are stored in Mt\mathcal{M}_t with their features and interval rewards. This evidence informs the next hypothesis, while only candidates satisfying ∅≠It(x)⊆R\varnothing \ne \mathcal{I}_t(x) \subseteq \mathcal{R} enter the output set. The proxy and LLM parameters remain fixed, and newly proposed candidates' true rewards are not used during search.

We have added GB1 protein sequence optimization to examine this process beyond materials discovery. The revised Figure 2 displays proxy predictions and interval rewards alongside experimentally measured fitness, with candidates grouped by their generating hypotheses. Several early hypotheses yield few accepted candidates, whereas later stages, particularly H20H_{20}, generate many true positives. This trajectory provides evidence about how interval-informed hypothesis revision accompanies successful discovery, rather than only reporting the final output set.

The quantitative evidence also considers discovery yield, not just rejection of false positives. Under 400 candidate sequences per method on GB1, CISE returns 60 TP and 18 FP, with precision 0.77 and best fitness 4.470; LEON returns 59 TP and 106 FP, with precision 0.36 and best fitness 3.876 (revised Table 1). The component comparison in the revised Table 3 further shows:

MethodWBG Prec.WBG TPSSE Prec.SSE TP
Proxy0.55630.5470
Proxy + CCI0.97600.9148
+ Hypothesis (CISE)0.93960.96136

The gain from Proxy + CCI to full CISE supports the role of the hypothesis-driven component, which couples generation with its calibration basis, in expanding successful discovery while retaining high precision.

AuthorsOpenReview ↗

[W3, Q2] Scope of the coverage result under adaptive calibration-data reuse

You are correct that conditioning on a generation state obtained from earlier interval feedback does not, by itself, preserve the required independence of the calibration samples. We have revised the theoretical statement and its interpretation to separate the i.i.d. reference setting from the experiments.

Nevertheless, Theorem 1 provides the i.i.d. reference for CCI interval construction; our central objective is self-evolution driven by calibrated interval rewards that improves both output precision and TP count. Its group-conditional guarantee motivates robustness to group-marginal shift: changes in group proportions with unchanged within-group distributions, under the reference independence assumptions.

CISE connects this principle to generation by restricting hypotheses to Hϕ\mathcal{H}_{\phi} over the fixed feature map ϕ\phi. The feature conditions in hth_t both guide candidate generation and define the calibration groups represented in Gt\mathcal{G}_t (Section 3.3). Refinements require executable predicates, sufficient inspection-data support, and a limited number of splits. Thus, generation and calibration use the same feature-defined groups, aiming to align search-induced shifts with distinctions captured by the calibration basis.

Section 4.1 distinguishes this reference guarantee from adaptive fixed-data reuse. The practical contribution is supported directly by Table 3: adding hypothesis-driven evolution to Proxy + CCI increases TP count from 60 to 96 on WBG and 48 to 136 on SSE, with final precision of 0.93 and 0.96, respectively. The new Figure 2 further shows productive later hypotheses on GB1, supporting interval feedback as a mechanism for guiding subsequent generation, rather than only filtering final outputs.

Reviewer_bhUROpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CISE, a framework that incorporates uncertainty calibration and conservative reward feedback into LLM-based self-evolving search. By leveraging conformal prediction, CISE replaces conventional point-valued proxy rewards with calibrated reward intervals that provide statistical coverage guarantees. These intervals are subsequently used to guide the evolutionary search process and improve the reliability of generated candidates under ground-truth physical constraints.

Critical strengths

The paper presents a framework that improves the reliability of LLM-based self-evolving search without modifying the underlying proxy reward models. By leveraging historical ground-truth physical data to estimate proxy prediction uncertainty, CISE encourages the search process to generate candidates that are more likely to satisfy the true physical constraints.

Critical weaknesses
  1. The related work section provides limited discussion of existing approaches that address false positives, imperfect reward feedback, or uncertainty in LLM-based self-evolving search. As a result, it is difficult to clearly identify the specific limitations of prior methods that CISE aims to overcome and how the proposed framework differs from existing approaches.

  2. The paper does not provide sufficient experimental evidence to justify the necessity of its relatively complex uncertainty calibration mechanism. In particular, it remains unclear whether Conditional Conformal Inference offers meaningful advantages over standard conformal prediction in this setting. Similarly, the contribution of the online density-ratio estimation component is not independently evaluated. Ablation studies replacing Conditional Conformal Inference with standard conformal prediction and removing the density-ratio estimation component would help clarify the necessity and effectiveness of these design choices.

  3. Although the main results in Table 1 demonstrate the effectiveness of the overall CISE framework across three material discovery tasks, they do not disentangle the contributions of interval-driven evolutionary feedback and conservative output filtering. It remains unclear whether the observed improvements primarily result from better evolutionary search guidance or from stricter candidate selection. A comparison with LLEMA augmented with post-hoc conformal filtering would help isolate the contribution of the proposed evolutionary feedback mechanism.

  4. Table 2 evaluates the methods under a specific setting, where six candidates are selected and the LLEMA candidates are ranked according to their predicted band-gap proximity to 1.2 eV. However, the paper does not sufficiently justify these choices or investigate whether the conclusions remain consistent under different target band-gap values and validation budgets. Evaluating a broader range of settings would provide stronger evidence for the generality and robustness of the reported improvements.

Questions for authors

Please see the weaknesses above.

4 comments
AuthorsOpenReview ↗

[W1] Expanded related work and comparisons with uncertainty-aware baselines

Thank you for pointing out that the relationship to existing uncertainty-aware methods needed to be clearer. We have expanded the revised Section 2 into ''Self-Evolving Search and Uncertainty-Aware Variants'' and ''Conformal Inference for Imperfect Reward Models''.

The discussion now includes reward-model ensembles (Coste et al., 2024), probabilistic improvement optimization (PIO; Chen & Li, 2025), LEON (Yao et al., 2026), SPADE (Nandakumar et al., 2026), and POME (Rimon et al., 2026). We also discuss conformal methods for claim filtering, LLM-as-a-Judge evaluation, preference learning, and peptide generation, including Cherian et al. (2024), Sheng et al. (2025), Chen et al. (2026), and van Weesep et al. (2026).

CISE calibrates the prediction errors of a fixed proxy reward model and uses the resulting interval rewards for two connected purposes: deciding whether candidates satisfy the success constraints and revising feature-based search hypotheses. These hypotheses guide subsequent generation and determine the groups used for conditional calibration. This focuses the contribution on how calibrated uncertainty enters the self-evolving search process, rather than on uncertainty-aware scoring in general.

We have also added POME and LEON to the empirical comparisons. The revised Tables 1–2 report:

MethodGB1 PrecGB1 TPWBG TPSSE TPPV TP
POME0.4727432
LEON0.365948509
CISE0.77609613616

The generation budget is 400 candidate sequences per method for GB1 and 200 raw generation attempts per method for each materials task. These comparisons complement the existing materials baselines MatterGen, LLMatDesign, and LLEMA. As specified in Appendix A.1, POME and LEON are task-specific adaptations with LLM-based proposal interfaces, retaining ensemble-based scoring and critic-guided search with reflection, respectively.

AuthorsOpenReview ↗

[W2] Ablating SCI, CCI, and the hypothesis-driven component

We have added a direct component comparison in the revised Table 3 to examine whether conditional calibration offers an empirical advantage over split conformal inference (SCI):

MethodWBG OutputWBG Prec.WBG FPWBG TPSSE OutputSSE Prec.SSE FPSSE TP
Proxy1150.5552631290.545970
Proxy + SCI351.00035450.89540
Proxy + CCI620.97260530.91548
+ Hypothesis (CISE)1030.937961420.966136

SCI demonstrates that calibrated interval selection alone can substantially improve precision, but it also reduces the number of true positives. Replacing SCI with CCI increases TP count from 35 to 60 on WBG and from 40 to 48 on SSE, with precision of 0.97 and 0.91. Thus, the benefit of CCI is not merely another increase in conservatism: it recovers additional successful candidates while retaining high output precision. This is consistent with the role of the calibration space G\mathcal{G} in Section 3.2: calibration adjustments can vary across represented candidate groups instead of relying only on a common adjustment. The empirical comparison establishes the benefit in the evaluated tasks, rather than a universal ordering between SCI and CCI.

AuthorsOpenReview ↗

[W3] The contribution of interval-driven search beyond final output filtering

We agree that reliable final selection and improved evolutionary feedback are distinct questions. To address the second question, we strengthen the search component through hypothesis-driven evolution and evaluate its contribution separately from the SCI/CCI comparison (revised Section 3.3 and Table 3).

The central distinction is that interval rewards are used before subsequent candidates are generated, not only after search has finished. At iteration tt, CISE generates Qt\mathcal{Q}_{t} using the current hypothesis hth_t and archive Mt−1\mathcal{M}_{t-1}. It then stores all evaluated candidates through

Mt=Mt−1∪{(x,ϕ(x),It(x)):x∈Qt}.\mathcal{M}_{t} = \mathcal{M}_{t-1} \cup \lbrace (x, \phi(x), \mathcal{I}_{t}(x)) : x \in \mathcal{Q}_{t} \rbrace.

Both accepted and rejected candidates contribute to the revision of ht+1h_{t+1}. The hypothesis identifies feature combinations associated with successful rewards and sufficiently short intervals; it guides subsequent proposals and determines Gt\mathcal{G}_t. Only candidates satisfying ∅≠It(x)⊆R\varnothing \ne \mathcal{I}_t(x) \subseteq \mathcal{R} enter the output set O\mathcal{O}. Historical inspection data initialize this process, but the true rewards of newly generated candidates are not supplied during search.

The revised Table 3 shows that adding the hypothesis-driven component to Proxy + CCI increases TP count from 60 to 96 on WBG and from 48 to 136 on SSE, with final precision of 0.93 and 0.96. This directly addresses the concern that uncertainty-aware selection might merely discard candidates without improving successful discovery. The GB1 trajectory in the revised Figure 2 provides complementary evidence about the search process. It displays proxy predictions and interval rewards alongside measured fitness, grouped by the hypotheses used for generation. Several early hypotheses yield few accepted candidates, whereas later hypotheses, particularly H20H_{20}, generate many true positives. The measured rewards in this analysis are used only after search. This ablation evaluates the coupled effect of hypothesis-guided generation and its calibration basis. It does not isolate a post-hoc filter applied to an identical LLEMA candidate pool. We therefore interpret it as evidence for the added hypothesis-driven mechanism, rather than as the exact fixed-pool LLEMA-plus-filtering control suggested in the review.

AuthorsOpenReview ↗

[W4] Full-output evaluation and validation beyond a single shortlist setting

We provide a more robust evaluation than the original six-candidate shortlist by comparing all returned candidates under the same generation budget, without top-6 truncation or ranking by proximity to 1.2 eV. The revised Table 2 evaluates both output precision and true-positive discovery across the complete output sets, demonstrating that CISE's improvements are not dependent on a particular shortlist construction or target-based ranking.

The revised Table 2 reports the following full-output comparison with LLEMA:

TaskLLEMA OutputLLEMA PrecLLEMA TPCISE OutputCISE PrecCISE TP
WBG120/2000.7286103/2000.9396
SSE164/2000.66108142/2000.96136
PV15/2000.731120/2000.8016

For PV, a TP has to satisfy 0.7≤Eg≤2.00.7 \le E_g \le 2.0 eV and Eform≤0.0E_{\mathrm{form}} \le 0.0 eV/atom, as well as the specified compositional restrictions. Reporting the complete output sets evaluates both reliability and successful-discovery count.

We additionally evaluate GB1 protein sequence optimization, where success is defined by experimentally measured fitness greater than 1 rather than by materials-property constraints. Under 400 candidate sequences per method, CISE returns 60 TP and 18 FP, achieving precision of 0.77 and best fitness of 4.470. The baseline LEON returns 59 TP and 106 FP, with precision of 0.36 and best fitness of 3.876 (revised Table 1). Together, the full-output materials results and the added protein benchmark broaden the evidence beyond the original shortlist setting.

Reviewer_E5BiOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how LLM-based systems improve through iterative search when the reward used to guide them is an incorrect approximation of the true objective. The authors argue that agentic systems tend to rely on proxy models to evaluate the quality of the generated candidates and these proxy models can produce false positives, which when stored, propagate the accumulated error in the subsequent generations. This results in unreliable search, which the paper tries to overcome using a calibrated prediction interval representing uncertainty in the proxy reward. Results on three datasets show that CISE produces candidates with no false positives compared to other baselines.

Critical strengths
  • The paper is clearly written and well-motivated. It studies an important problem associated with proxy rewards in the field of material discovery.

  • The paper studies an important problem caused by false positives, which can contaminate the search.

  • Provides high fidelity validation for all baselines.

Critical weaknesses
  • The work needs to be evaluated more rigorously against more datasets. The idea is good, but needs more robust evaluations.

  • Results on more LLMs can help understand the generalizability of the method and help understand if the gains are LLM-specific or not.

  • What is the reason for lower candidates generated? Keeping the false positives aside, LLEMA is still seen to provide a higher number of valid candidates. What is the significance of reducing false positives then? How can the authors claim that false positives contaminate the search if LLEMA is still able to produce higher valid candidates?

  • The paper lacks component-level ablations. It does not establish which parts of CISE are responsible for the observed gains.

  • Adding all the provided components significantly adds to the compute and engineering overhead, how does it compare with the baselines in terms of the cost vs the performance? Again if baselines are providing more valid candidates then how is the cost justified?

  • In table 2, why is the comparison only shown with the top-6, why not all the valid candidates? what happens when all the valid candidates are compared? Even in top-6, LLEMA produces a candidate with 1.07eV compared to 1.08eV for CISE, so the difference seems minimal.

  • CISE is seen to return lesser valid candidates. This is okay when reliability is the priority, but a clear disadvantage if the goal is discovering promising candidates. The shear volume of promising candidates can outperform a smaller reliable set, and this point needs a deeper discussion as none of the candidates are actually synthesized. The goal of such a discovery process is to provide promising candidates faster, which can then be synthesized post filtering by scientists.

Questions for authors

See weaknesses.

5 comments
AuthorsOpenReview ↗

[W1] Empirical validation on another benchmark

Thank you for emphasizing the need for broader empirical validation. We have added GB1 protein sequence optimization, extending the evaluation beyond materials discovery to a different candidate space and a different source of ground-truth rewards. GB1 candidates vary at four amino-acid positions, and success is defined by experimentally measured fitness greater than 1. A fixed proxy evaluates candidates during search; the experimental measurements of newly generated candidates are used only for post-search evaluation.

Table 1 in the revised manuscript compares CISE with GA and the uncertainty-aware methods POME and LEON under a budget of 400 candidate sequences per method:

MethodBest FitnessOutputPrecFPTP
GA3.875128/4000.0911612
POME3.47958/4000.473127
LEON3.876165/4000.3610659
CISE4.47078/4000.771860

CISE achieves the highest best fitness and output precision, while returning 60 true positives. Relative to LEON, CISE reduces FP from 106 to 18 and improves precision from 0.36 to 0.77. Thus, the reliability improvement is not obtained by sacrificing the number of successful candidates in this comparison. Together with WBG, SSE, and PV, this provides evidence across four tasks in two scientific domains (revised Sections 4.1–4.2).

AuthorsOpenReview ↗

[W3, W7] Improving true-positive discovery, not only returning a smaller shortlist

We agree that high precision alone is insufficient when it comes at the expense of discovering substantially fewer successful candidates. This concern motivates our emphasis on both output precision and TP count under the same candidate-generation budget.

First, the number of returned candidates should be distinguished from the generation budget. CISE accepts a candidate only when its interval reward satisfies ∅≠It(x)⊆R\varnothing \ne \mathcal{I}_t(x) \subseteq \mathcal{R}. A promising point prediction can therefore be rejected if its interval crosses a constraint boundary. This explains why conservative interval selection alone can reduce the output size and lose true positives.

To address this limitation, we strengthen interval-driven search with hypothesis-driven evolution (revised Section 3.3). Historical inspection data Dins\mathcal{D}_{\mathrm{ins}} initialize a hypothesis about feature combinations associated with successful rewards and short intervals. At iteration tt, the hypothesis hth_t guides candidate generation and determines the calibration space Gt\mathcal{G}_t. The archive Mt\mathcal{M}_t retains the features and interval rewards of both accepted and rejected candidates, allowing subsequent hypotheses to use unsuccessful proposals as evidence rather than relying only on accepted examples. The proxy reward model and LLM parameters remain fixed, and newly generated candidates' true rewards are not used during search.

Table 2 in the revised manuscript reports the following comparison with LLEMA, the strongest materials baseline in TP count:

TaskMethodOutputPrecFPTP
WBGLLEMA120/2000.723486
WBGCISE103/2000.93796
SSELLEMA164/2000.6656108
SSECISE142/2000.966136
PVLLEMA15/2000.73411
PVCISE20/2000.80416

In these experiments, CISE improves precision and TP count on all three tasks. The result therefore addresses the discovery-yield concern directly, rather than relying only on the value of a smaller shortlist.

AuthorsOpenReview ↗

[W4] Component-level ablations of calibration and hypothesis-driven evolution

We have added a component-level ablation on WBG and SSE in the revised Table 3. It compares point-valued proxy rewards, split conformal inference (SCI), conditional conformal inference (CCI), and the addition of hypothesis-driven evolution:

MethodWBG Prec.WBG TPSSE Prec.SSE TP
Proxy0.55630.5470
Proxy + SCI1.00350.8940
Proxy + CCI0.97600.9148
+ Hypothesis (CISE)0.93960.96136

The comparison makes the reliability–discovery trade-off explicit. SCI substantially reduces false positives, but also reduces TP count from 63 to 35 on WBG and from 70 to 40 on SSE. CCI recovers more true positives (60 and 48, respectively) while maintaining high precision. Adding hypothesis-driven evolution then raises TP count to 96 and 136. These results support complementary roles: conformal calibration improves the reliability of selection, while hypothesis-driven evolution helps recover and expand successful discovery. The last comparison evaluates the hypothesis-driven component together with its associated calibration basis, as described in Section 3.3.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Dear Reviewers

Thank you for your feedback again. In response to your feedbacks, we have substantially revised the methodology, empirical results, theoretical results, and provided detailed comparisons with new baselines. We understand you are very busy, but we would be sincerely grateful if you could take a moment to review our rebuttal and revised manuscript. If you feel that your concerns have been sufficiently addressed, we would greatly appreciate your consideration of an updated rating.

AuthorsOpenReview ↗

Global Response

We thank all three reviewers (E5Bi, bhUR, TUc6) and the Area Chair for their constructive feedback. Below, we summarize the strengths recognized by the reviewers and the main revisions addressing their concerns.

Strengths highlighted by reviewers

  • [E5Bi, TUc6] An important problem: imperfect proxy rewards can reinforce erroneous candidates during scientific search.
  • [bhUR] Improving search reliability without modifying the underlying proxy reward models.
  • [E5Bi, TUc6] High-fidelity validation and extensive experimental analysis.

Main concerns and our responses

Broader evaluation and stronger baselines ([E5Bi, bhUR]). We added GB1 protein sequence optimization and comparisons with POME and LEON adapted to our tasks, extending evaluation to four tasks across two scientific domains. On GB1, CISE returns 60 TP and 18 FP, compared with 59 TP and 106 FP for LEON, while improving best fitness from 3.876 to 4.470 (Table 1).

Discovery yield and full-output evaluation ([E5Bi, bhUR]). Revised Table 2 evaluates all returned candidates under the same generation budget within each task. Compared with LLEMA, CISE increases TP count from 86 to 96 on WBG, 108 to 136 on SSE, and 11 to 16 on PV, while improving precision on all three tasks. These results directly address the concern that higher reliability comes at the expense of successful discovery.

Component-level ablations ([E5Bi, bhUR]). Table 3 compares Proxy → SCI → CCI → Hypothesis. Replacing SCI with CCI recovers additional true positives while retaining high precision. Adding hypothesis-driven evolution further increases TP count from 60 to 96 on WBG and 48 to 136 on SSE. This final comparison evaluates the contribution of hypothesis-guided generation together with its associated calibration basis.

How interval feedback guides evolution ([bhUR, TUc6]). Section 3.3 and Figure 1 explain how interval rewards from both accepted and rejected candidates inform hypotheses that guide subsequent generation and define calibration groups. The new GB1 trajectory in Figure 2 provides complementary evidence of productive later hypotheses. The proxy and LLM parameters remain fixed, and newly generated candidates’ ground-truth rewards are used only for evaluation after search.

Theoretical scope and presentation ([TUc6]). Section 4.1 explicitly distinguishes Theorem 1’s reference setting, with independently fixed calibration components and i.i.d. observations, from the experiments’ adaptive reuse of a fixed calibration dataset. The theorem motivates the calibration design but does not establish coverage for arbitrary adaptive search trajectories. We have also highlighted the relevant revisions in dark blue and provided their specific locations to facilitate review.

These revisions strengthen the evidence that calibrated interval feedback and hypothesis-driven evolution jointly improve reliable discovery under imperfect proxy rewards. We appreciate Reviewer TUc6’s follow-up confirming that most concerns have been addressed, and thank all reviewers and the Area Chair for their continued consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.