acceptodds
Under review as a conference paper at ICLR 2027

EPOCH: Reliable Discovery through Evidence-Governed Search

Abstract

AI research agents are increasingly used to search over programs, mathematical constructions, and proofs. However, existing systems typically optimize evaluator feedback without adequately governing how that feedback is interpreted, challenged, and reused. As a result, promising but fragile candidates can be promoted as discoveries, while benchmark improvements, finite certificates, and theorem-level claims are too easily conflated. We introduce EPOCH, an evidence-governed architecture designed to close this gap. EPOCH implements an evidence-governed discovery loop by combining explicit task contracts, typed memory, active falsification, admission checks, and independent replay, so that each candidate is evaluated against the strength and scope of the claim it supports. EPOCH achieves state-of-the-art aggregate performance on AlgoTune, substantially exceeding the strongest baseline in mean normalized score (0.65 vs. 0.53), and attains the highest mean score on the internal Math14 suite (0.57). It further shows favorable held-out behavior under official-test replay and leads the descriptive aggregate on AgentHPO. Across ten discovery problems, EPOCH delivers substantial task-specific advances, including improved executable constructions, optimized algorithms, counterexamples, and proof-supported results. These advances demonstrate its ability to convert search into concrete progress across mathematical and computational domains. Together, the results suggest that evidence governance is a necessary step toward AI research agents that produce not only stronger solutions, but also more trustworthy scientific discoveries.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
AI4AI, self discovery, scientific discovery agents, large language model agents, automated algorithm discovery, AI for mathematics, discovery loop

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_dFw4OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

Authors argue that current methods are optimizing the evaluator verdict without actually interpreting it and governing it in a right way. In this regard, they introduce a new harness/scaffold that tries to improve on this limitation. They empirically validate their method on a set of well-known benchmarks in an open-ended discovery domain and show its superior performance over a set of baselines. Furthermore, authors study several cases of novel discoveries their system produced.

Critical strengths

The paper’s most compelling contribution is a portfolio of ten verifiable results across algebraic complexity, selection, reversible circuits and combinatorics (e.g., eleven improved border-rank bounds, a 54-comparison median-of-23 procedure, a 14.1% T-count reduction for AES, a Lean-checked matroid counterexample family). Each result comes with explicit certificates, and the paper is careful to scope what is and is not proven (Table 4).

Critical weaknesses

The empirical evaluation does not adequately establish the incremental contribution of EPOCH's evidence-governance architecture.

The paper reports encouraging aggregate benchmark results and an interesting portfolio of discovery outcomes. However, the evaluation lacks several simple but important baselines that would establish whether these improvements result from EPOCH's proposed mechanisms rather than from the underlying model, evaluator, or search budget.

This concern is supported by recent empirical findings. Gideoni et al. (2025) show that random LLM program sampling can match sophisticated code-evolution methods on several tasks. Wang et al. (2026) demonstrate that automatic harness evolution does not consistently outperform simple test-time scaling under matched inference and feedback budgets. More directly, Brilliantov et al. (2026) find that a minimal, single-session coding agent can match sophisticated autonomous ML-engineering harnesses under controlled comparisons.

These results make it important to include (i) best-of-N independent proposals, (ii) multiple short autonomous agent runs, and (iii) a single long-running coding agent, using the same underlying model and comparable budgets.

Moreover, the comparison is based on one reported model configuration and equalized primarily through semantic-call counts, without comprehensive inference-cost accounting. This is important because recent studies show that harness choice can substantially affect cost without producing commensurate performance gains (Pan et al., 2026; Liu & Han, 2026). A model–harness quality–cost Pareto analysis would provide a substantially stronger evaluation.

Finally, Tables 1–3 report aggregate means without sufficient uncertainty characterization, while the task-level win/tie/loss comparisons are mixed. Reporting seed-level dispersion, paired confidence intervals, and matched-budget comparisons is necessary to assess the robustness of the claimed performance advantages.

Why this is consequential: The ablation results (Table 3) provide evidence that EPOCH's components improve aggregate performance relative to their removal on the six AgentHPOBench tasks. However, these experiments do not establish that evidence governance provides an advantage over alternative search strategies or simpler autonomous agents. Consequently, the reported benchmark gains cannot be attributed specifically to the proposed governance mechanisms rather than other differences in search strategy, computational expenditure, or implementation. Given recent evidence that simple sampling and minimally orchestrated agents can be highly competitive, matched-budget comparisons against these alternatives are essential. Without them, the paper demonstrates that its components are useful within EPOCH, but not that the proposed architecture offers a meaningful advantage over simpler approaches.

Insufficient attribution of the discovery results to EPOCH's proposed architecture.

The paper's strongest contribution is arguably its portfolio of ten mathematical and algorithmic discovery cases. These are interesting results, and I appreciate the effort to provide verifiable artifacts and distinguish established claims from remaining proof obligations. However, the experiments do not establish how much the proposed evidence-governance architecture contributes to obtaining these discoveries.

In particular, the paper does not provide a sufficiently detailed account of the model, computational budget, number of attempts, and human involvement for each discovery. More importantly, none of the discovery problems is evaluated with a comparable baseline, making it difficult to distinguish the contribution of EPOCH from the capabilities of the underlying model, extensive search, or human-guided experimentation.

This is especially consequential given prior demonstrations of AI-assisted mathematical discovery. Georgiev et al. (2025) already demonstrated AlphaEvolve's ability to produce mathematical advances across a portfolio of 67 problems. More recently, general-purpose frontier models have also demonstrated substantial mathematical discovery capabilities, including an autonomous disproof of a longstanding Erdős conjecture (OpenAI, 2026). Thus, demonstrating that an AI-based system can produce mathematical discoveries does not, by itself, establish the significance of a particular agent architecture.

Why this matters: Because the discovery portfolio is central to the paper's contribution, the absence of controlled comparisons substantially limits what can be concluded about EPOCH's effectiveness. At minimum, I would expect detailed per-discovery provenance and matched-budget plain-agent baselines on a representative subset of these problems. A comparison showing that EPOCH enables less expensive models to achieve discoveries that otherwise require substantially more capable or costly agents would provide an additional, compelling contribution. As presented, the paper demonstrates interesting discovery outcomes but does not establish the marginal value of EPOCH in producing them.

Questions for authors

Q1. Simple baselines and the incremental value of evidence governance.

Have the authors evaluated EPOCH against simpler baselines, such as best-of-N independent proposals, multiple short autonomous agent runs, or a single long-running coding agent, using the same underlying model and comparable inference budgets? If such experiments have been conducted but are not reported, could the authors share the results, including computational costs? These comparisons would help clarify whether EPOCH's governance mechanisms provide a meaningful advantage over simpler approaches.

Q2. Discovery provenance and attribution.

Could the authors provide more details about how the ten reported discoveries were obtained? In particular, which model(s) were used, what were the computational budgets and numbers of attempts per problem, and how much human intervention was involved in problem formulation, search, candidate selection, and verification? Were these discoveries produced through autonomous EPOCH runs, or did they involve additional manual experimentation and refinement? Clarifying this process, ideally with representative search trajectories and success rates across attempts, would help assess how much of the reported discovery capability can be attributed to EPOCH itself.

Reviewer_g4f5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an evidence-governed framework for automated mathematical and algorithmic discovery. It combines explicit task contracts, structured memory of successful and failed attempts, active falsification, claim-aware verification, and independent replay to ensure that discovered solutions are supported by appropriate evidence. Experiments report improvements over existing search frameworks, including an AlgoTune mean score of 0.65 versus 0.53 for the strongest baseline, competitive performance on Math14 and AgentHPOBench, and several mathematical and algorithmic discovery case studies.

Critical strengths
  1. The paper addresses an important gap in automated discovery: distinguishing promising candidates from results supported by sufficient evidence.

  2. The framework achieves substantial improvements on AlgoTune, while the component ablations provide evidence that admission checks, falsification, and phase control contribute to performance.

Critical weaknesses
  1. Most components, including memory, falsification, verification, and replay, are established ideas. The main novelty appears to lie in their integration, but the paper does not clearly establish why this particular combination represents a substantial methodological advance over existing agent frameworks.

  2. The paper gives insufficient evidence for generalizable improvements. AlgoTune covers only 12 tasks, while the held-out replay covers seven. Math14 shows only a small advantage over OpenEvolve, with EPOCH winning only 4 of 14 tasks. Stronger statistical analysis and broader evaluations would help establish robustness.

  3. The framework introduces multiple verification and memory-management stages, but the computational overhead relative to simpler search frameworks is not comprehensively quantified.

Questions for authors
  1. What is the key technical innovation beyond combining existing memory, verification, and evolutionary search techniques? Could a simpler framework with verification and failure memory achieve comparable results?

  2. Can the authors provide stronger statistical evidence, more held-out tasks, and comparisons under matched total computational costs rather than primarily matched proposal budgets?

  3. Can the authors directly evaluate whether EPOCH reduces unsupported discoveries or incorrect claim admission, rather than primarily measuring search performance?

Reviewer_JxkWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes EPOCH, an "evidence-governed" architecture for LLM-driven discovery search. Each task has a contract that fixes the editable artifact, evaluators, budget and allowed claim scope. Successful and failed trials are stored in a typed memory, every proposal must state a hypothesis and an intended falsifier, and candidates are admitted only when all checks pass. The final endpoint is replayed on held-out data and sealed together with its evidence and claim scope. With DeepSeek-V4-Flash as the backbone, EPOCH reports a mean normalized score of 0.65 versus 0.53 for AdaEvolve on a 12-task AlgoTune subset, and the highest mean on an internal Math14 suite (0.57 vs. 0.56). It also reports component ablations on AgentHPOBench and ten discovery case studies, for example new border-rank bounds, a 54-comparison median-of-23 procedure, a T-count reduction for the AES S-box, and Lean-checked combinatorial results.

Critical strengths
  1. The emphasis on claim scoping is valuable and timely for AI-for-math pipelines. The paper separates finite certificates, formal kernels and open proof bridges, distinguishes infrastructure failures from scientific failures, and keeps held-out replay out of the search loop. The appendix is commendably explicit about which bridges remain open in each case.

  2. Some discovery artifacts are concrete and checkable. Two examples are the hypercube determinant inequality with its equality characterization and the nine-element matroid counterexample to Hβ front-loading, both of which have Lean-checked finite cores.

Critical weaknesses
  1. The evidence for better search performance is weak, and the fairness of the comparison cannot be verified. The AlgoTune headline is measured on the 8 instances that were visible during search. The held-out replay covers only 7 of the 12 tasks, and the +0.04 replay advantage over AdaEvolve comes entirely from one task. At the task level, the records are 6/0/6 against both AI-Scientist variants on AlgoTune, and 4/3/7 against OpenEvolve and 5/3/6 against AdaEvolve on Math14; no confidence intervals are reported. The scores are not defined and the Math14 objectives are not disclosed. The AgentHPO ablation has no external baselines and an unstated split, and its min–max normalization turns 0.71 vs. 0.70 accuracy into a 10-point gap. Only the backbone and a cap of 48 undefined "semantic calls" are matched across methods. Tokens, wall-clock time and evaluator executions are not reported, baseline configurations are not given, 48 calls is very small for population-based methods, and only one backbone is tested.

  2. The novelty is limited, and the distinctive part of the method is never evaluated. Most components have close precedents that the paper does not discuss: failure memory (Reflexion, AlphaEvolve's program database), falsification (POPPER, AlphaEvolve's evaluation cascades), correctness-gated replacement (which App. C.1 describes as part of the lifecycle common to all methods), and held-out replay. The search itself is single-incumbent hill climbing. The distinctive contribution, claim-level governance, is never measured. No experiment compares how often each method's retained results fail independent verification, and it is unclear why governance should raise speed scores. The discovery section also departs from the paper's own principle. The median-of-23 result is called "record-setting", but per App. I.4 it is an unformalized candidate. The AES S-box result is compared only with the authors' own baselines, and no case reports its budget, human involvement or same-task baselines.

  3. Key algorithmic details are missing: the prompts, how the memory view is selected, the phase-controller rule, and how challenges are generated. The heavy terminology and the references to a "supplied research dossier" make it hard to tell what EPOCH itself produced.

Questions for authors
  1. Fairness and significance. What exactly is a "semantic call", and do EPOCH's challenge, admission and replay steps count toward the 48-call budget? Could you report tokens, wall-clock time and evaluator executions per method, the baseline configurations, and results at a budget suited to population-based baselines? How were the 12 AlgoTune tasks chosen, and why is the replay limited to 7 of them? Could you report replay on all 12 tasks with confidence intervals? Please also define the normalized scores, list the Math14 objectives, and explain why AgentHPO has no external baselines and which split its metrics are reported on.

  2. Mechanism, novelty and reliability. Which component drives the AlgoTune gains, and could the ablation be run on AlgoTune or Math14 instead of hyperparameter tuning? How does EPOCH differ from Reflexion, POPPER and AlphaEvolve's evaluation cascades? Since the core claim is more reliable discovery, can you measure reliability directly, for example the fraction of each method's retained endpoints or claims that fail independent verification? Please also provide the prompts, the memory-view selection procedure, the phase-controller rule and the challenge-generation procedure.

  3. Discovery cases. For each headline case, what backbone, budget and degree of human involvement were used, and were baselines run on the same task? How do the results compare with the best known prior results: the bounds for V₁₂(23), border-rank bounds over F₂, and the T-counts of published quantum AES S-box circuits? How is the "record-setting" claim for the median-of-23 result consistent with its status as a candidate with an open bridge?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.