acceptodds
Under review as a conference paper at ICLR 2027

SEER: Learning Scientific Belief Evolution through Falsification-Driven Exploration

Abstract

Scientific discovery requires explanations that evolve with evidence, whereas most AI-for-Science systems optimize fixed predictive objectives. We introduce Falsification-driven Scientific Belief Learning (FSBL), a paradigm that frames discovery as the evolution of structured, falsifiable scientific beliefs. We instantiate FSBL as SEER (Scientific Exploration through Evidence and Reasoning), a neuro-symbolic system combining neural perception, knowledge-graph-grounded hypothesis generation, Bayesian soft-evidence updating, lifecycle-managed revision, and falsification-driven experiment selection. Each theory specifies a mechanism, quantitative predictions, an executable falsification condition, and a posterior belief. An expected-scientific-value planner selects experiments that discriminate among competing theories and challenge well-supported ones. We evaluate SEER across five closed domains with verifiable ground truth and one exploratory superconductivity problem. Passive coverage fails to distinguish SEER from random search, motivating active metrics for rule recovery, held-out agreement, and interventional validity. SEER recovers all 20 hidden rules in the synthetic world and achieves theory-grounded recovery across real-data domains. Its theories replicate on held-out data, whereas post-hoc rules derived from optimization baselines replicate poorly. Ablations show that removing falsification probes leaves spurious theories supported, while removing the belief system eliminates theory production. On the open problem, SEER sustains multi-generation refinement and identifies descriptor patterns consistent with known physics, with all open-domain claims explicitly scoped to surrogate-level evidence.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Scientific Discovery, AI for Science, Scientific Reasoning, Neuro-Symbolic AI

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_zsx4OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes a falsification-driven paradigm for scientific belief learning and implements it in a system called SEER. The system maintains a set of falsifiable theories simultaneously, selects an experiment at each step to test them, and updates the confidence in each theory based on the results. The scientific theory mentioned in the article actually boils down to single-variable intervals or interval combination rules for two variables. The revised theory is mainly reflected in adjusting the scope of application for these rules. Therefore, this implementation is much more specific and narrower than the scientific discovery described in the abstract.

Critical strengths

The paper targets a good problem. Moving AI-for-science systems from optimizing a fixed objective toward forming explicit, falsifiable hypotheses and revising them as evidence comes in is an important direction.

Critical weaknesses
  1. The readability of this article feels very poor. For example, Figure 1 looks very fancy, but in reality, I find it incomprehensible; it cannot explain itself. The figure contains many elements that are not explained in the legend, caption, or main text. The peak chart in the middle of Figure 1 doesn't have defined axes, and it's unclear what the positions of the peaks represent. There is also no explanation of what the points and connecting lines actually are. The curve in the bottom left of Figure 1 also has a flaw. The caption says these are illustrative posteriors, but the probability curve goes over 1.0 and ends up around 1.2. A probability can't be greater than 1.

  2. There’s a disconnect between the problem the authors claim to solve and the work they actually do, and they are missing necessary baseline comparisons. In the introduction, the authors state that scientific discovery needs to answer why a system behaves a certain way and when that explanation fails. The challenge, they argue, is being able to autonomously build, evaluate, and revise those beliefs. Then, in section 3.1, they state that the learning objective is the explanatory quality of the final B_T, but this explanatory quality is never defined anywhere in the paper. When it comes to the actual experiments, however, the task shifts entirely:

  • SEER are given a queryable black-box function f(x) with a query budget of 300 to 500 steps.
  • It outputs a batch of interval rules or box rules in the form of "when x is in the closed interval [a, b], y is higher."
  • They compare these against pre-defined ground-truth rules to see how much recall they ultimately achieve.

So while the problem they claim to tackle is grand, the actual problem they solve is very specific, and they fail to bridge the gap between the two.

Furthermore, the three groups of existing methods they contrast SEER with (optimization and active learning, causal discovery, and LLM agents) were not designed to recover box rules from black-box functions. Criticizing them for not producing explanations is essentially grading other people’s work against a task of the authors’ own definition. The problem categories that are actually closest to this real task (such as active level set estimation, subgroup discovery, or PRIM) are completely unmentioned.

These are also the baselines the paper is missing. The main comparison (Table 2) gives Random/GA/BO a post-hoc extractor that keeps at most one interval rule per dimension (Appendix E.3). More appropriate baselines would include:

Questions for authors

I have no questions for the authors. My main concerns are described in the Weaknesses section.

Reviewer_pYpCOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SEER, a neuro-symbolic framework for scientific discovery that maintains structured hypotheses, updates their confidence using evidence, and performs falsification-driven experiments and theory revisions. The authors evaluate the system across synthetic and scientific domains, reporting improvements in rule recovery and hypothesis replication over optimization-based baselines.

Critical strengths
  • The emphasis on explicit hypothesis representation, falsification, and auditable theory evolution is interesting.

  • The paper includes multiple domains, ablations, and analyses of belief dynamics.

Critical weaknesses
  • Limited novelty and inadequate comparison with existing automated discovery methods. Prior systems such as FunSearch (Romera-Paredes et al., 2024) and AlphaEvolve (Novikov et al., 2025) already demonstrate iterative generation, executable evaluation, and refinement, while AI-Hilbert (Cory-Wright et al., 2024) integrates scientific background knowledge with symbolic discovery. AIGS (Liu et al., 2024) explicitly incorporates automated falsification. Although several of these works are cited, their relationship to SEER is insufficiently analyzed, and AlphaEvolve is omitted entirely. More importantly, the paper lacks strong comparisons against existing symbolic discovery methods such as PySR (Cranmer, 2023) and AI Feynman (Udrescu & Tegmark, 2020). Without such comparisons, it is difficult to determine whether SEER's handcrafted belief-evolution machinery offers substantive advances over established discovery and refinement frameworks.

  • Questionable belief formulation. The proposed Bayesian updates do not use theory-specific likelihoods or prediction errors. Instead, applicable theories receive the same normalized observation score, making it unclear whether the resulting confidence values meaningfully represent belief in each theory. This undermines the central methodological contribution.

  • Weak baselines and potentially misleading metrics. The comparisons omit strong symbolic regression methods (e.g., PySR and AI Feynman) and head-to-head evaluations against modern automated discovery agents such as AlphaEvolve. Moreover, stronger decision-tree extraction already matches or exceeds SEER's rule recovery on several domains. The structural rule-matching metric is also permissive: synthetic rule recovery reaches 0.99 despite strict hypothesis accuracy of only 0.49, raising concerns about whether the reported recovery reflects genuinely correct theories.

ref: Novikov, A., Vũ, N., Eisenberger, M., Dupont, E., Huang, P.-S., Wagner, A. Z., Shirobokov, S., Kozlovskii, B., Ruiz, F. J. R., Mehrabian, A., Kumar, M. P., See, A., Chaudhuri, S., Holland, G., Davies, A., Nowozin, S., Kohli, P., & Balog, M. (2025). AlphaEvolve: A coding agent for scientific and algorithmic discovery

Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J. R., Ellenberg, J. S., Wang, P., Fawzi, O., Kohli, P., & Fawzi, A. (2024). Mathematical discoveries from program search with large language models. Nature, 625, 468–475.

Udrescu, S.-M., & Tegmark, M. (2020). AI Feynman: A physics-inspired method for symbolic regression. Science Advances, 6(16), Article eaay2631.

Questions for authors

Since the consistency score is derived from normalized observations rather than theory-specific prediction errors or likelihoods, how can the resulting posterior meaningfully distinguish correct theories from incorrect or mutually contradictory ones? How does SEER fundamentally differ from existing iterative scientific discovery frameworks such as AI-Hilbert, AIGS, FunSearch, and AlphaEvolve?

Reviewer_pphLOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents a system called SEER (Scientific Exploration through Evidence and Reasoning) for automated discovery of scientific theories through a framework comprising of iterative experimentation and belief updates. The paper then describes an evaluation on synthetic and real data to discover "literature-curated ground-truth rule sets".

Critical strengths

None.

Critical weaknesses
  1. The methodology description is, unfortunately, quite hard to parse in the writing currently. The core process of how theories are generated from observations, including the role of the neural encoder and KG are not well explained.
  2. Based on my understanding, the space of hypotheses is constrained by domain-specific vocabularies and predefined mechanisms, which seems limiting in real-world discovery scenarios.
  3. The evaluation choices are not motivated well enough. It was unclear to me whether the synthetic construction provides SEER an unfair advantage or not. Additionally, how the "real" part of the evaluation was curated was not made clear either.
  4. A major concern with the presented manuscript is also the lack any external baselines and discussion of work that is very related to the concepts proposed (e.g., AutoDiscovery, POPPER, AI Scientist, etc.)
  5. For belief updates, it was unclear whether the evidence is appropriately reasoned over to determine support or contradiction for particular theories; instead the mechanism that seems to be in place is a coarse support detection that assigns scores equally to all theories. This seems like an important aspect to analyze.
  6. The results presented seem to indicate a mixed effect when removing falsification, challenging the main motivation of the paper. Also, removing the encoder seems to improve performance on the real-data.
Questions for authors

Please see weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.