acceptodds
Under review as a conference paper at ICLR 2027

Self-Supervised Discovery of Useful Theorems in Formal Axiomatic Systems

Abstract

Artificial intelligence (AI) systems have recently advanced significantly in mathematical reasoning. Many approaches, including large language models (LLMs), use human prior knowledge from mathematical text, code, or theorem libraries. Although these approaches are highly effective in practice, it remains an open question whether an agent can autonomously discover useful theorems without such human priors. To examine this question, we develop an agent for discovering useful theorems from axioms alone. Our self-supervised theorem-discovery agent alternates between proof search and useful-theorem extraction, building a library of theorems reused as lemmas for further search. The agent discovers tens of thousands of theorems across three Hilbert-style propositional axiom systems, and its classical-logic instance also proves human-written benchmark problems. Providing the discovered theorems as prompt lemmas improves proof performance across multiple external LLMs on human-written and synthetic benchmarks. The benefits are larger on harder problems and persist in our evaluation with GPT-6 Astra. Qualitative examples show that both well-known and less obvious theorems contribute to proof construction. These results show that theorem discovery without human-provided theorem or problem data can reveal useful lemmas that humans did not explicitly provide.

Primary area:
reinforcement learning
Keywords:
Self-Supervised Learning, Formal Theorem Discovery, Proof Search

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_8sdFOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper investigates whether an autonomous learning agent can discover useful mathematical theorems starting solely from the axioms and inference rules of a formal system, without relying on human-provided theorem libraries, proofs or training problems.

The authors propose a self-supervised theorem-discovery framework based on a stack-machine representation of Hilbert-style propositional logic. A goal-conditioned Transformer policy learns to construct proofs through sequences of axiom applications and MP. Theorems reached during exploration are reused as training goals through a hindsight-style learning mechanism. The system subsequently selects reusable theorems based on generality and reprovability. Selected theorems are introduced as additional actions for subsequent generations of proof search.

The method is evaluated on three propositional axiom systems: classical logic, positive implicational logic and intuitionistic logic. The authors report the discovery of tens of thousands of theorems and demonstrate that the classical-logic agent can prove 30% of a human-written benchmark without having been trained on those problems.

The paper further evaluates whether the discovered theorems help external LLMs construct formal proofs. Experiments on human-written and synthetic benchmarks show improvements for several GPT models when the extracted theorems are supplied as prompt lemmas.

Critical strengths
  • Interesting research problem.
  • The self-supervised learning framework is simple and easy to understand.
  • There is evidence that the discovered theorems improve LLMs.
Critical weaknesses
  • Low novelty. The core learning mechanism is closely related to established goal-conditioned supervised learning and hindsight experience relabelling; adding previously proved lemmas as reusable proof actions is a familiar technique in automated theorem proving and library learning.

    Moreover, the proposed theorem-selection mechanism combines two relatively simple heuristics: filtering specialised theorems and ranking the remaining candidates by empirical reprovability. The paper does not appear to have new learning principle or algorithmic contribution.

  • The definition of theorem usefulness. The paper uses low reprovability as a proxy for theorem usefulness. However, a theorem that is difficult for the current policy to reprove is not necessarily useful for proving other theorems. Conversely, a theorem that is easy to prove may be extremely valuable if it is frequently applicable as an intermediate lemma.

    The selection procedure does not explicitly account for the frequency with which a theorem can be reused, the number of subsequent proofs it facilitates or the resulting proof-length reduction. Therefore, the relationship between the selection criterion and the stated objective of identifying useful theorems is indirect.

  • Limited experiments. The theorem-selection baselines include random selection, shortest-theorem selection and most-frequently-reached selection. These are relatively simple heuristics. The paper does not compare against more sophisticated criteria based on lemma reuse frequency, proof compression, theorem centrality or downstream proof success.

    Moreover, many of the discovered theorems are just standard ones from propositional logic. It is unclear whether the proposed discovery mechanism identifies particularly effective lemmas, rather than simply providing useful derived inference rules.

    Furthermore, all three evaluated systems are variants of Hilbert-style propositional logic, which share the same underlying proof-search formulation.

Questions for authors

Q1. What is the main algorithmic contribution beyond combining hindsight goal relabelling, goal-conditioned supervised learning and reusable lemma libraries? In particular, how does the proposed method differ technically from the theorem-discovery approaches of [1] and [2]?

  1. Gabriel Poesia et al. Learning Formal Mathematics from Intrinsic Motivation. NeurIPS’24.
  2. Timothe Kasriel, et al. Usefulness-Driven Learning of Formal Mathematics. 5th Workshop on Mathematical Reasoning and AI (MATH-AI), NeurIPS, 2025.

Q2. Why should low reprovability be a reliable predictor of a theorem's usefulness in proving other theorems?

Q3 Can the authors compare the discovered theorem library with a similarly sized collection of standard propositional lemmas, or with lemmas selected using proof-compression or downstream-utility criteria?

Q4 What aspects of the method are specific to propositional logic and the finite-action stack-machine formulation? Is there evidence that the approach can scale to richer formal systems, such as first-order logic?

Update: The authors mostly addressed my concerns in their rebuttal, and I am delighted to raise the score to weak acceptance.

6 comments
AuthorsOpenReview ↗

Official Comment to Reviewer 8sdF from Authors (1/3)

Thank you for the time and expertise you devoted to reviewing our paper. We appreciate that you have kindly raised the following points in the weaknesses and questions sections.

  • W1 and Q1: Methodological novelty and differences from prior work
  • W2-1 and Q2: Motivation for low-reprovability lemma selection
  • W2-2, W3-1, and Q3: Comparisons with standard lemmas and alternative selection criteria
  • W3-2 and Q4: Applicability beyond propositional logic

We address each point below.

W1, Q1: Methodological Novelty and Differences from Prior Work

Our approach differs methodologically in how discovered theorems are retained and selected for reuse. Unlike Minimo by Poesia et al., our method accumulates discovered theorems in a persistent library and reuses them as proof actions in subsequent search. We share UseFor's goal of discovering useful lemmas from axioms, but differ in the selection mechanism. Whereas UseFor by Kasriel et al. evaluates reuse in other proofs and improvements in proof likelihood, our method selects lemmas based on generality and their own reprovability under the current policy. This enables exploration, learning, and library construction without additional downstream usefulness tests. Removing either criterion reduced the agent’s theorem discovery count and benchmark proof success rate (Table 2). The additional comparisons with proof compression and downstream utility are discussed in our response to Q3. Together, these results support the effectiveness of our methodological advances. In the revised manuscript, we clarified the selection procedure in Section 4.2 and its differences from prior work in Section 6, and added the baseline comparisons to Table 2 in Section 5.1.

We would like to note here that the contribution of this study is not only methodological but also empirical. We demonstrate that theorems discovered from axioms benefit not only the discovering agent but also independent large language models, including the frontier reasoning model GPT-6 Astra, by improving their actual proof success rates. Our qualitative analysis further shows how both familiar, interpretable lemmas and less obvious theorems guide proof construction. These quantitative and qualitative demonstrations of external usefulness are key distinctions from Minimo and UseFor. We clarified these empirical contributions in the Introduction and Section 6 of the revised manuscript.

W2-1, Q2: Motivation for the Low-Reprovability Criterion

As the reviewer has kindly mentioned, we use low reprovability under the current policy as a criterion for lemma selection. We consider how best to quantify theorem usefulness an important open research question. With this question in mind, we base our criterion on the intuition that retaining hard-to-reprove theorems as reusable lemmas allows the agent to bypass difficult proof paths when proving other theorems, thereby facilitating further exploration.

To illustrate this intuition, we created Figure 3. We consider a goal whose proof requires both intermediate theorems HH and EE, with respective proof success probabilities p1p_1 and p2p_2 under the current policy, where p1<p2p_1<p_2. Assuming that the two subproofs succeed independently, that making one theorem available as a lemma leaves the other's success probability unchanged, and that lemma calls and the final combination always succeed, retaining the harder theorem HH as a lemma yields a goal-proof success probability of p2p_2, compared with p1p_1 when retaining the easier theorem EE. In other words, replacing the difficult subproof with a lemma call leaves only the easier part for the agent to prove on its own. To make this intuition clearer to readers, we added this figure and the accompanying explanation to Section 4.2 of the revised manuscript. We thank the reviewer for the opportunity to clarify this important point.

AuthorsOpenReview ↗

Official Comment to Reviewer 8sdF from Authors (2/3)

W2-2, W3-1, Q3: Standard-Lemma and Theorem-Selection Comparisons

Following the reviewer's suggestions, we added comparisons with standard lemmas and with selection criteria based on proof compression and downstream utility.

Comparison with standard lemmas. We provided GPT-6 Astra with either standard lemmas from Metamath [3], which contains basic theorems of classical propositional logic and their formal proofs, or lemmas discovered by our method. We matched the number of lemmas and the evaluation problem sets. For the conditions without lemmas and with our lemmas, we reused existing evaluation results. To measure the external usefulness of the lemma collections, we evaluated the proof success rate of the same LLM with different lemma sets.

Problem setWithout lemmasWith standard lemmasWith our lemmas
Human-written90%90%93%
Easy100%100%100%
Hard96%94%97%
Extreme Hard54%70%75%

Although standard lemmas also improved proof success on Extreme Hard, our lemmas achieved the same success rate on Easy and higher rates on the other three problem sets. These results suggest that, without using a human-provided theorem library, our agent can discover lemmas that yield proof performance matching or exceeding that obtained with the standard lemma collection evaluated here.

Comparison with additional selection criteria. We replaced lemma selection in each generation with Proof compression, based on the reduction in stored proof length obtained by replacing subproofs with candidate lemma calls, and Downstream utility, based on the number of subsequent proofs a candidate lemma facilitates. We used the same seeds, policy architecture, training budget, number of generations, and number of added lemmas as in Table 2(a). Rather than merely reselecting a final library, we reran learning, extraction, and reuse with the alternative selection rule in each generation. To assess how selection affects the agent's own theorem discovery and performance on human-written problems, we used the same evaluation metrics as for the existing random selection, shortest-theorem selection, and most-frequently-reached selection baselines: the number of discovered theorems and the agent's proof success rate on Kleene's problem set.

Theorem-selection rule# of discovered theoremsProof success rate
Proof compression7.0×1037.0\times10^317%
Downstream utility8.1×1038.1\times10^317%
Generality & low reprovability (ours)2.1×104\boldsymbol{2.1\times10^4}30%

Our selection rule achieved higher values on both metrics. This supports the effectiveness of generality and low reprovability in promoting both subsequent theorem discovery and benchmark proof performance, even compared with criteria that directly prioritize observed proof-length reductions or the number of targets whose proofs can be shortened.

In the revised manuscript, we added the standard-lemma comparison to Table 5 and the accompanying discussion in Section 5.2, and the two additional selection baselines to Table 2(a) and the accompanying discussion in Section 5.1. Appendix A provides the source, selection procedure, and list of standard lemmas, while Appendix A.1 gives the definitions, aggregation procedures, and experimental settings for the additional selection criteria.

[3] Norman D. Megill and David A. Wheeler. Metamath: A Computer Language for Mathematical Proofs. Lulu Press, 2019.

AuthorsOpenReview ↗

Official Comment to Reviewer 8sdF from Authors (3/3)

W3-2, Q4: Extensions Beyond Propositional Logic

Following the reviewer's feedback, we conducted additional experiments extending our framework to first-order logic. Specifically, we constructed this environment using the quantified logic system S5 [4]. This extension adapts the formula representation, axioms, and inference operations to support quantifiers while preserving the finite-action stack-machine formulation and the procedures for learning, lemma selection, and reuse.

Internal usefulness. We compared the proposed method with three ablations that separately remove generality filtering, reprovability ranking, and lemma reuse. The numbers of discovered theorems are shown below.

Condition# of discovered theorems
Proposed1.3×104\boldsymbol{1.3\times10^4}
Without generality filtering7.6×1037.6\times10^3
Without reprovability ranking3.3×1033.3\times10^3
Without lemma reuse5.1×1025.1\times10^2

The proposed method discovered the most theorems among the four conditions. Removing each component reduced the number of discovered theorems, indicating that generality- and reprovability-based selection, together with lemma reuse, supports further theorem discovery in this first-order logic environment.

External usefulness. We also constructed a synthetic problem set containing quantifiers, with three difficulty groups defined by goal-formula length: Easy, Hard, and Extreme Hard, each containing 100 problems. We evaluated GPT-6 Astra's proof success rates with and without the lemmas discovered by our agent.

Problem setWithout lemmasWith our lemmasImprovement
Easy80%93%+13 pt
Hard56%80%+24 pt
Extreme Hard17%64%+47 pt

The lemmas improved proof success in all three difficulty groups, with the largest gain on the most challenging group. These results support our central claim that, from axioms alone, an agent can autonomously discover theorems useful to both itself and other reasoning agents, also in the evaluated first-order logic environment.

Applicability and extensions to richer systems. Our framework assumes a proof environment in which proof trees can be constructed through a finite set of operation types with defined applicability conditions. Adapting it to a different system requires corresponding formula representations, axioms, inference operations, unification procedures, and legality checks. In contrast, the principles of learning from reached theorems and selecting lemmas based on generality and reprovability under the current policy are not tied to particular axioms. Richer systems that require managing induction hypotheses or generating terms not determined by unification alone would require additional design to incorporate these capabilities into the states and actions. Our experiments demonstrate an extension that retains the finite-action stack-machine formulation, which we view as a concrete step toward applications to more general mathematics.

In the revised manuscript, we added these experiments to the "First-Order Logic" paragraph in Section 5.3, with experimental details in Appendix F. Figure 7 shows theorem-discovery counts for the four conditions in the center and external LLM proof success rates on the right. We also revised Section 7 to distinguish the scope of the current experiments from future extensions and to identify induction-hypothesis management and term generation as key challenges.

[4] Richard Zach. Boxes and Diamonds: An Open Introduction to Modal Logic. Open Logic Project, Fall 2025 edition, updated July 2026.


We again thank the reviewer for the thoughtful and constructive feedback. If the reviewer has any further questions or comments, we would be happy to address them.

Reviewer_CALfOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper investigates whether an agent can autonomously discover useful formal theorems starting strictly from axioms and inference rules, without human-provided theorem libraries, proofs, or informal problem corpora. Across three Hilbert-style propositional logic systems (classical, positive implicational, intuitionistic), the agent discovers tens of thousands of theorems and solves 30% of a 30-problem human-written benchmark from Kleene's textbook. Furthermore, prompting external LLMs (GPT-4o, GPT-5 variants, and GPT-6 Astra) with a subset of the extracted theorems as lemmas yields gains in proof success on both the Kleene benchmark and a synthetic propositional dataset.

Critical strengths
  1. Clean formulation of self-supervised bootstrap in an axiomatic setting. Framing Hilbert-style proof search as a deterministic stack machine with unification-driven Modus Ponens cleanly connects proof trees to sequential decision processes. The selection criteria—combining generality filtering to remove syntactically subsumed instances with low Bayesian reprovability to identify bottlenecks—provide a principled heuristic for growing a persistent lemma library without explicit external rewards or human demonstrations.

  2. Empirical verification of lemma transfer to independent reasoning systems. Unlike prior work that evaluates discovered theorems solely through internal reuse or LLM-as-a-judge scoring, the paper demonstrates tangible downstream utility: prompting frozen, external frontier models (up to GPT-6 Astra) with the discovered lemmas improves their actual proof success on unseen problems, with controls against random valid formulas confirming that the selection heuristic captures non-trivial utility.

Critical weaknesses
  1. Limited expressiveness and practical relevance of propositional logic. The entire framework and evaluation are confined to Hilbert-style propositional calculi. Propositional logic is decidable (with modern SAT/SMT solvers deciding formulas in milliseconds) and lacks the expressive apparatus—such as quantifiers, inductive types, functions, and dependent types—central to modern mathematics and formal verification in interactive theorem provers like Lean 4, Isabelle, or Coq. In realistic proof assistants, proof search requires synthesizing arbitrary terms and managing unbounded action spaces, making it unclear whether the proposed stack-machine formulation and discovery heuristics can scale beyond toy propositional systems.

  2. Overstated claims of discovery "free from human priors." While the discovery policy itself is trained from scratch on self-generated trajectories, the downstream transfer evaluation relies entirely on pretrained frontier models (GPT-4o, GPT-5, GPT-6 Astra) solving human-curated textbook problems (Kleene). Because these external models are heavily pretrained on human mathematical literature and logic, it is impossible to conclude that the system is free from human priors; the injected lemmas may simply serve as in-context cues that align with the pretrained models' existing representations.

Questions for authors
  1. Could the core discovery framework (or a simplified fragment) be extended to a modern interactive theorem prover like Lean 4 (for instance, on a restricted fragment of First-Order Logic or basic algebraic structures) to assess whether autonomous lemma discovery and reprovability ranking remain viable when proofs involve quantifiers, binders, or typed terms?

Update: The authors addressed my concerns in their rebuttal, and I am therefore raising my review score to 3.

2 comments
AuthorsOpenReview ↗

Official Comment to Reviewer CALf from Authors (1/1)

We are grateful for your insightful comments and suggestions. We appreciate that you have kindly raised the following points in the weaknesses and questions sections.

  • W1 and Q1: Lemma discovery in Lean 4 and first-order logic
  • W2: Scope of the prior-free claim

We address each point below.

W1, Q1: Lemma Discovery in Lean 4 and First-Order Logic

Following the reviewer's suggestion, we conducted additional experiments in a Lean 4 environment supporting predicates, implication, and universal quantification. At each inference step during search, the environment uses native Lean types, bound variables, and proof objects to construct and verify proofs. The reached theorems are then used for learning and lemma selection. Reprovability is also evaluated within this environment by measuring whether the current policy can prove each theorem again. We ran six generations of autonomous theorem discovery, lemma selection, and reuse, comparing four conditions under the same training budget.

Condition# of discovered theorems
Proposed4.1×103\boldsymbol{4.1\times10^3}
Without generality filtering2.7×1032.7\times10^3
Without reprovability ranking3.4×1033.4\times10^3
Without lemma reuse1.1×1021.1\times10^2

The proposed method discovered the most theorems among the four conditions. In particular, the comparison with the condition without reprovability ranking supports the contribution of this ranking to theorem discovery in the Lean environment. Of the 38 acquired lemmas, 22 contain quantifiers, and all 38 passed re-verification in Lean. These results demonstrate that autonomous lemma discovery and reprovability-based selection remain viable in a Lean environment involving quantifiers, bound variables, and types.

As a complementary evaluation, we assessed the external usefulness of lemmas discovered in an environment implementing a restricted fragment of first-order logic using S5 [1]. We evaluated GPT-6 Astra with and without these lemmas on three difficulty groups, Easy, Hard, and Extreme Hard, each containing 100 problems.

Problem setWithout lemmasWith our lemmasImprovement
Easy80%93%+13 pt
Hard56%80%+24 pt
Extreme Hard17%64%+47 pt

Taken together, these results suggest that our framework is not limited to propositional logic. Further extensions to more expressive proof environments remain an important research direction.

In the revised manuscript, we added the Lean and complementary first-order logic experiments to Section 5.3 and Figure 7, with experimental details in Appendix F. We also revised Section 7 to clarify the scope of the current experiments and identify extensions to richer proof environments as an important direction for future work.

[1] Richard Zach. Boxes and Diamonds: An Open Introduction to Modal Logic. Open Logic Project, Fall 2025 edition, updated July 2026.

W2: Clarifying the Scope of the Prior-Free Claim

Our method discovers and selects theorems starting solely from axioms and inference rules, without using human-provided theorems, proofs, or training problems. Kleene's problem set and pretrained LLMs are not used in this discovery and selection process. They are used only to evaluate whether the independently discovered theorems are useful on external problems and to other reasoning agents. Accordingly, our claim of independence from human prior knowledge refers specifically to the theorem-discovery and selection process.

To clarify this distinction, we revised the Abstract and Introduction. We also clarified in the "Setup" paragraph of Section 5.2 that external LLMs do not participate in theorem discovery or selection and are used to evaluate the usefulness of independently discovered lemmas.


We are grateful for the reviewer's valuable feedback. If any questions or concerns remain, we would be happy to address them.

AuthorsOpenReview ↗

Thank You for Your Updated Review

Dear Reviewer CALf,

Thank you for taking the time to review our rebuttal. We are glad to hear that our rebuttal addressed your concerns. We are also grateful that you reconsidered your assessment and raised your score to 3.

We appreciate the time and care you devoted to reviewing our paper and helping us improve it.

Sincerely,

The Authors

Reviewer_oKExOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Starting from axioms and inference rules, the paper explores a self-supervised framework for discovering useful theorems in Hilbert-style propositional logic. The authors formalize proof search as a stack-machine decision process and propose a self-supervised algorithm that enables the agent to discover theorems during proof search, use proven theorems as subsequent goals to train a goal-conditioned policy, and select “useful” theorems to add to the action space as reusable lemmas. Experiments cover three propositional-logic axiom systems and evaluate whether these lemmas can help an external LLM generate proofs.

Critical strengths
  1. The theorem-discovery process is self-supervised and is well aligned with the characteristics of Hilbert-style propositional logic, in which all theorems can be derived from a finite set of axioms and inference rules. The experiments across three axiom systems also provide some evidence that the same discovery process can be applied to related but distinct logical settings and may therefore generalize.

  2. Extensive experiments demonstrate the usefulness of the discovered theorems in terms of performance improvements. GPT-5-high improves from solving 35/161 problems to 57/161 on the synthetic set, while GPT-6 Astra improves from 54% to 75% on the Extreme Hard benchmark. Several ablation studies also yield highly informative results.

Critical weaknesses
  1. The experiments demonstrate that the discovered lemmas are useful for proof construction. My question concerns their mathematical significance beyond computational convenience: are there concrete examples that reveal interesting logical structure or provide mathematical insight, rather than mainly shortening derivations? Or, in this restricted Hilbert-style propositional-logic setting, is mathematical significance beyond proof efficiency not a meaningful criterion? Please clarify how such value should be understood and assessed here.

  2. The applicability of the approach remains relatively limited, as it operates within Hilbert-style propositional logic. I understand that this is a very challenging question, but I would like to know the authors’ views on whether the method can generalize to domains that do not enjoy similarly favorable properties as the Hilbert-style propositional-logic framework.

Questions for authors
  1. Please clarify the specific methodological advances over existing condensed-detachment and stack-based proof-construction methods, including Metamath Solitaire, and distinguish them from the learning and library-construction methods already discussed in §6.

  2. What is the intended scope of the main usefulness claims: reusing lemmas within the evaluated propositional-logic systems, or a more general method for mathematical discovery? Please distinguish the conclusions supported by the current experiments from future extensions to richer systems. As a future direction, could this approach discover mathematically meaningful theorems that were previously unknown in the mathematical literature, rather than only new to the agent? What additional capabilities would be needed beyond the current propositional-logic setting?

4 comments
AuthorsOpenReview ↗

Official Comment to Reviewer oKEx from Authors (1/2)

Thank you for your careful and constructive review. We appreciate that you have kindly raised the following points in the weaknesses and questions sections.

  • W1: Mathematical insights beyond proof shortening
  • W2: Applicability beyond Hilbert-style propositional logic
  • Q1: Methodological advances over existing proof-construction methods
  • Q2: Scope of usefulness and future mathematical discovery

We address each point below.

W1: Mathematical Insights Beyond Proof Shortening

We agree with the reviewer that it is important not only to prove theorems but also to provide humanity with insights. In fact, a discovered theorem can reveal a proof strategy that applies across different problems. A concrete example is Thm2, discovered by our agent and discussed in the qualitative analysis. In the original manuscript, we noted that this seemingly non-obvious theorem helps an LLM construct proofs.

To illustrate the strategy behind its use, we created Figure 6. Suppose we want to establish D→(A→C)D\to(A\to C) and have proofs of D→(A→B)D\to(A\to B) and A→(B→C)A\to(B\to C). The intuitive plan is to use DD and AA to derive BB, then use BB together with the same premise AA to derive CC. In this setting, Thm2 turns the second implication into D→((A→B)→(A→C))D\to((A\to B)\to(A\to C)), which we can combine with D→(A→B)D\to(A\to B) under the shared assumption DD to obtain D→(A→C)D\to(A\to C).

This pattern appears both in the proof using explosion in the main text and in the proof using transitivity of implication in Appendix D. In both examples, the LLM failed without supplied lemmas but produced a correct proof using this pattern when given our lemmas. Together, these examples suggest that discovered lemmas can reveal proof strategies that apply across different problems, rather than merely shortening individual derivations.

In the revised manuscript, we added Figure 6 and an explanation of this pattern to Section 5.2. Appendix D presents the corresponding substitutions and detailed derivations for the two concrete examples, clarifying the logical structure and proof-construction insight provided by the discovered lemma beyond proof shortening.

W2: Applicability and Extensions of Our Approach

We consider the ability to construct proof trees using a finite set of action types to be the key condition for applying our framework. This enables the MDP formulation shown in Figure 1 and makes our learning procedure and mechanisms for lemma selection and reuse applicable in principle.

We conducted additional experiments to examine this applicability beyond propositional logic while also taking feedback from other reviewers into account. Specifically, we used S5 [1] to implement a restricted fragment of first-order logic. Our method discovered 1.3×1041.3\times10^4 theorems, exceeding all three ablations. In external evaluations with 100 problems per difficulty group, the discovered lemmas improved GPT-6 Astra's proof success rate by up to 47 percentage points. We also verified that autonomous theorem discovery, lemma selection, and reuse operate in a restricted first-order logic environment in Lean 4. These results demonstrate that our method's applicability is not limited to propositional logic. We added these experiments to Section 5.3 of the revised manuscript.

At the same time, the current finite action set may not express all necessary proof operations in systems requiring induction-hypothesis management or the generation of terms not determined by unification alone. Developing mechanisms to handle potentially infinite action spaces in such environments is an important direction for future research. In Section 7 of the revised manuscript, we distinguish the scope of the current experiments from future extensions and identify the key technical challenges in extending the current finite-action formulation.

[1] Richard Zach. Boxes and Diamonds: An Open Introduction to Modal Logic. Open Logic Project, Fall 2025 edition, updated July 2026.

AuthorsOpenReview ↗

Official Comment to Reviewer oKEx from Authors (2/2)

Q1: Methodological Advances over Existing Proof-Construction Methods

We thank the reviewer for pointing us to Metamath Solitaire as relevant prior work. Our understanding is that Metamath Solitaire [2] provides an interactive environment in which users construct proofs and reuse previously proved theorems. Our methodological advance is to build on condensed detachment and stack-based proof construction with a procedure that autonomously repeats theorem collection, learning, lemma selection, and reuse. We exploit the stack structure to collect theorems and training data from search trajectories and make selected lemmas available for subsequent search. In Section 3 of the revised manuscript, we added citations to condensed detachment and Metamath Solitaire and clarified their relationship to our framework.

Relative to the learning and library-construction methods discussed in §6, a distinguishing feature of our algorithm is its use of lemma selection criteria based on generality and reprovability under the current policy. These criteria have the advantage of not requiring additional proof search for lemma selection. Our experimental results show that the discovered lemmas help the agent discover more theorems and improve proof success for external LLMs. Our ablation study also shows that removing either selection criterion reduces both the number of theorems the agent discovers and its proof success rate. These results demonstrate the effectiveness of our proposed criteria in the evaluated settings. We clarified these methodological advances in Section 6 of the revised manuscript.

[2] Norman D. Megill. Metamath Solitaire. Software and online documentation, 1997.

Q2: Scope of Usefulness and Future Mathematical Discovery

Our aim is to develop an agent that starts solely from axioms and inference rules and can autonomously discover theorems that benefit both its own search and proof construction by external LLMs not involved in discovery or selection. The current experiments establish this usefulness in the evaluated propositional-logic systems and the first-order logic environments examined in our additional experiments. The central claim of this paper is the demonstrated effectiveness of theorem discovery and reuse within this scope.

At the same time, learning from reached theorems and selecting lemmas based on generality and reprovability under the current policy are not principles specific to propositional logic. Our long-term vision is to extend these principles to more general mathematics and discover mathematically meaningful theorems previously unknown in the literature. This requires support for richer proof operations such as managing induction hypotheses and generating terms not determined by unification alone. We would also need to check whether a discovered theorem is already known in the literature and seek expert judgment on its mathematical significance. We view the present results as a first step in this direction.

In Section 7 of the revised manuscript, we clarified the scope supported by the current experiments and distinguished it from this long-term vision and the additional capabilities needed.


We sincerely appreciate the time and care devoted to this review. If there are any remaining questions or comments, we would be happy to address them.

Thank you for your rebuttal. I will maintain my score.

AuthorsOpenReview ↗

Thank You to Reviewer oKEx for the Response

Dear Reviewer oKEx,

Thank you for taking the time to read and respond to our rebuttal. We also appreciate your decision to maintain your score of 3.

Thank you again for your thoughtful review, which helped us improve our paper. We sincerely appreciate the time and care you devoted to reviewing our work.

Sincerely,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Official Comment to All Reviewers from Authors

Dear Area Chair and Reviewers,

Thank you very much for taking the time to review our paper. We appreciate your constructive suggestions and your recognition of the evidence that the discovered lemmas help external LLMs construct proofs.

To address your concerns and questions, we have conducted additional experiments and uploaded a revised manuscript with changes highlighted in red. The main updates are summarized below.

Additional experiments and comparisons.

  • Lemma-selection baselines (8sdF): Our method discovers more theorems and solves more of Kleene's problems than the proof-compression and downstream-utility baselines (Section 5.1, Table 2(a); Appendix A.1).
  • Standard lemmas (8sdF): Our lemmas match or outperform a size-matched set of standard Metamath lemmas on all four GPT-6 Astra benchmarks (Section 5.2, Table 5; Appendix A).
  • Beyond propositional logic (8sdF, CALf, oKEx): We added discovery experiments and ablations in restricted Lean 4 and S5-based first-order logic environments, and tested the S5 lemmas with GPT-6 Astra on quantified problems (Section 5.3, Figure 7; Appendix F).

Clarifications.

  • Low reprovability (8sdF): We added a new figure illustrating how reusing a hard-to-reprove theorem as a lemma can bypass a difficult subproof (Section 4.2, Figure 3).
  • Mathematical insight (oKEx): We show how Thm2 provides a reusable construction pattern in two LLM-generated proofs (Section 5.2, Figure 6; Appendix D).
  • Relation to prior work (8sdF, oKEx): We clarify how our approach differs from Minimo, UseFor, and Metamath Solitaire (Sections 3, 4.2, and 6).
  • Scope of the claims (CALf, oKEx): We distinguish axioms-only discovery from evaluation with pretrained LLMs, and the tested logical environments from future extensions (Abstract, Introduction, and Sections 5.2 and 7).

Our point-by-point responses provide further details. We will remain available throughout the discussion period, which ends on November 18, 2026, AoE. Please let us know if you have any remaining questions or concerns. If our revisions and responses address your concerns, we would be grateful if you could reconsider your evaluation and score.

Thank you again for your time and effort in helping us improve our work.

Sincerely,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.