- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper investigates whether an autonomous learning agent can discover useful mathematical theorems starting solely from the axioms and inference rules of a formal system, without relying on human-provided theorem libraries, proofs or training problems.
The authors propose a self-supervised theorem-discovery framework based on a stack-machine representation of Hilbert-style propositional logic. A goal-conditioned Transformer policy learns to construct proofs through sequences of axiom applications and MP. Theorems reached during exploration are reused as training goals through a hindsight-style learning mechanism. The system subsequently selects reusable theorems based on generality and reprovability. Selected theorems are introduced as additional actions for subsequent generations of proof search.
The method is evaluated on three propositional axiom systems: classical logic, positive implicational logic and intuitionistic logic. The authors report the discovery of tens of thousands of theorems and demonstrate that the classical-logic agent can prove 30% of a human-written benchmark without having been trained on those problems.
The paper further evaluates whether the discovered theorems help external LLMs construct formal proofs. Experiments on human-written and synthetic benchmarks show improvements for several GPT models when the extracted theorems are supplied as prompt lemmas.
Critical strengths
- Interesting research problem.
- The self-supervised learning framework is simple and easy to understand.
- There is evidence that the discovered theorems improve LLMs.
Critical weaknesses
-
Low novelty. The core learning mechanism is closely related to established goal-conditioned supervised learning and hindsight experience relabelling; adding previously proved lemmas as reusable proof actions is a familiar technique in automated theorem proving and library learning.
Moreover, the proposed theorem-selection mechanism combines two relatively simple heuristics: filtering specialised theorems and ranking the remaining candidates by empirical reprovability. The paper does not appear to have new learning principle or algorithmic contribution.
-
The definition of theorem usefulness. The paper uses low reprovability as a proxy for theorem usefulness. However, a theorem that is difficult for the current policy to reprove is not necessarily useful for proving other theorems. Conversely, a theorem that is easy to prove may be extremely valuable if it is frequently applicable as an intermediate lemma.
The selection procedure does not explicitly account for the frequency with which a theorem can be reused, the number of subsequent proofs it facilitates or the resulting proof-length reduction. Therefore, the relationship between the selection criterion and the stated objective of identifying useful theorems is indirect.
-
Limited experiments. The theorem-selection baselines include random selection, shortest-theorem selection and most-frequently-reached selection. These are relatively simple heuristics. The paper does not compare against more sophisticated criteria based on lemma reuse frequency, proof compression, theorem centrality or downstream proof success.
Moreover, many of the discovered theorems are just standard ones from propositional logic. It is unclear whether the proposed discovery mechanism identifies particularly effective lemmas, rather than simply providing useful derived inference rules.
Furthermore, all three evaluated systems are variants of Hilbert-style propositional logic, which share the same underlying proof-search formulation.
Questions for authors
Q1. What is the main algorithmic contribution beyond combining hindsight goal relabelling, goal-conditioned supervised learning and reusable lemma libraries? In particular, how does the proposed method differ technically from the theorem-discovery approaches of [1] and [2]?
- Gabriel Poesia et al. Learning Formal Mathematics from Intrinsic Motivation. NeurIPS’24.
- Timothe Kasriel, et al. Usefulness-Driven Learning of Formal Mathematics. 5th Workshop on Mathematical Reasoning and AI (MATH-AI), NeurIPS, 2025.
Q2. Why should low reprovability be a reliable predictor of a theorem's usefulness in proving other theorems?
Q3 Can the authors compare the discovered theorem library with a similarly sized collection of standard propositional lemmas, or with lemmas selected using proof-compression or downstream-utility criteria?
Q4 What aspects of the method are specific to propositional logic and the finite-action stack-machine formulation? Is there evidence that the approach can scale to richer formal systems, such as first-order logic?
Update: The authors mostly addressed my concerns in their rebuttal, and I am delighted to raise the score to weak acceptance.
6 comments
Official Comment to Reviewer 8sdF from Authors (2/3)
W2-2, W3-1, Q3: Standard-Lemma and Theorem-Selection Comparisons
Following the reviewer's suggestions, we added comparisons with standard lemmas and with selection criteria based on proof compression and downstream utility.
Comparison with standard lemmas. We provided GPT-6 Astra with either standard lemmas from Metamath [3], which contains basic theorems of classical propositional logic and their formal proofs, or lemmas discovered by our method. We matched the number of lemmas and the evaluation problem sets. For the conditions without lemmas and with our lemmas, we reused existing evaluation results. To measure the external usefulness of the lemma collections, we evaluated the proof success rate of the same LLM with different lemma sets.
| Problem set | Without lemmas | With standard lemmas | With our lemmas |
|---|---|---|---|
| Human-written | 90% | 90% | 93% |
| Easy | 100% | 100% | 100% |
| Hard | 96% | 94% | 97% |
| Extreme Hard | 54% | 70% | 75% |
Although standard lemmas also improved proof success on Extreme Hard, our lemmas achieved the same success rate on Easy and higher rates on the other three problem sets. These results suggest that, without using a human-provided theorem library, our agent can discover lemmas that yield proof performance matching or exceeding that obtained with the standard lemma collection evaluated here.
Comparison with additional selection criteria. We replaced lemma selection in each generation with Proof compression, based on the reduction in stored proof length obtained by replacing subproofs with candidate lemma calls, and Downstream utility, based on the number of subsequent proofs a candidate lemma facilitates. We used the same seeds, policy architecture, training budget, number of generations, and number of added lemmas as in Table 2(a). Rather than merely reselecting a final library, we reran learning, extraction, and reuse with the alternative selection rule in each generation. To assess how selection affects the agent's own theorem discovery and performance on human-written problems, we used the same evaluation metrics as for the existing random selection, shortest-theorem selection, and most-frequently-reached selection baselines: the number of discovered theorems and the agent's proof success rate on Kleene's problem set.
| Theorem-selection rule | # of discovered theorems | Proof success rate |
|---|---|---|
| Proof compression | 17% | |
| Downstream utility | 17% | |
| Generality & low reprovability (ours) | 30% |
Our selection rule achieved higher values on both metrics. This supports the effectiveness of generality and low reprovability in promoting both subsequent theorem discovery and benchmark proof performance, even compared with criteria that directly prioritize observed proof-length reductions or the number of targets whose proofs can be shortened.
In the revised manuscript, we added the standard-lemma comparison to Table 5 and the accompanying discussion in Section 5.2, and the two additional selection baselines to Table 2(a) and the accompanying discussion in Section 5.1. Appendix A provides the source, selection procedure, and list of standard lemmas, while Appendix A.1 gives the definitions, aggregation procedures, and experimental settings for the additional selection criteria.
[3] Norman D. Megill and David A. Wheeler. Metamath: A Computer Language for Mathematical Proofs. Lulu Press, 2019.
Official Comment to Reviewer 8sdF from Authors (3/3)
W3-2, Q4: Extensions Beyond Propositional Logic
Following the reviewer's feedback, we conducted additional experiments extending our framework to first-order logic. Specifically, we constructed this environment using the quantified logic system S5 [4]. This extension adapts the formula representation, axioms, and inference operations to support quantifiers while preserving the finite-action stack-machine formulation and the procedures for learning, lemma selection, and reuse.
Internal usefulness. We compared the proposed method with three ablations that separately remove generality filtering, reprovability ranking, and lemma reuse. The numbers of discovered theorems are shown below.
| Condition | # of discovered theorems |
|---|---|
| Proposed | |
| Without generality filtering | |
| Without reprovability ranking | |
| Without lemma reuse |
The proposed method discovered the most theorems among the four conditions. Removing each component reduced the number of discovered theorems, indicating that generality- and reprovability-based selection, together with lemma reuse, supports further theorem discovery in this first-order logic environment.
External usefulness. We also constructed a synthetic problem set containing quantifiers, with three difficulty groups defined by goal-formula length: Easy, Hard, and Extreme Hard, each containing 100 problems. We evaluated GPT-6 Astra's proof success rates with and without the lemmas discovered by our agent.
| Problem set | Without lemmas | With our lemmas | Improvement |
|---|---|---|---|
| Easy | 80% | 93% | +13 pt |
| Hard | 56% | 80% | +24 pt |
| Extreme Hard | 17% | 64% | +47 pt |
The lemmas improved proof success in all three difficulty groups, with the largest gain on the most challenging group. These results support our central claim that, from axioms alone, an agent can autonomously discover theorems useful to both itself and other reasoning agents, also in the evaluated first-order logic environment.
Applicability and extensions to richer systems. Our framework assumes a proof environment in which proof trees can be constructed through a finite set of operation types with defined applicability conditions. Adapting it to a different system requires corresponding formula representations, axioms, inference operations, unification procedures, and legality checks. In contrast, the principles of learning from reached theorems and selecting lemmas based on generality and reprovability under the current policy are not tied to particular axioms. Richer systems that require managing induction hypotheses or generating terms not determined by unification alone would require additional design to incorporate these capabilities into the states and actions. Our experiments demonstrate an extension that retains the finite-action stack-machine formulation, which we view as a concrete step toward applications to more general mathematics.
In the revised manuscript, we added these experiments to the "First-Order Logic" paragraph in Section 5.3, with experimental details in Appendix F. Figure 7 shows theorem-discovery counts for the four conditions in the center and external LLM proof success rates on the right. We also revised Section 7 to distinguish the scope of the current experiments from future extensions and to identify induction-hypothesis management and term generation as key challenges.
[4] Richard Zach. Boxes and Diamonds: An Open Introduction to Modal Logic. Open Logic Project, Fall 2025 edition, updated July 2026.
We again thank the reviewer for the thoughtful and constructive feedback. If the reviewer has any further questions or comments, we would be happy to address them.
Official Comment to Reviewer 8sdF from Authors (1/3)
Thank you for the time and expertise you devoted to reviewing our paper. We appreciate that you have kindly raised the following points in the weaknesses and questions sections.
We address each point below.
W1, Q1: Methodological Novelty and Differences from Prior Work
Our approach differs methodologically in how discovered theorems are retained and selected for reuse. Unlike Minimo by Poesia et al., our method accumulates discovered theorems in a persistent library and reuses them as proof actions in subsequent search. We share UseFor's goal of discovering useful lemmas from axioms, but differ in the selection mechanism. Whereas UseFor by Kasriel et al. evaluates reuse in other proofs and improvements in proof likelihood, our method selects lemmas based on generality and their own reprovability under the current policy. This enables exploration, learning, and library construction without additional downstream usefulness tests. Removing either criterion reduced the agent’s theorem discovery count and benchmark proof success rate (Table 2). The additional comparisons with proof compression and downstream utility are discussed in our response to Q3. Together, these results support the effectiveness of our methodological advances. In the revised manuscript, we clarified the selection procedure in Section 4.2 and its differences from prior work in Section 6, and added the baseline comparisons to Table 2 in Section 5.1.
We would like to note here that the contribution of this study is not only methodological but also empirical. We demonstrate that theorems discovered from axioms benefit not only the discovering agent but also independent large language models, including the frontier reasoning model GPT-6 Astra, by improving their actual proof success rates. Our qualitative analysis further shows how both familiar, interpretable lemmas and less obvious theorems guide proof construction. These quantitative and qualitative demonstrations of external usefulness are key distinctions from Minimo and UseFor. We clarified these empirical contributions in the Introduction and Section 6 of the revised manuscript.
W2-1, Q2: Motivation for the Low-Reprovability Criterion
As the reviewer has kindly mentioned, we use low reprovability under the current policy as a criterion for lemma selection. We consider how best to quantify theorem usefulness an important open research question. With this question in mind, we base our criterion on the intuition that retaining hard-to-reprove theorems as reusable lemmas allows the agent to bypass difficult proof paths when proving other theorems, thereby facilitating further exploration.
To illustrate this intuition, we created Figure 3. We consider a goal whose proof requires both intermediate theorems H and E, with respective proof success probabilities p1 and p2 under the current policy, where p1<p2. Assuming that the two subproofs succeed independently, that making one theorem available as a lemma leaves the other's success probability unchanged, and that lemma calls and the final combination always succeed, retaining the harder theorem H as a lemma yields a goal-proof success probability of p2, compared with p1 when retaining the easier theorem E. In other words, replacing the difficult subproof with a lemma call leaves only the easier part for the agent to prove on its own. To make this intuition clearer to readers, we added this figure and the accompanying explanation to Section 4.2 of the revised manuscript. We thank the reviewer for the opportunity to clarify this important point.