acceptodds
Under review as a conference paper at ICLR 2027

GenLimitLib: A Formal Library for Language Generation in the Limit and AI-Assisted Mathematical Research

Abstract

We present GenLimitLib, a source-aligned Lean 4 library for language generation in the limit. Introduced by Kleinberg and Mullainathan at NeurIPS 2024, language generation in the limit studies a theoretical question motivated by LLMs: how to generate valid new strings from observed examples. This young and rapidly evolving field offers a natural testbed for studying large-scale formalization. GenLimitLib contains formal developments for 30 papers. It extracts shared definitions and reusable proof components while preserving paper-specific assumptions and statements, and records relationships across papers. In this way, GenLimitLib provides a concrete and structured view of the literature. We show through mathematical case studies and LLM experiments how our library can support both human mathematical research and AI-assisted research.

Primary area:
learning theory
Keywords:
Formal library, language generation in the limit, AI-assisted mathematical research

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_VqbcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces GenLimitLib, a Lean 4 library for research on language generation in the limit. It contains formal developments for 30 papers and tracks 405 scoped claims, with full coverage of 217. Core, Support, and Bridge modules organize shared definitions, reusable proofs, and cross-paper relationships. Mathematical case studies include a proof repair, a transferred impossibility construction, and an exact characterization of a special-case tradeoff. Across five new theorem tasks, access to the full library increases proof-generation success from 48% to 79%. Reading experiments also show improvements from relevant library-derived content.

Critical strengths
  1. The library provides substantial reusable infrastructure for a connected research area. Its separation of shared definitions, paper-specific assumptions, and cross-paper relationships is useful, and the formalization boundaries are explicitly disclosed.
  2. The mathematical case studies demonstrate value beyond collecting existing results. The proof-generation experiments also include meaningful controls: identical theorem targets, source papers, environments, and time budgets, together with proof-availability and reasoning-effort comparisons.
Critical weaknesses
  1. The completed source-alignment audits need clearer reporting. Section 3.3 defines three audit levels covering statements, algorithms, and proof dependencies, but does not summarize how many papers or claims have completed each level. The reported formalization coverage therefore does not directly show how extensively these different aspects of source fidelity have been checked.
  2. The proof-generation evidence is concentrated in a small, library-informed task set. The 300 runs cover only five theorem tasks, generated using the same library and five-paper neighborhood. Table 2 shows that Task D contributes 20 of the 31 additional successes under PML. Benefits also occur elsewhere, but the aggregate improvement has limited evidence for transfer to independently selected research problems.
Questions for authors
  1. Could you briefly summarize the audit levels completed, the review procedures, and the semantic, computational, and probabilistic guarantees checked within the declared formalization scope? This would help readers assess source alignment, a central contribution of the library.
  2. How representative are the five proof-generation tasks of the broader library? Could you validate the findings on a small set of independently selected theorems, or clarify the extent to which the conclusions are specific to the evaluated research neighborhood?
  3. Section 5.2 acknowledges that the reading experiments primarily measure the benefit of curated information. Could you add a natural-language control containing the same mathematical information, to distinguish the contribution of Lean representation from that of additional relevant facts?
5 comments

W1/Q1: Audit coverage, procedures, and scope

Thank you for suggesting that we report source-alignment audits more clearly.

Coverage. At submission, human audits covered selected main results in eight papers, including statement-level review (Level 1) of 35 source claims within their recorded scopes. Level 2 audits covered core algorithms and constructions in P00, P01, and P39. These include foundational algorithms in the area and the main algorithm used in our Section 5.1 experiments. A Level 3 audit covered P01’s semantic construction of the KM algorithm.

We have since expanded level 1 human audits, which check statements and their relevant definitions. These now cover 167/405 source claims (41.2%), including theorems, lemmas, definitions, examples, and other claims. They cover 122/147 main theorems (83.0%) in the source papers. We have additional LLM-based alignment checks for all source claims. Detailed audit records are available at: https://anonymous.4open.science/r/audit-evidence-review-2247/README.md

Initial human audits focused on selected results because early checks found few mismatches between Lean statements and their sources. This may be because definitions and theorem statements in language generation in the limit are generally precise, with little ambiguity.

Most apparent mismatches we found during the audit are Lean correcting source paper statements, proving stronger results under weaker assumptions, adding more details from the source proof into its statement, or splitting a source theorem into separate Lean statements. We documented these mismatches in our library. Among the statements reviewed, we found only one unintended translation error: the source stated (A⇒B)∧(C⇒D)(A \Rightarrow B) \land (C \Rightarrow D), but the Lean statement was (A∧C)⇒(B∧D)(A \land C) \Rightarrow (B \land D). The original implications were in fact proved in Lean.

Review procedure. Human audits are conducted by domain experts familiar with the source papers. Level 1 checks definitions, assumptions, quantifiers, conclusions, and access models. Level 2 additionally checks algorithms and constructions. Level 3 checks intermediate lemmas and proof dependencies. We document stronger results and other justified differences, correct unintended translation errors, and also formalization scopes.

Besides human audit, we have Lean kernel checks and LLM-based audits. All formalizations compile without sorry, admit, or project-defined axioms. The Lean kernel verifies proofs against their formal statements, but does not establish source alignment. In a separate LLM-based alignment check, an LLM translates Lean statements into natural language without seeing the paper, then compares the translation with the source. Human audits complement these checks by addressing ambiguities and errors that LLM reviews may miss.

Guarantees and scope. Our audits generally follow the scope of the corresponding Lean statements. We record which semantic, computational, and probabilistic guarantees were checked, where applicable. We also record any restrictions and aspects outside the audit scope.

We have added our human audit coverage and audit details to Section 3.3 (in blue in the revision).

W2: Task concentration and transfer

Thank you for this point.

We agree that Task D contributes most of the additional successes. The task-level results suggest that success gains are closely related to library reuse (Table 2(b)). However, efficiency benefits extend across tasks. PML lowers the API cost per valid proof across all tasks where both P and PML succeed by approximately 10% on A, 28% on B, and 57% on C (Table 10).

Extension to independently selected research problems. To extend the evaluation beyond the five theorems developed using the library, we tested Theorems 1.4 and 1.5 of a recent paper Hallucination Rates in Language Generation (arXiv:2607.23361). The paper is outside the five-paper sub-library. The paper was first posted on July 25, 2026, after the July 9 gpt-5.6-sol model snapshot used in our experiments. Neither its text nor its Lean proofs were provided to the model. We followed the Section 5.1 protocol without a supplied proof, with five runs per theorem, resource condition (P or PML), and effort level (high or medium).

  • Theorem 1.4: P achieved 8/10 successes and PML 9/10. Among successful runs, PML reduced the API cost per valid proof by 33.0% at high effort and 27.4% at medium effort and reduced mean time to the first valid proof by 32.5% and 30.5%, respectively. The successful PML proofs reused fresh-value helpers and sparse-set counting and limit lemmas from the sub-library.
  • Theorem 1.5: Neither condition proved the full statement (0/10 each). Both proved the complete special case where (α,β)=(0,1)(\alpha,\beta)=(0,1) in 5/10 runs. The PML partial proofs mainly reused interface definitions and basic helpers and developed the central diagonal constructions themselves.

Theorem 1.4 provides evidence of efficiency benefits for a published theorem outside our task generation using the sub-library.

In a future revision, we plan to evaluate more new, nontrivial theorems from papers outside the selected sub-library. We aim to cover a wide range of research questions in this area and potential library reuse, which is challenging in a limited time. Our new experiment on Theorems 1.4 and 1.5 provides a starting point for this broader evaluation.

We have added our current experiment in Appendix B.4 (in blue in the revision).

Q2: Task representativeness

Our evaluation uses dense generation (Paper P39) as a starting point and considers its interactions with noise, omissions, feedback, and infinite contamination (Papers P06, P12, P17, P19). This models a common research setting, where new results were developed by combining ideas from closely related papers.

Within this neighborhood, the five tasks span a wide range of observed library reuse: 16.8%–91.9% among successful PML proofs. Our experiment results suggest that the availability of reusable results is an important factor in the library’s benefit. Our additional tests of Theorems 1.4 and 1.5 in a recent paper extend the evaluation beyond the original task-generation pipeline.

We expect similar benefits in other research neighborhoods where relevant results can be reused, and plan to test this through broader evaluation.

Q3: Section 5.2 natural language control

Thank you for this suggestion. We have added the control.

Setup. The new arm takes the statements of the "Paper + relevant Lean" block, in the same order, and rewrites each one as fluent mathematical prose of about the same length. Identifiers are read as ordinary words (e.g. HasInfiniteClosureDimension H becomes "H has infinite closure dimension"), and no Lean syntax remains. The median block is 1,829 tokens, vs. 1,724 for the Lean.

The prose was written by Claude Opus 5.5, which saw only the Lean statements (no question, option or answer). To make sure each line says exactly what its Lean statement says, every line was checked by Claude Sonnet 5.5, which caught all 147 errors we planted as a test, and a careful line-by-line audit of 240 random lines by Claude Opus 5.5 found one minor omission. The prose was frozen before the run. All other settings match Table 4.

Reference materialAccuracy (%)Difference from relevant Lean
Paper69.7−8.87 [−11.96, −5.91]
Paper + unrelated Lean71.5−7.12 [−9.95, −4.48]
Paper + relevant Lean78.6—
Paper + the same statements as prose, length-matched78.6−0.04 [−1.39, 1.25]

Interpretation. The prose recovers the full gain over the paper alone (+8.8 [6.1, 11.8] points, vs. +8.9 for Lean), so the gain comes from the content of the selected statements rather than from Lean syntax. This content is what formalization provides. The prose is a direct translation of the library’s Lean statements, yet the paper text they were formalized from gives only 69.7%, and unrelated Lean of the same length adds only 1.8 points. Formalization states each result explicitly and precisely, with its hypotheses, quantifiers and conclusion spelled out, and these statements help the model whether they are given in Lean or in English. We have added this control and its interpretation to Section 5.2, with details in Appendix C (in blue in the revision).

Reviewer_KqcHOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces GenLimitLib, a Lean 4 formal library for research on language generation in the limit. It organizes definitions and theorems from 30 papers and connects results across papers to support checking and reuse. The authors also present proof issues and new mathematical results found during formalization, and evaluate how the library helps models with theorem proving and paper understanding.

Critical strengths
  1. The resource has a clear research use. Definitions and results from different papers are brought into a shared framework, with clear links to source claims and proof dependencies. This makes existing results easier to reuse in subsequent research.
  2. Concrete results support the value of the resource. The authors identify and repair an issue in an existing proof and obtain new theoretical results. The necessary-and-sufficient characterization under replay is particularly interesting. Access to the full library also raises proof success from 48% to 79%, showing that it can help models complete proofs.
Critical weaknesses
  1. The proof tasks need a more concrete description, and their coverage is limited. The evaluation includes only five mathematical tasks, developed and selected using the library. Their mathematical statements and advances over existing results are not presented in enough detail, making it difficult to assess their difficulty and how broadly the findings apply.
  2. The reading experiments are missing an informative control. Relevant Lean material improves accuracy, but it is not compared with equally informative natural-language material. It therefore remains unclear how much of the gain comes from formal representation versus access to additional relevant knowledge.
Questions for authors
  1. Could the authors provide the mathematical statements of the five proof tasks and explain what new arguments each requires beyond the existing library?
  2. What is the main advance of the necessary-and-sufficient characterization under replay over prior results, and what are the key limitations of its scope?
  3. Have the authors considered equally informative natural-language material as a control for the reading experiments?
3 comments

We thank the reviewer for their careful reading and constructive suggestions.

W1/Q1: Task statements, relation to prior results, and evaluation scope

Thank you for asking us to clarify the mathematical proof tasks and their coverage. Below, we summarize the five theorem tasks.

TaskInformal statementRelationship to existing resultsSuccesses /20: P / PML / PML-Oracle
AAn uncountable family is uniformly generatable without samples. However, every deterministic generator that generates in the limit for this family must have upper density zero on some target language, even with one target-membership query per round.Combines membership-query feedback with a density obstruction.13 / 15 / 15
BFor a finite language family, the presenter reveals an infinite set E contained in the infinite target language K. One generator achieves lower density at least max⁡(0.5dK(I(E)),dK(I(E)∖E))\max ( 0.5 d_K(I(E)), d_K(I(E)\setminus E) ), where I(E)I(E) is the intersection of all family members containing E and dKd_K is the lower density.Strengthens partial-enumeration guarantees of paper P39 for finite family.19 / 19 / 19
CA single oblivious stream is a valid vanishing-contamination presentation of two nested target languages. Any randomized generator that generates in the limit almost surely for both has expected upper density zero on the larger target.Extends the vanishing-contamination deterministic construction in P17 to randomized generators and an oblivious-stream formulation; the full task also establishes a finite-family minimax result.16 / 20 / 18
DEvery countable family admits a generator achieving lower density 1/2 with finitely many corrupt input occurrences, allowing repeated input observations.Combines the finite-expansion approach in P17 with the half-density guarantee in P39, handling repeated inputs and transferring the density guarantee back to the original target.0 / 20 / 20
EFor each q≥0q\ge 0: (i) every countable family admits a generator with lower density 1/2 under at most q distinct noisy values; (ii) there exists an uncountable family that admits lower density 1/4 at noise level q, but no generator can generate in the limit at level q+1.Part (i) combines finite expansion in P17 with half-density generation in P39. Part (ii) uses P12’s noise-level separation construction, but adds a new least-unannounced sweep and a density analysis.0 / 5 / 9

The five tasks cover a wide range of potential library reuse. A and B develop new arguments, C extends an existing construction, and D and E rely more heavily on adapting and combining published results. This is also reflected in our experiment: The library dependency shares of successful PML proofs range from 16.8% to 91.9% (Table 2(b)). The largest success gains occur on D and E, which have the highest dependency shares. For these two tasks, library access provides substantial mathematical components that condition P must reconstruct.

We have added the task summary and full theorem statements to Appendix B.1.1 and Appendix B.3 (in blue in the revision, also in our experiment repository at submission).

Extension. To extend the evaluation beyond the five theorems developed using the library, we tested Theorems 1.4 and 1.5 from a recent paper, Hallucination Rates in Language Generation (arXiv:2607.23361), outside our library. The paper was first posted on July 25, 2026, after the July 9 gpt-5.6-sol model snapshot used in our experiments. Neither its text nor its Lean proofs were provided to the model. We followed the Section 5.1 protocol without a supplied proof, with five runs per theorem, resource condition (P or PML), and effort level (high or medium).

  • Theorem 1.4: P achieved 8/10 successes and PML 9/10. Among successful runs, PML reduced the API cost per valid proof by 33.0% at high effort and 27.4% at medium effort, and reduced the mean time to the first valid proof by 32.5% and 30.5%, respectively. The successful PML proofs reused fresh-value helpers and sparse-set counting and limit lemmas from the sub-library.
  • Theorem 1.5: Neither condition proved the full statement (0/10 each). Both proved a special case where (α,β)=(0,1)(\alpha,\beta)=(0,1) in 5/10 runs. The PML partial proofs mainly reused basic helpers and developed the central arguments themselves.

Theorem 1.4 provides evidence of efficiency benefits for a published theorem outside our task generation using the sub-library.

In a future revision, we plan to evaluate more new, nontrivial theorems from papers outside our selected sub-library. We aim to cover a wide range of research questions in the area and potential library reuse, which is challenging within a limited time. Our new experiment on Theorems 1.4 and 1.5 provides a starting point for this broader evaluation.

We have added our current experiment in Appendix B.4 (in blue in the revision).

W2/Q3: Reading experiments informative control

Thank you for this suggestion. We have added the control.

Setup. The new arm takes the statements of the "Paper + relevant Lean" block, in the same order, and rewrites each one as fluent mathematical prose of about the same length. Identifiers are read as ordinary words (e.g. HasInfiniteClosureDimension H becomes "H has infinite closure dimension"), and no Lean syntax remains. The median block is 1,829 tokens, vs. 1,724 for the Lean.

The prose was written by Claude Opus 5.5, which saw only the Lean statements (no question, option or answer). To make sure each line says exactly what its Lean statement says, every line was checked by Claude Sonnet 5.5, which caught all 147 errors we planted as a test, and a careful line-by-line audit of 240 random lines by Claude Opus 5.5 found one minor omission. The prose was frozen before the run. All other settings match Table 4.

Reference materialAccuracy (%)Difference from relevant Lean
Paper69.7−8.87 [−11.96, −5.91]
Paper + unrelated Lean71.5−7.12 [−9.95, −4.48]
Paper + relevant Lean78.6—
Paper + the same statements as prose, length-matched78.6−0.04 [−1.39, 1.25]

Interpretation. The prose recovers the full gain over the paper alone (+8.8 [6.1, 11.8] points, vs. +8.9 for Lean), so the gain comes from the content of the selected statements rather than from Lean syntax. This content is what formalization provides. The prose is a direct translation of the library’s Lean statements, yet the paper text they were formalized from gives only 69.7%, and unrelated Lean of the same length adds only 1.8 points. Formalization states each result explicitly and precisely, with its hypotheses, quantifiers and conclusion spelled out, and these statements help the model whether they are given in Lean or in English. We have added this control and its interpretation to Section 5.2, with details in Appendix C (in blue in the revision).

Q2: Replay result's main advance and scope

Thank you for this question. In short, our theorem turns P22's single impossibility example into a complete classification. For every finite family, it says exactly when proper generation is still possible once the generator's own outputs are fed back into its data.

Prior result. P22 showed, with a single four-language family, that replay can make proper generation impossible even for finite families, although every finite family admits proper generation without replay. Which finite families replay actually breaks was left open.

Main advance. Our theorem answers this exactly, for every finite family. Since the condition is both necessary and sufficient, nothing is left open for finite families. It also shows that more memory does not help. Whenever some generator succeeds, even one that uses its whole history, a generator with at most 1+N2N1+N2^N states also succeeds. The insight behind it is that only the first proposal matters. Replay looks like an infinite game in which each new proposal hands the adversary new material to replay. But once the generator proposes LjL_j, the adversary can already replay all of LjL_j, so targets that agree outside LjL_j can never be told apart, and the family must contain a language inside all of them. Conversely, when this holds, our generator keeps every later proposal inside the union of LjL_j and the target, so later proposals give the adversary nothing new. Success is therefore decided by a single choice, the first proposal. This explains the three-language example (whatever the first proposal, the other two languages agree outside it, and no language lies inside both) and shows that it is the smallest obstruction. We found the result by noticing, through the library, that a construction from the bounded-memory paper P31 also defeats replay, and asking why.

Scope. The theorem (verified in Lean) covers finite families, deterministic generators, and P22's replay model; the countably infinite case remains open.

Dear reviewer, as the discussion period is nearing its end, we wanted to check whether we could clarify anything or address any additional questions. Thank you again for your time and constructive feedback!

Reviewer_dQnXOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents GenLimitLib, a library of formalized definitions, statements, and kernel-checked proofs sourced from the literature on Language Generation in the Limit. The authors introduce the design of the library and showcase its utility in two ways. First, three direct results are presented as follow-ups of the formalization effort. Then, an evaluation is presented as evidence that GenLimitLib can help LLMs develop new Lean proofs and better understand papers in this line of work.

Critical strengths
  1. The paper's idea of lean formalization is timely and aligned with recent strides in using LLMs for formalized proof generation.

  2. The paper shows clear use cases of the library, with Section 4 containing a repaired gap in a published proof, a construction transferred between papers, and a resolution of a special case of an open problem.

Critical weaknesses
  1. The correctness of GenLimitLib relies on the formalization of the 405 claims. However, the description of the audit of these claims is unclear. Without a better description of the entire auditing process, it is difficult to judge the soundness of the formalization, which is the foundation for all subsequent results.

  2. The causes of the improvement in Table 2 are not entirely clear. In particular, only two of the five tasks show any significant change from P to PML. Additionally, these tasks are never stated, so it is difficult to judge why D and E are so different from A-C, and in general it's impossible to judge if these tasks are substantive results or routine corollaries.

  3. The experiments in Section 5.1 are conducted using gpt-5.6-sol as problem solver. However, newer models have been released which have shown promising results in their general-purpose harnesses. The long-term utility of GenLimitLib is unclear if we reach frontier models that can rebuild the relevant infrastructure with the same resources.

  4. The experiments in Section 5.2 rely on selecting "Relevant Lean" excerpts, which implies that such parts would be chosen to be relevant to both the statement and the target solution. This undermines the generalizability of these results for the realistic case where we send queries to the LLM whose answers we simply do not know.

Questions for authors
  1. Can you provide additional information for tasks A-E, such as natural language statements, which allow further judgement of the difficulty of the tasks, further clarifying the utility of GenLimitLib?

  2. Can you re-run the experiments from 5.1 using a more capable model such as GPT-6 Astra? This will help highlight the long-term utility of GenLimitLib as newer and more capable models get released.

  3. As stated in the conclusion, the method is general and can be applied to other research areas. What are the challenges in expanding this system to multiple lines of work or more mature research areas with larger paper libraries?

5 comments

We thank the reviewer for their careful reading and constructive suggestions.

W1: Audit description

Thank you for asking us to clarify the audit. Our audit has three layers:

  1. Lean kernel verification. All formalizations compile without sorry, admit, or project-defined axioms. The Lean kernel verification guarantees the correctness of the formal proof, but does not guarantee that the formal statement faithfully captures the corresponding claim in the source paper. So, we use LLM-based and human audits to check faithfulness.
  2. LLM-based alignment check. In a separate review step, an LLM first translates each Lean statement into natural language without seeing the source paper; it then compares the resulting description with the corresponding claim in the paper to identify potential mismatches. We have LLM-based checks for all source claims.
  3. Human audit. This audit is done by human experts familiar with the source papers. They compare Lean statements with the corresponding claims in the source papers to assess the faithfulness of the formalization. This review step is important because natural-language statements can be ambiguous, and an LLM may not reliably identify the intended meaning. Human audits currently cover 122/147 (83.0%) of the main theorems in papers and 167/405 (41.2%) of the source claims (including supporting lemmas, definitions, examples, and other claims).

We have included a description of the audit and its coverage in Section 3.3 (in blue in the revision).

W2/Q1: Task statements, relation to prior results, and causes of improvement

Thank you for suggesting us to clarify the tasks.

Below, we summarize the five tasks and the mathematical work needed. In short, tasks A and B develop new arguments, C extends an existing construction, and D and E rely more heavily on adapting and combining existing results, with E adding a new density analysis. Tasks A-E cover a wide range of potential library reuse.

TaskInformal statementRelationship to existing resultsSuccesses /20: P / PML / PML-Oracle
AAn uncountable family is uniformly generatable without samples. However, every deterministic generator that generates in the limit for this family must have upper density zero on some target language, even with one target-membership query per round.Combines membership-query feedback with a density obstruction.13 / 15 / 15
BFor a finite language family, the presenter reveals an infinite set E contained in the infinite target language K. One generator achieves lower density at least max⁡(0.5dK(I(E)),dK(I(E)∖E))\max ( 0.5 d_K(I(E)), d_K(I(E)\setminus E) ), where I(E)I(E) is the intersection of all family members containing E and dKd_K is the lower density.Strengthens partial-enumeration guarantees of paper P39 for finite family.19 / 19 / 19
CA single oblivious stream is a valid vanishing-contamination presentation of two nested target languages. Any randomized generator that generates in the limit almost surely for both has expected upper density zero on the larger target.Extends the vanishing-contamination deterministic construction in P17 to randomized generators and an oblivious-stream formulation; the full task also establishes a finite-family minimax result.16 / 20 / 18
DEvery countable family admits a generator achieving lower density 1/2 with finitely many corrupt input occurrences, allowing repeated input observations.Combines the finite-expansion approach in P17 with the half-density guarantee in P39, handling repeated inputs and transferring the density guarantee back to the original target.0 / 20 / 20
EFor each q≥0q\ge 0: (i) every countable family admits a generator with lower density 1/2 under at most q distinct noisy values; (ii) there exists an uncountable family that admits lower density 1/4 at noise level q, but no generator can generate in the limit at level q+1.Part (i) combines finite expansion in P17 with half-density generation in P39. Part (ii) uses P12’s noise-level separation construction, but adds a new least-unannounced sweep and a density analysis.0 / 5 / 9

Why D and E are different. Tasks D and E differ from A-C in their heavier reuse of existing formalized results in the sub-library. Their successful PML proofs have library dependency shares of 91.9% and 87.3%, the highest among the five tasks (Table 2(b)). Library access provides substantial mathematical components that P must reconstruct. This helps explain why PML has the largest success gains in D and E.

On tasks A-C, PML also lowers API cost per valid proof by approximately 10%, 28%, and 57%, respectively (Table 10). Success counts also increase on A and C (13 → 15 for A and 16 → 20 for C); B is already near the ceiling (19/20 for both P and PML).

We have added the task summary table and full theorem statements to Appendix B (also in our experiment repository at submission).

W3/Q2: GenLimitLib utility with a stronger model Astra

Thank you for raising this question. We reran Tasks A-E under P and PML with gpt-6-astra at high and medium reasoning, without a supplied proof. We used five runs per task, resource condition, and effort level. All 100 runs succeeded. This includes condition P on tasks D and E, where gpt-5.6-sol failed in all 40 P runs. On these tasks, the stronger model can reconstruct the required infrastructure with the same time budget and resources.

Library access still improves efficiency on four of the five tasks. On tasks D and E, PML reduces runtime by 59%-74% and output tokens by 60%-72% across the two effort levels. B and C also show reductions: 12%-32% on runtime and 16%-28% on tokens. A shows no consistent reduction, possibly because its relatively limited reuse does not offset the time and tokens needed to read library material.

Thus, for stronger models, the benefit of the library may shift from proof success toward proof-development efficiency.

We report percentage changes in runtime and token usage under PML relative to P, with high/medium reasoning in each cell. Since we ran these experiments using a GPT subscription, we do not report API cost. We have added this experiment in Appendix B.4 (in blue in the revision).

TaskRuntime (%)Output tokens (%)Model-written lines (%)
A+13 / −1+9 / +1+8 / 0
B−12 / −24−16 / −21−7 / −4
C−32 / −29−28 / −24−28 / −24
D−74 / −70−72 / −69−74 / −72
E−62 / −59−64 / −60−64 / −66

W4: Section 5.2 experiment Lean selection

Thank you for this point. We agree that, besides the question, the original selection of "relevant Lean" also used the source of each answer (the result in the paper the answer is based on), which one cannot know without knowing the answer. We have described this selection more clearly in Appendix C, and we have added an experiment in which the Lean is chosen from the question alone.

In the new experiment, for each question we take the Lean statements of the paper's library module and the shared Core library, rank them by how many words they share with the question, and keep the top-ranked ones up to the same 6,000-character limit as before. We make sure that the selection uses only what is known before answering, namely the paper and the question; it never sees the answer, the options or the answer's source. The model therefore sees the same amount of Lean as before (about 44 statements); only the choice of statements changes.

Reference materialAccuracy (%)
Paper69.7
Paper + unrelated Lean (same length)71.5
Paper + Lean chosen from the question only (new)76.9
Paper + Lean chosen using the answer's source (original)78.6

Lean chosen from the question alone improves accuracy by 7.2 points over the paper (95% CI [4.5, 10.1]) and by 5.5 points over unrelated Lean of the same length ([2.8, 8.2]). Our interpretation is that most of the benefit does not depend on knowing where the answer comes from: about four fifths of the original gain (7.2 of 8.9 points) remains when only the paper and the question are used, and the original selection serves as a best case. We have added this result to Section 5.2 and Appendix C (in blue in the revision).

Q3: Challenges in expanding the system

Thank you for this interesting question. We see the following challenges.

Multiple research lines. The main challenges are finding real connections between different research lines and designing shared foundations that stay small, readable, and reusable. In GenLimitLib, we let human experts decide which definitions belong in Core, which proof components belong in Support, and which cross-paper relations to record as Bridges. This becomes harder when papers are from different research lines.

Larger libraries. In more mature areas, textbooks and surveys can provide stable definitions and foundations. However, larger literatures also bring challenges in retrieval, audit, and maintenance. Retrieval becomes harder as the search space grows. In GenLimitLib, we have claim and declaration cards to support retrieval. Papers are also revised over time, so updates should stay local. GenLimitLib tracks changes to audited statements and definitions. It flags only the affected parts for reaudit.

We are optimistic that our method will be useful across more research lines and larger libraries. We plan to test this in future work.

Dear reviewer, as the discussion period is nearing its end, we wanted to check whether we could clarify anything or address any additional questions. Thank you again for your time and constructive feedback!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.