- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Paper2Lean, an end-to-end agentic framework that translates mathematical papers into Lean. The system organizes the task using a dependency graph and uses multiple specialized agents for statement formalization, semantic checking, and proof generation. The system is evaluated on 20 mathematical optimization papers. The authors report that all automatically generated projects compile without proof placeholders. However, expert review gives a source-fidelity rate of 95.7%, identifying cases that compile but are unfaithful to the original papers. After correcting the identified errors, the authors release OptPaper-20, a corpus of 20 paper-level formalizations reviewed by experts.
Critical strengths
-
The authors show the system's ability to handle interdependent mathematical content in 20 mathematical optimization papers and release the formalization corpus OptPaper-20 following human review and semantic repair. The corpus includes Lean code and corresponding natural language descriptions, offering reusable resources for developing and evaluating paper-level autoformalization systems.
-
With both compilation checks and expert review, the authors identify cases that compile but are unfaithful to the original papers. These analyses help readers understand the system's specific failure modes and provide useful insights for improving semantic fidelity.
Critical weaknesses
-
The novelty over M2F is not clear. M2F already supports document-level formalization, dependency management, and separate statement and proof generation. Without a direct comparison or ablation studies, it is difficult to judge what improvements come from Paper2Lean's additional designs.
-
The criteria for semantic fidelity need clarification. The pipeline allows assumptions to be added during proof generation without requiring another automated semantic check. It is also unclear how expert review distinguishes recovering existing assumptions from adding conditions that weaken the original statement, as illustrated by Appendix D.1. This affects the interpretation of the reported fidelity rate.
Questions for authors
-
What are the key additions in Paper2Lean compared with M2F? Could the authors explain what additional capabilities or improvements they bring, and whether there are existing comparison or ablation results supporting them?
-
In Appendix D.1, is the added convergence condition already part of the original theorem's assumptions, or can it be derived from them? Please provide the relevant source text or derivation. How does the final expert review track these statement changes and distinguish recovering existing assumptions from adding conditions that weaken the original statement?