- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes IneqAgent, which combines lemma-level MCTS, LLM reasoning, and symbolic computation to prove inequalities in Lean 4. It solves 93.68% of 190 benchmark problems and verifies three mathematical problems involving matrix permanents.
Critical strengths
-
Combines multiple symbolic methods with Lean to generate formally verified proofs.
-
Solves 93.68% of 190 problems and demonstrates applications to matrix permanent conjectures.
Critical weaknesses
-
MCTS, SOS, Bernstein, and difference substitution are existing methods. Although the paper introduces lemma-level search and multiple symbolic backends, it mainly combines existing techniques without clear new algorithmic contributions.
-
IneqAgent solves only 8 more problems than the Base Agent (178 vs 170), without statistical significance tests. Qwen and Kimi perform worse without cost limits, and the cost analysis excludes symbolic computation and Lean verification.
Questions for authors
-
What are the new algorithmic contributions of IneqAgent compared with LIPS, NSPI, and existing SOS methods? Can their effects be evaluated separately?
-
Can the authors test the significance of the gains under equal end-to-end compute budgets, and explain why Qwen and Kimi perform worse without cost limits?
Response to Reviewer RNmW
We thank the reviewer for the valuable comments on algorithmic contributions, the significance of the gains, and computational budgets. We respond in two parts below.
Q1: Algorithmic contributions and differences from existing methods
IneqAgent is not a simple combination of existing techniques such as MCTS, SOS, Bernstein, and difference substitution. Its contribution lies in the design of a lemma-level MCTS search coupled with verifiable symbolic certificate generation. Numerical computation in SOS and Bernstein solving only assists exact certificate recovery; final certificates are still checked exactly and verified in Lean.
For the search, under the same 2-hour limit, lemma-level MCTS solves 178 problems versus 172 for BFS, with mean times of 142.60 s versus 196.90 s on the 169 solved by both. We evaluated the SOS and Bernstein procedures separately against existing approaches on all instances of the relevant subsets; an instance counts as solved only if its exact certificate passes checking.
SOS certificate generation (115 polynomial optimization instances):
Bernstein exact solving (45 box-constrained instances):
Our SOS procedure solves 51 instances versus 22 for Isabelle SOS, and reduces mean runtime by 38.6% on the 20 commonly solved instances. In the inspected Isabelle SOS timeout cases, we observed near-zero estimated minima, rational-recovery failures, and subsequent degree increases before the time limit was reached. For Bernstein, both use the same degree, subdivision, and time limit, with screening time included. Exact solving of the full LP is costly and most instances time out at the initial degree; the reduced LP raises solved instances from 5 to 28 and reduces mean runtime by 95.9% on the 5 commonly solved instances.
Q2: Computational budget, significance of the gains, and Qwen/Kimi results
Computational budget. We additionally measured the average per-problem time of IneqAgent with the main backbone, DeepSeek, on solved instances:
For comparison, the corresponding end-to-end time of Direct Agent (the Base Agent in the review) is 336.24 s. The 2-hour per-problem wall-clock limit includes all symbolic computation and Lean verification, and all local computations use the same hardware. The monetary cost covers only LLM API fees, not total compute; we will state this explicitly.
Significance of the gains. We thank the reviewer for the valuable suggestion. With DeepSeek, based on per-problem paired outcomes of IneqAgent and Direct Agent (9 problems solved only by IneqAgent, 1 solved only by Direct Agent), a two-sided exact McNemar test gives p=0.0215, significant at the 0.05 level for this DeepSeek configuration.
Qwen and Kimi. We acknowledge that these results show the framework does not improve the final success rate under every model and cost setting. In the inspected Kimi failures, the framework steers the model to call the backends early, but Kimi tends to hand the whole goal to them instead of first constructing lemmas, so calls time out and the budget is exhausted. Under the same 2-hour limit, Direct Agent instead spends this time on direct proof attempts. We have not yet determined the cause for Qwen. We will include this analysis in the revision and narrow our claims about gains across models accordingly.
We thank the reviewer again for the valuable suggestions. We hope these responses address the concerns, and we would be glad to discuss any further questions.