acceptodds
Under review as a conference paper at ICLR 2027

Maestro: Agentic Formal Proof Development by Coordination

15 views

Abstract

Lean is increasingly used to verify mathematical proofs and software produced by LLM agents, but writing formal proofs remains hard. Existing prover systems take a divide-and-conquer approach that recursively decomposes hard proof obligations into easier lemmas and splits again any lemma that resists proof. However, a lemma can resist proof for different reasons, including insufficient guidance, a false statement, or a flaw in the decomposition that produced it. Existing systems cannot tell these apart as a failed attempt reports little beyond its failure, and responses are confined to the failing node without considering the rest of the proof or revising the plan above it. The system may thus keep decomposing obligations along a flawed proof direction until the budget runs out, causing the whole proof attempt to fail. We introduce Maestro, a Lean prover harness that handles failures through global coordination. Provers report why they failed, not just that they failed, and a coordinator that sees the whole proof chooses an appropriate response, such as guiding a stuck prover, correcting a false lemma, or retracting a flawed subtree and replanning it. The same harness extends to verified code synthesis, where a checked counterexample sends the code back for repair. We evaluate Maestro with four frontier LLMs on seven mathematics and three software verification benchmarks. It proves more targets than each model's native coding agent (Claude Code or Codex) on every benchmark the agent has not already saturated, with the largest gains on research-level mathematics. With Claude Opus 5, it discovers its own proofs of 34 of the 48 theorems in the June 2026 release of arXivLean, against 19 for the native agent. It also outperforms all published prover systems on every benchmark they share, solving 669/672 PutnamBench, 59/60 Lean-IMO-Bench, and 6/6 IMO 2026.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Automated theorem proving, Formal verification, Lean, Agent coordination

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7SGbOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors present a harness for formal math reasoning and code verification using Lean. The core idea is to have a coordinator model that will revise the entire proof plan based on the feedback from prover subagents. The paper applies the harness to strong frontier models (Fable, Sol, Opus) and shows strong results on formal math and code verification benchmarks.

Critical strengths
  • The main results (Table 1) are strong. The proposed harness outperforms the native harnesses of the base models (Claude Code and Codex respectively), at any level of reasoning effort. The improvement on FATE-X, a less saturated benchmark, is particularly noteworthy.

  • The case studies in Section 4.5 (particularly, the first one) supports the motivation that repairing a subgoal locally can get stuck in a loop, whereas the harness can escape it via a global coordinator revising the entire proof strategy.

Critical weaknesses
  1. (Major) The novelty of the paper seems overstated.
  • The paper claims that its main contribution is: its global coordinator is able to receive feedback from prover subagents and repair the overall proof strategy (lines 20-24, 142-149). However, the authors seem to mischaracterize related works, e.g., Godel-Architect [1] and Hilbert [2]. Appendix C.3 in the Goedel-Architect paper shows that their blueprint refining step is also allowed to change the global structure of the blueprint: “If the lemma is structurally unfixable, drop it and re-route the nodes that depended on it.” Hilbert also mentions that “Any remaining unsolved subgoals at this point trigger a failed proof attempt, prompting us to restart the subgoal decomposition process for the theorem” (Section 3.2.2).
  1. (Major) The paper does not control for, or report sufficiently, on the cost.
  • The paper misses most details about the task budget allocated to different components of the harness: e.g., per-agent budget, maximum number of retries, total amount of API cost / number of tokens.

  • Prior systems (Section 4.2) mostly use older or cheaper models (e.g., DeepSeek-V4-Flash). It is unclear if the improvement in the performance is due to an improvement in the base model or the harness design.

  • The paper also compares against the same base model + native harness (Claude Code or Codex). However, since the native harness is a general-purpose harness, it is not surprising that a specially designed harness would outperform it. Also, when comparing the computation cost of the two harnesses (Appendix A), there are a few questionable choices: 1) on “Medium” questions where the native harness gets 1 or 2 correct out of 3 tries, the comparison is between 1 run of Maestro vs. all 3 runs of native harness; 2) no cost is reported on runs where Maestro fails. Intuitively, if the coordinator repeatedly tries to repair a proof, the total cost might blow up.

  • Similarly, when comparing against the baseline without using a global coordinator (Section 4.4), replicating the efforts of previous harness designs, the paper does not report any compute cost of the baseline.

  1. (Major) Missing details on the advantage of the global coordination
  • The paper misses quantitative analysis of 1) how often the global coordinator revises the proof; 2) whether revising the proof meaningfully contributes to a success.

  • Even in Section 4.4 where they compare against a baseline without using a global coordinator, the authors fail to provide enough experimental detail for this baseline. The list of agent roles is missing and so is the system prompt for each agent.

[1] Chung et al., Goedel-Architect: Streamlining Formal Theorem Proving with Blueprint Generation and Refinement, arXiv 2026.

[2] Varambally et al., Hilbert: Recursively Building Formal Proofs with Informal Reasoning, ICLR 2026.

Questions for authors
  1. What exactly is the difference between Maestro and existing blueprint-based systems like Goedel-Architect and Hilbert? Please clarify which part of the “global coordination” is novel in this system.

  2. How does Maestro compare against other baselines under the same total cost budget? Please provide a matched cost comparison: e.g., accuracy vs. budget results against at least the native harness and the no-coordinator baselines. Please also clarify the total / per-agent resource limits. Please also report on the cost of the unsuccessful runs.

  3. (Minor) How were the submitted Lean proofs validated? Were there checks on ‘cheats’ (e.g., agents rewriting the target statements)?

  4. (Minor) Can the authors clarify some minor discrepancies in the manuscript? For example, Appendix B says each configuration is run three times, whereas the caption in Figure 5 says the numbers were averaged across two or three runs. Also, Section 3.1 says “The planner revises until the judge accepts” (lines 211-212), but Appendix E shows that “If six proposer-judger revisions do not pass, the latest translatable ordinary plan may publish with a JUDGER-REJECTED FALLBACK label.” (lines 1681-1682) Even more minor, but please clarify that ArxivLean008 in line 460 is Problem 8 from the March 2026 release of the ArxivLean. As of right now, it is confusing as to whether it is from the March / June split.

2 comments

We thank the reviewer for the thoughtful review. The suggestions are valuable for us to improve our work. Below, we provide a point-by-point response to the key concerns in our paper.

Q1 & W1: Clarification on the contribution of this paper

Our central motivation is that when a proof plan leads to a subgoal that resists proof, there are two important decisions to make: whether and how to revise the proof plan, and how to pursue the open goal. A mathematician might give a promising argument more time, try an idea that worked on a related subgoal, introduce more helper lemmas, or reconsider the strategy that produced the subgoal, and they do this by careful reasoning over the current overall status. In a fixed pipeline like Goedel-Architect, however, these two decisions are largely fixed, and not determined from any reasoning. Maestro gives both decisions to a coordinator that reasons from the current proof state and the evidence gathered across the proof development history. The central technical details were provided in the original submission, and we will reiterate in detail how, in Maestro, the agent made these two important decisions compared to Goedel-Architect.

Revising the proof plan. Indeed, Goedel-Architect already uses structured diagnoses and formal disproofs to revise its blueprint globally. Its refinement prompt defines a largely fixed behavior, where the model is asked to repair or remove a lemma judged false and to introduce helper lemmas when a statement appears true but too hard to prove. This basically maps “wrong lemma” to a local lemma repair, and “a hard yet true lemma” to further decomposition. However, this decision is not optimal. It’s very possible that the whole proof development has gone down an unproductive route, and the suitable repair is to fully discard the current route and reconsider another possible direction. Maestro therefore extends this refinement mechanism by letting the coordinator decide whether and how to revise the proof blueprint itself based on its reasoning about the whole proof development trajectory instead of any fixed mapping. For example, the coordinator was observed to revise a hard yet true lemma that resists several rounds of reproof and decomposition by discarding the whole route and building scaffolds that lead to a completely new proof strategy.

Pursuing the open goal. When a goal is diagnosed as PROOF_TOO_HARD, Goedel-Architect’s refinement prompt directs the model to add helper lemmas. Maestro extends this behavior by supporting multiple forms of worker dispatch (section 3.2) and letting the coordinator choose how to pursue each open goal. Dispatching a decomposer recovers Goedel-Architect’s prescribed response to this diagnosis, but the coordinator can also give a promising prover more budget, retry with targeted guidance, or launch an independent attempt. For example, if progress on a sibling lemma or the coordinator itself sees a useful reduction, the coordinator can pass it along as a hint and give the prover another chance to close the goal directly, without adding new proof scaffolding. If the same obstacle persists after several rounds of adding helper lemmas or resists several rounds of different strategies, the coordinator can reconsider the proof plan itself, launch independent explorers to investigate alternatives, and abandon the stalled route if a more promising one emerges. All these described behaviors are outside of Goedel-Architect’s capability.

W2 & Q2 & W3: Comparison with Goedel Architect with matched budget

We add two same-backbone comparisons with Goedel-Architect at matched budgets. In particular, we use 1) Claude Opus 5 with high reasoning effort as the backbone on the FATE-X dataset; 2) Qwen3.8 27B with xhigh reasoning effort as the backbone on the AlgoVeri dataset with only proof generation. We compare Maestro with Goedel-Architect’s official code release and report the solved rate with a matched cost budget. We can see that, for each budget, our method achieves a similar or higher solve rate compared to Goedel-Architect. With a growing budget, Goedel-Architect’s solve rate plateaus at 80% on FATE-X and 36% on AlgoVeri while our proposed method reached 93% and 56%, respectively. This demonstrates the superiority of our method. We revised our paper by adding a visualization of these results in Appendix C.

Table 1: Same backbone comparison with matched budget on FATE-X

Method$0.25$0.50$1$2$5$10$20$50$100$200$500
Goedel-Architect0%2%6%22%43%50%61%74%79%79%80%
Maestro1%3%6%21%37%51%63%80%84%90%93%

Table 2: Qwen3.8 backbone comparison with matched budget on AlgoVeri

Method$0.01$0.02$0.05$0.10$0.20$0.50$1$2$5
Goedel-Architect0%3%6%8%13%22%32%36%36%
Maestro1%4%9%13%22%30%42%48%58%

Regarding the cost of failed runs, Maestro keeps attempting a proof until the budget is exhausted. When given a hard task, we want the agent to continue pursuing a solution and trying different ideas instead of giving up. Although this can incur high costs (the budget limit) on failed problems, it also increases the chance of solving a problem.

We also analyzed our proof traces on FATE-X and found that only approximately 5.6% of coordinator turns request decomposition alone. Other turns can combine redirecting provers, launching independent explorers, and making substantial edits to the workspace. Moreover, we believe the benefits of our pipeline can be clearly shown through the same-backbone, matched-budget comparison with Goedel-Architect.

Q3: Lean checking details

All proofs were checked by Lean, and we additionally verify that the original statement remains unchanged and audit the final proof to scan for any “sorry” or “admit” placeholders and disallowed axioms. A proof is counted as successful only if it passed all these checks.

Q4: Response to minor discrepancies

  1. We thank the reviewer for identifying the discrepancies. Figure 5 reports results averaged over three runs. The caption was incorrectly retained from an earlier version of the figure. We will correct it in the revision.

  2. In the implementation, we limit the planner-judge revision loop to six rounds to prevent repeated rejections from exhausting the available budget. If no plan is accepted within this limit, the latest translatable ordinary plan may be published with a JUDGER-REJECTED FALLBACK label, as described in Appendix E. The rejection feedback allows the coordinator to reassess the overall proof development and may prompt a more substantial revision of the proof plan. We will clarify this detail in Section 3.1.

  3. Unclarity in arXivLean release. We thank the reviewer for pointing out the minor discrepancies. We fixed them in the paper.

We thank the reviewer for the constructive feedback. If our response and revision address your concerns, we would be sincerely grateful if you would consider raising your score.

Reviewer_KAdnOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes Maestro, a multi-agent harness for the formal math proving and the verified code generation with Lean. A coordinator agent receives worker agents’ proofs, and decides on actions for proving. In experiments, it reports higher solve counts than standard Codex/Claude Code and even some of prior systems across multiple backbone models and benchmarks. However, I assume that the paper does not clearly explain the truly new method contribution it introduced in this paper compared to the previous studies. Compute-controlled comparisons from the prior studies are not fully presented. The experimental results show strong performance, but I assume the current manuscript is still insufficient as a research contribution/technical novelty cannot be fully demonstrated.

Critical strengths

S1. Revising a global proof plan based on local failures is practically important for long formal proofs.

S2. This paper covers multiple scenarios of formal math with multiple backbone models. The reported gains on research-level mathematics benchmarks can be notable. Comparisons with both commercial and publicly-available models are also performed.

Critical weaknesses
  1. The method contribution.

W1. The main technical contribution is still unclear. In this paper, authors repeatedly state “global coordination” as the main contribution, while it leaves unclear what are the new factorized mechanisms compared to the existing agent orchestration or proof-plan refinement methods. Although the paper lists techniques without explicitly clarifying the proposed or existing, the core scientific contribution in this paper is a bit unclear.

W2. The description for the difference from the prior work is limited. Goedel-Architect have already provided the global blueprint refinement. It is recommended that authors should explicitly address this overlap and identify what Maestro newly adds.

W3. Methods section does not adequately support human understanding and assessment. Section 3 concentrates on conceptual and illustrative descriptions of a coordinator making appropriate decisions, without clarifying the technical innovation.

  1. Experimental results.

W4. The performance gains aren’t fully separated from compute budgets difference. Table 1 seemingly reports the same backbone results while the total compute budgets may not match. If Maestro used further workers, the higher solve counts don't sorely come from the proposed. Appendix A is helpful to some extent while it covers only some models on FATE-X and mainly compares output tokens conditioned on Maestro’s solve counts.

W5. The ablation studies are included but limited. In Section 4.4, the demonstrated experiment only shows that the full maestro system outperforms the naive decomposition variation with limited details. I recommend a comparison with Goedel-Architect, which has already supported the similar global refinement, otherwise the effectiveness of the claim doesn’t sorely become clear.

W6. Strong previous results are still omitted in Table 2, for example, Logical Intelligence’s Aleph Prover, which reported 668/672 and later 672/672 on PutnamBench. Although their experimental conditions may vary, it would be good to discuss and clarify the relation to those differences.

Questions for authors

See weakness.

2 comments

We thank the reviewer for the thoughtful review. The suggestions are valuable for us to improve our work. Below, we provide a point-by-point response to the key concerns in our paper.

W1 & W2 & W3: Central Contribution

Our central motivation is that when a proof plan leads to a subgoal that resists proof, there are two important decisions to make: whether and how to revise the proof plan, and how to pursue the open goal. A mathematician might give a promising argument more time, try an idea that worked on a related subgoal, introduce more helper lemmas, or reconsider the strategy that produced the subgoal, and they do this by careful reasoning over the current overall status. In a fixed pipeline like Goedel-Architect, however, these two decisions are largely fixed, and not determined from any reasoning. Maestro gives both decisions to a coordinator that reasons from the current proof state and the evidence gathered across the proof development history. The central technical details were provided in the original submission, and we will reiterate in detail how, in Maestro, the agent made these two important decisions compared to Goedel-Architect.

Revising the proof plan. Indeed, Goedel-Architect already uses structured diagnoses and formal disproofs to revise its blueprint globally. Its refinement prompt defines a largely fixed behavior, where the model is asked to repair or remove a lemma judged false and to introduce helper lemmas when a statement appears true but too hard to prove. This basically maps “wrong lemma” to a local lemma repair, and “a hard yet true lemma” to further decomposition. However, this decision is not optimal. It’s very possible that the whole proof development has gone down an unproductive route, and the suitable repair is to fully discard the current route and reconsider another possible direction. Maestro therefore extends this refinement mechanism by letting the coordinator decide whether and how to revise the proof blueprint itself based on its reasoning about the whole proof development trajectory instead of any fixed mapping. For example, the coordinator was observed to revise a hard yet true lemma that resists several rounds of reproof and decomposition by discarding the whole route and building scaffolds that lead to a completely new proof strategy.

Pursuing the open goal. When a goal is diagnosed as PROOF_TOO_HARD, Goedel-Architect’s refinement prompt directs the model to add helper lemmas. Maestro extends this behavior by supporting multiple forms of worker dispatch (section 3.2) and letting the coordinator choose how to pursue each open goal. Dispatching a decomposer recovers Goedel-Architect’s prescribed response to this diagnosis, but the coordinator can also give a promising prover more budget, retry with targeted guidance, or launch an independent attempt. For example, if progress on a sibling lemma or the coordinator itself sees a useful reduction, the coordinator can pass it along as a hint and give the prover another chance to close the goal directly, without adding new proof scaffolding. If the same obstacle persists after several rounds of adding helper lemmas or resists several rounds of different strategies, the coordinator can reconsider the proof plan itself, launch independent explorers to investigate alternatives, and abandon the stalled route if a more promising one emerges. All these described behaviors are outside of Goedel-Architect’s capability.

W4 & W5: Compute & Comparison with Goedel Architect with matched budget

We appreciate the reviewer’s suggestion in adding a same-backbone comparison with Goedel-Architect and agree that adding this experiment helps better illustrate the contribution of Maestro’s harness. We add two same-backbone comparisons with Goedel-Architect at matched budgets. In particular, we use 1) Claude Opus 5 with high reasoning effort as the backbone on the FATE-X dataset; 2) Qwen3.8 27B with xhigh reasoning effort as the backbone on the AlgoVeri dataset with only proof generation. We compare Maestro with Goedel-Architect’s official code release and report the solved rate with a matched cost budget. We can see that, for each budget, our method achieves a similar or higher solve rate compared to Goedel-Architect. With a growing budget, Goedel-Architect’s solve rate plateaus at 80% on FATE-X and 36% on AlgoVeri while our proposed method reached 93% and 56%, respectively. This demonstrates the superiority of our method does not come from the additional compute. We revised our paper by adding a visualization of these results in Appendix C.

Table 1: Same backbone comparison with matched budget on FATE-X

Method$0.25$0.50$1$2$5$10$20$50$100$200$500
Goedel-Architect0%2%6%22%43%50%61%74%79%79%80%
Maestro1%3%6%21%37%51%63%80%84%90%93%

Table 2: Qwen3.8 backbone comparison with matched budget on AlgoVeri

Method$0.01$0.02$0.05$0.10$0.20$0.50$1$2$5
Goedel-Architect0.0%2.6%6.5%7.8%13.0%22.1%32.5%36.4%36.4%
Maestro1.3%3.9%9.1%13.0%22.1%29.9%41.6%48.1%58.4%

W4: Comparison with Aleph Prover

We appreciate the reviewer’s suggestion to add Aleph Prover as a baseline. We updated Table 2 by adding Aleph Prover to the comparison. Both Aleph Prover and Maestro saturated the two benchmark datasets. We have revised the paper to also include Aleph Prover in Table 2.

SystemBase modelPutnamBenchLean-IMO-Bench (60)Verina (189)AlgoVeri (77)
Aleph Prover—668—189—
MaestroGPT-5.4——18870
Maestro4 evaluated models663-669/67257-5918975-76

We thank the reviewer for the constructive feedback. If our response and revision address your concerns, we would be sincerely grateful if you would consider raising your score.

Reviewer_9ARxOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper considers the topic of formal reasoning with Lean language and the task of automated theorem proving. It devises an agentic system which keeps track of the global structure of a generated formal proof and tries to either complete the proof or discard it as soon as it can verify the proof is not plausible. The agentic system consists of components such as provers, planners, a falsifier, etc. It experiments on 4 closed source models (Claude Opus 4.8, Claude Opus 5, Claude Fable 5, and GPT-5.6 Sol). It compares the resulting success rates with several baselines in the literature, and reports improvements in the number of proven theorems on benchmarks such as PutnamBench.

Critical strengths

The topic is very interesting.

The paper is well written and it is easy to understand the results and experiments.

It makes an attempt to improve SOTA accuracies on various benchmarks.

Critical weaknesses

I find the results premature and perhaps preliminary. There are many shortcomings and lingering questions in the current results in my understanding of the paper. Some of them might be lack of reporting, but others might be lack of consideration by the authors.


First, I found the comparisons in Table 2 not well designed. For example, the accuracy reported for Maestro and Goedel-Architect are based on completely different LLMs, possibly with very different computational costs. Maestro uses Claude Opus 4.8, Claude Opus 5, Claude Fable 5, and GPT-5.6 Sol while Goedel-Architect uses DeepSeek-V4-Flash. Moreover, the paper reports, that for Maestro, high reasoning effort is used while it seems that DeepSeek-V4-Flash does not use a high reasoning mode. Therefore, it is not clear where the marginal accuracy gains of Maestro come from.

It would be critical for the paper to report exact and clear computational costs including token budgets of all the components in its own system and contrast it with those of the baselines in the literature considered in its Table 2.


Second, all the experiments are on close source proprietary models. I did not see any results reported on open source models. It would be necessary for the paper to support its claims by demonstrating that it can improve the accuracy of open source models as well. This also relates to the matter of reproducibility by the community. The proprietary models that paper has experimented with will not remain accessible to the community in the near future when new versions are rolled out.


Third, I found the literature review considerably weak. Below I mention a few papers from the literature but I highly encourage the paper to enhance its literature review going beyond these papers.

Lego-Prover: Neural theorem proving with growing libraries, ICLR 2024.

ProofAug: Efficient Neural Theorem Proving via Fine-grained Proof Structure Analysis, ICML 2025

Editable Proof Sketch for Automated Theorem Proving, ICML 2026

Questions for authors

Please see weaknesses. I'd be happy to see any clarifications or additional results.

1 comment

We thank the reviewer for the thoughtful review. The suggestions are valuable for us to improve our work. Below, we provide a point-by-point address of the key concerns in our paper.

W1: Comparison with the same backbone LLMs & Cost Report & W2: Open-source models

We want to clarify the results in Table 2. At the time of ICLR submission, the published methods did not have code open-sourced yet (Goedel-Architect's code was released on Sep 25 after the ICLR submission deadline), so we took the methods’ solve rates from the corresponding papers. We also want to highlight that our method outperforms IDS with the same backbone model GPT5.4 on the Verina and the AlgoVeri datasets.

We appreciate the reviewer’s suggestion in adding a same-(opensource)backbone comparison with Goedel-Architect and agree that adding this experiment helps better illustrate the contribution of Maestro’s harness. We add two same-backbone comparisons with Goedel-Architect at matched budgets. In particular, we use 1) Claude Opus 5 with high reasoning effort as the backbone on the FATE-X dataset; 2) Qwen3.8 27B with xhigh reasoning effort as the backbone on the AlgoVeri dataset with only proof generation. We compare Maestro with Goedel-Architect’s official code release (where we found that the default reasoning is also high) and report the solved rate with a matched cost budget. We can see that, for each budget, our method achieves a similar or higher solve rate compared to Goedel-Architect. With a growing budget, Goedel-Architect’s solve rate plateaus at 80% on FATE-X and 36% on AlgoVeri while our proposed method reached 93% and 56%, respectively. This demonstrates the superiority of our method. We revised our paper by adding a visualization of these results in Appendix C.

Table 1: Claude Opus 5 backbone comparison with matched budget on FATE-X

Method$0.25$0.50$1$2$5$10$20$50$100$200$500
Goedel-Architect0%2%6%22%43%50%61%74%79%79%80%
Maestro1%3%6%21%37%51%63%80%84%90%93%

Table 2: Qwen3.8 backbone comparison with matched budget on AlgoVeri

Method$0.01$0.02$0.05$0.10$0.20$0.50$1$2$5
Goedel-Architect0%3%6%8%13%22%32%36%36%
Maestro1%4%9%13%22%30%42%48%58%

W3: Literature review

Thank you for pointing out the missing references. We have extended our literature review and have modified the paper’s related work section to include all the mentioned papers and more.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.