acceptodds
Under review as a conference paper at ICLR 2027

Formal Theorem Proving with Lean World Models

28 views

Abstract

Most recent neural provers write a whole formal proof before Lean checks it, so they run open-loop: without intermediate observations, a wrong assumption about the proof state can derail the attempt. Many also use informal reasoning distilled from larger, more expensive models before writing Lean proofs. We propose to train a Lean world model, where the prover predicts Lean's goal states and error messages alongside its proof steps, learning explicit Lean-level reasoning from Lean's feedback rather than from a large LLM teacher. Because it predicts errors, it can repair a proof within the same response, using the failed attempt and its predicted diagnosis. We train 4B and 8B Qwen3 models by supervised fine-tuning on feedback-annotated proofs followed by reinforcement learning (GRPO), and compare them with a standard prover trained through the same stages without Lean annotations. The 8B world-model prover outperforms the standard prover, reaching 75.61% pass@32 on miniF2F and 16.44% on ProofNet against 71.31% and 13.21%. Without informal reasoning, the world-model prover solves as many miniF2F problems as the standard prover further trained on teacher-written reasoning. Adding informal reasoning raises its miniF2F pass@32 to 77.05%, above DeepSeek-Prover-V2-7B and below Goedel-Prover-V2-8B, whose post-training techniques could complement our focus on learning from Lean's feedback.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
automated theorem proving, Lean, large language models, world models, reinforcement learning, proof-state prediction, compiler feedback, self-repair

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_erWNOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper trains Qwen3 4B and 8B provers to generate tactics with predicted Lean states and diagnostics, allowing repair within one response before final verification. Training combines annotated proof and repair data, SFT, and GRPO with feedback supervision and replay. The 8B model reaches 75.61% versus 71.31% pass@32 on combined miniF2F validation/test problems, and 16.44% versus 13.21% on ProofNet.

Critical strengths
  1. Predicted diagnostics and within-response repair extend proof-state prediction in a useful direction. The auxiliary feedback loss addresses a genuine mismatch: a valid final proof does not guarantee accurate predicted states or errors.
  2. The contribution includes substantial data construction and model training, supported by model-size comparisons, repair experiments, and feedback-accuracy measurements. The integrated training recipe is worthwhile.
Critical weaknesses
  1. The baseline omits both replay and feedback supervision, while the models use different self-distillation data and filtered RL pools. This prevents attribution of the gain specifically to feedback prediction. Section 2 discusses DeepSeek-Prover-V1.5, but the relation to ProofOptimizer and Spark-Prover-X1 also needs clarification. Novelty lies in the diagnostic-prediction and repair combination, not state prediction alone.
  2. Table 3 shows 6,153 versus 1,095 generated tokens per miniF2F response and 6,727 versus 897 on ProofNet. Both WM8 and S8 avoid intermediate Lean calls, so their comparison does not establish a speedup from replacing those calls. Token-matched and wall-clock comparisons against direct generation and actual-feedback repair are missing, as are finalized uncertainty estimates.
Questions for authors
  1. Does the gain persist with matched replay and RL data, and over a state-prediction-only model? Please compare the training and inference mechanisms with ProofOptimizer, Appendix A.1, and Spark-Prover-X1, Section 3.1.2.
  2. Under equal token or time budgets, how does predicted-feedback repair compare with direct generation and repair using actual Lean feedback?
  3. Please provide the counts and evaluation cohorts for Table 15. Table 3 implies 61 solved ProofNet problems for WM8, but 7.69% is not a rounded integer share of 61.
5 comments
AuthorsOpenReview ↗

Post 7: Response to Reviewer erWN (1/3)

Q1: Matched replay, RL data, and a state-prediction-only model

Reviewer comment/question. Does the gain persist when replay and RL data are matched between the models? Does it persist relative to a model that predicts proof states but not diagnostics or repairs?

Author response. WM8 keeps its advantage when S8 also uses replay, and over a model without repair supervision.

Replay. We add the supervised replay of WM8's GRPO stage to S8's GRPO stage. Replay has little effect on S8: over three evaluations, its pass@32 goes from 70.36 ± 0.66 to 71.45 ± 0.31 on miniF2F and stays at 13.93 on ProofNet, while WM8 reaches 75.54 ± 0.43 and 16.35 ± 0.41. WM8 keeps a lead of 4.1 and 2.4 points over S8 with replay. We will report this control in Section 5.2 of the revision.

RL data. Every prover selects its GRPO problems with the same rule, keeping the problems on which its own success rate is above zero and at most 0.75. This follows the (0, 0.75] range of Goedel-Prover-V2 and the moderate-success-rate selection of DeepSeek-Prover-V1.5 (Appendix A.3). The pools differ only because each prover applies this rule to its own success rates, so that it trains on problems at its own level of difficulty.

State prediction only. WM8-NR is this model. It keeps feedback prediction and both auxiliary losses but sees no repair examples. In initial SFT and self-distillation, it therefore learns to predict proof states without any error targets, and before GRPO it never repairs (Section 5.2). During GRPO, the auxiliary feedback loss adds error targets from its own failed attempts. Comparing it with WM8 and S8 shows two things (Table R8):

  • WM8 is stronger, and repair supervision makes the difference. After GRPO, WM8 reaches 75.54 on miniF2F and 16.35 on ProofNet, against WM8-NR's 73.91 and 16.17 (means of three evaluations). The training recipes of the two differ only in repair supervision. WM8 also leads right after initial SFT, where WM8-NR predicts only proof states (74.66 against 73.70 and 14.74 against 13.12, also means of three evaluations). At the same GPU budget, WM8-NR draws 9 more independent samples per problem than WM8 (41 against 32) and still loses by 1.5 points on all 859 problems (95% CI [+0.3, +2.7]). Repair also solves problems that no first attempt solves: in the separate pass@8 run of Table 15, 4.3% and 7.7% of the problems WM8 solves on miniF2F and ProofNet are solved only through repair.
  • Even without repair supervision, feedback prediction stays ahead of S8. At the same GPU budget, WM8-NR is ahead of S8 after GRPO, although S8 gets 4.7 and 3.2 times as many samples as WM8-NR.

Table R8. Pass-rate differences in points, each row at the same GPU budget, with paired bootstrap 95% confidence intervals over problems. Sample counts are per problem (for S8, on miniF2F and ProofNet).

Comparison at the same GPU budgetDifference [95% CI]: miniF2FDifference [95% CI]: ProofNet
WM8 (32 samples) − WM8-NR (41 samples)+1.8 [+0.2, +3.5]+1.1 [−0.2, +3.0]
WM8-NR (32 samples) − S8 after GRPO (149 and 103 samples)+1.0 [−0.1, +2.9]+0.6 [−0.6, +2.5]
AuthorsOpenReview ↗

Post 8: Response to Reviewer erWN (2/3)

Q2: Relation to ProofOptimizer and Spark-Prover-X1

Reviewer comment/question. Please clarify the relationship between the proposed training and inference mechanisms and ProofOptimizer (Appendix A.1) and Spark-Prover-X1 (Section 3.1.2). The novelty appears to lie in combining diagnostic prediction with repair rather than in state prediction alone.

Author response. We thank the reviewer for pointing out these connections. Both works use state prediction only as training data. Our prover goes further in three ways that neither work shares:

  • It predicts Lean's errors, not only states. WM8 predicts the proof state after each block of tactics, and Lean's error message when a tactic fails, inline in its own response.
  • It repairs from its own predictions. A predicted error starts a new attempt in the same response, without calling Lean. ProofOptimizer's repair scheme instead hands Lean's actual error messages to a separate, larger model, and Spark-Prover-X1 generates whole proofs without any repair from predicted errors.
  • It keeps learning these predictions during RL. The auxiliary feedback loss trains WM8 on Lean's annotations of its own sampled proofs throughout GRPO.

Our experiments show that these choices pay off. From initial SFT to the end of GRPO, WM8's miniF2F pass@32 rises from 74.66 to 75.54, while the standard prover's falls from 72.00 to 70.36 (means of three evaluations). At the same GPU budget, on all 859 problems together, WM8 beats the standard prover by 2.2 points (95% CI [+0.9, +3.6]) and WM8-NR, a feedback-predicting prover trained without repair examples, by 1.5 points (95% CI [+0.3, +2.7]).

In ProofOptimizer (Appendix A.1), tactic and proof-state prediction is one of five tasks used to fine-tune its general Lean base model: given the state before a tactic, the model predicts the tactic and the resulting state, within a mixture of about one billion tokens. At inference, ProofOptimizer shortens existing proofs iteratively, and its separate repair scheme passes Lean's error messages to Goedel-Prover-V2-32B (Section 3.2). Spark-Prover-X1 (Section 3.1.2) extracts "state-before, tactic, state-after" triples, has an LLM predict the next state with a natural-language explanation, keeps the predictions that match the ground-truth states, and uses them as pre-training data for state prediction. We will add both works to the related work and state these differences explicitly.

AuthorsOpenReview ↗

Post 9: Response to Reviewer erWN (3/3)

Q3: Equal-token or equal-time evaluation

Reviewer comment/question. Under equal token or wall-clock budgets, how does predicted-feedback repair compare with direct generation and with repair using actual Lean feedback?

Author response. We compare provers at the same GPU time, measured as wall-clock time on one H100 per problem, with the same n-gram speculative decoding for every prover. We use GPU time rather than tokens because tokens differ in cost: with speculative decoding, WM8's feedback tokens, which restate hypotheses and goals already in the context, are cheaper than its tactic tokens. The note at the end of our general response explains how we measure GPU time and why we use speculative decoding.

WM8 gets 32 samples per problem, and each prover in Table R9 gets as many samples as fit into the same average GPU time per problem. WM8 beats the standard prover S8, which writes proofs directly without feedback, although S8 gets 2.3 to 6 times as many samples. On all 859 problems together, WM8 leads S8 after GRPO by 2.2 points and S8 before GRPO by 1.4 points, both with 95% confidence intervals above zero. Self-repair also beats spending the same compute on more independent samples: WM8-NR predicts feedback but sees no repair examples in training, gets 41 samples to WM8's 32, and WM8 still leads by 1.5 points, again with a 95% confidence interval above zero. This budget is lenient toward S8, since it does not charge for verifying S8's extra candidates.

Table R9. WM8 after GRPO (32 samples per problem) against each prover at the same GPU budget. Samples: that prover's samples per problem on miniF2F / ProofNet. Other columns: WM8 minus that prover, in pass-rate points, with paired bootstrap 95% confidence intervals over problems. miniF2F + ProofNet: all 859 problems as one set.

ProverSamplesWM8 − prover [95% CI]: miniF2FWM8 − prover [95% CI]: ProofNetWM8 − prover [95% CI]: miniF2F + ProofNet
S8 after GRPO193 / 133+2.7 [+0.8, +4.7]+1.6 [−0.5, +3.8]+2.2 [+0.9, +3.6]
S8 before GRPO73 / 122+0.6 [−1.1, +2.4]+2.5 [+0.6, +4.6]+1.4 [+0.1, +2.8]
WM8-NR after GRPO41 / 41+1.8 [+0.2, +3.5]+1.1 [−0.2, +3.0]+1.5 [+0.3, +2.7]

Repair with actual Lean feedback costs far more for little gain. We measured it directly: giving WM8 Lean's actual feedback after every tactic, instead of its own predictions, makes inference take about three times as long on our 64-core machine and improves miniF2F pass@32 by only about one percentage point over WM8. In this run, time spent in Lean dominates inference. Each sample waits for Lean after every tactic. Informal reasoning gives WM8 a larger gain (75.61% to 77.05%, Table 3). WM8 instead predicts the feedback itself, repairs within one response, and sends only its final candidate to Lean.

Q4: Table 15 counts and evaluation cohorts

Reviewer comment/question. Please provide the counts and evaluation cohorts used for Table 15. Table 3 appears to imply that WM8 solves 61 ProofNet problems, but 7.69% is not a rounded integer share of 61. Please reconcile these quantities.

Author response. We thank the reviewer for checking these numbers so carefully, which helps us make the paper clearer. Table 3 and Table 15 come from different evaluation runs. Table 3 is the main pass@32 evaluation, in which WM8 solves 61 of the 371 ProofNet problems. The repair statistics of Table 15 come from a separate run with 8 samples per problem (pass@8), in which WM8 solves 52 ProofNet problems. Four of these are solved only through repair, which gives the 7.69% (4 of 52). The revised paper states that Table 15 comes from a separate run in the caption of the new main-text table that summarizes it, and we will add the sample count and these counts to both captions.

We thank the reviewer again for the careful reading and helpful suggestions. We hope our answers address the reviewer's concerns, and we are happy to answer any further questions.

Reviewer_Z8NEOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose a novel method for training theorem-proving language models using Lean. The main idea is training the language model as a Lean world model, i.e. the model is trained to produce proof steps, or tactics, (seen as actions taken by the model and affecting the underlying Lean state) as well as the feedback obtained by Lean when verifying those steps. At inference time, the resulting model directly predicts its own Lean feedback after each produced tactic (instead of relying on an explicit Lean call) and, then, to repair its proof based on it. To achieve this, the paper includes a complex training pipeline involving a first pretraining phase, a SFT phase on proofs with Lean feedback and repairs and a final RL phase with verifiable rewards (given by Lean).

Critical strengths

The paper introduces an interesting and novel way to enable formal reasoning capabilities into LLMs, leveraging the accurate Lean feedback on both successful and unsuccessful proofs. The method combines different ideas to achieve this, from the data-processing to the multi-staged training, the additional losses and the world-modeling analogy. Moreover, the extensive experiment sections demonstrate the benefits of all proposed components.

Critical weaknesses

Currently, I see two main related weaknesses. The first is in its motivation: throughout the paper, the underlying assumption is that there's an intrinsic value in the model learning a good world-model and replace the Lean feedback with its predicted diagnostics. However, I find this assumption a bit weak and not really obvious to me. I would appreciate if the authors could further comment on this. As far as I understand, the (necessarily more accurate) Lean feedback should be preferable to the (noisy) predicted feedback from the model.

In other words, I wonder what are the expected benefits of this world-modelling formulation (i.e. before running the experiments, what are the authors expecting to find?).

  1. Is it a training-time benefit because of the additional gradient coming from fitting the Lean feedback's tokens?
  2. Is it a inference-time benefit since it's cheaper to generate the feedback with the LLM than via a single Lean call? If so, can this be backed with some measurements? Is it both? Unfortunately, this is not really discussed in the current manuscript.

This reflects in the lack of an important baseline (if I understood correctly), which is the second weakness. Why not testing a version of the model which uses the real Lean feedback at inference time rather than the generated one? I expect this to achieve higher accuracy that the current model, but I guess with some delays if the inference-time benefit 2) above is valid. Similarly, the authors could consider the same baseline but trained masking the Lean feedback; if the proposed method works better, the results would confirm the training-time benefit 1) above.

Questions for authors

Did you consider the two baselines in the "weakness" section?

Sec 4.1, 2nd paragraph: is the baseline still including feedback in context (but not optimized through its logits) or is it just trained on the chain of proof "actions" / tactics? In the first case, does the baseline then use the real Lean feedback at inference time then? In the second, does it mean that the baseline is training on about 3.5x less tokens (according to Table 1)?

The comparison in Sec. 5.4 seems a bit biased as it focuses on the method comparing to the two baselines' worse performance on the two datasets, but I get the authors' point. Still, I wonder if there's other baseline models out there which would be more fair and interesting to compare to.

Minor question:

Sec. 4.2: Does WM8-NR see error-prediction targets during GRPO? or do you mean that during the auxiliary losses that are used together with GRPO?

3 comments
AuthorsOpenReview ↗

Post 5: Response to Reviewer Z8NE (1/2)

Q1: Expected benefits of the world-model formulation

Reviewer comment/question. What are the expected benefits of the world-model formulation before observing the experimental results? Is the benefit at training time, because fitting the Lean-feedback tokens supplies an additional gradient; at inference time, because model-generated feedback is cheaper than a Lean call; or both? If an inference-time benefit is claimed, can it be supported with measurements?

Author response. We thank the reviewer for this question. It touches on all the benefits we see in feedback prediction, and we will make them more prominent in the motivation of the revised paper.

Our starting point was that learning a world model is broadly useful in machine learning: agents that model the effects of their actions learn and plan better, as in world models (Ha and Schmidhuber, 2018) and Dreamer (Hafner et al., 2025). We expected the same for theorem proving. A prover that predicts the states and errors produced by its own tactics should understand Lean better and write better proofs. We expected both benefits that the reviewer names:

  • Training time. Lean's states and diagnostics give the model additional targets from the same proofs, at the cost of one Lean run per proof attempt. They teach the model how its tactics change the proof state.
  • Inference time. A prover that predicts the feedback can notice a failure and repair it within the same response, without re-running Lean. Large batched vLLM runs are never interrupted to wait for Lean.

Our measurements support both. WM8 learns from the same proofs as the standard prover S8 and also learns to predict Lean's feedback. At the same GPU budget, it beats S8 after GRPO by 2.2 points on all 859 problems (95% CI [+0.9, +3.6]), although S8 gets 193 and 133 samples per problem on miniF2F and ProofNet against WM8's 32. The inference benefit shows when we give WM8 Lean's actual feedback after every tactic instead of its own predictions: inference takes about three times as long on our 64-core machine, while miniF2F pass@32 improves by only about one percentage point over WM8. In this run, time spent in Lean dominates inference. Each sample waits for Lean after every tactic. WM8 instead runs without interruption and sends only its final candidate to Lean, where verification takes about 8 CPU-core-seconds per candidate.

Q2: Predicted versus actual Lean feedback, and the masked-feedback baseline

Reviewer comment/question. Lean's feedback is necessarily more accurate than the model's predicted diagnostics. Why should predicted feedback be preferable, and how should the accuracy–cost trade-off be understood? Did you consider (i) a version of the model that receives real Lean feedback at inference time and (ii) a corresponding model trained with the Lean-feedback tokens masked from the loss? The first comparison would test the proposed inference-time benefit, while the second would test the proposed training-time benefit.

Author response. On miniF2F, predicted feedback gives WM8 almost the same result as actual feedback, in about a third of the inference time. We measured the first baseline. The second would depend on the same costly actual feedback at inference, so we did not train it.

(i) Actual feedback at inference. Giving WM8 Lean's actual feedback after every tactic, instead of its own predictions, makes inference take about three times as long on our 64-core machine and improves miniF2F pass@32 by only about one percentage point over WM8. In this run, time spent in Lean dominates inference. Each sample waits for Lean after every tactic. WM8's predicted feedback is already accurate where it matters. When WM8 flags an attempt as failing, Lean indeed rejects it in 98.5% of cases on miniF2F and in 100% on ProofNet (Table 14), and on valid programs its predicted states match Lean's almost exactly (block F1 of 0.97 and 0.95, Table 13). We believe this is why actual feedback adds so little. Informal reasoning gives WM8 a larger gain than actual Lean-compiled feedback does: it raises WM8's miniF2F pass@32 from 75.61% to 77.05% (+1.44 percentage points, Table 3), against about one percentage point from actual feedback.

(ii) Feedback masked from the loss. A model trained with the feedback tokens masked from the loss is never trained to write feedback. At inference, it would therefore need Lean's actual feedback after every tactic, the costly setting of (i). There, actual feedback gives even WM8 only about one more percentage point, for about three times the inference time. This is why we did not train this variant.

AuthorsOpenReview ↗

Post 6: Response to Reviewer Z8NE (2/2)

Q3: Clarification of the Section 4.1 baseline

Reviewer comment/question. Does the Section 4.1 baseline include feedback in its context while excluding feedback tokens from the optimized logits, or is it trained only on proof actions/tactics? In the first case, does it use real Lean feedback at inference time? In the second case, is it trained on roughly 3.5 times fewer tokens, as suggested by Table 1?

Author response. S8 is trained only on proof-action targets, so feedback tokens never appear in its targets. For direct examples, its prompt contains only the problem and its target is the clean proof. For repair examples, its prompt contains the failed attempt together with Lean's real feedback, including the error message, and its target is only the corrected proof. Lean's feedback therefore appears in S8's context only in supervised repair examples, and it is never optimized through the logits.

At inference, S8 does not receive Lean feedback. It writes its proof without intermediate Lean calls, and only the final candidate is checked. WM8 instead generates its own predicted feedback and conditions later tactics and repairs on it.

Yes, WM8's targets are about 3.3 times as long as S8's on average (Table 1), because they add Lean's feedback to the same proofs. Both provers learn from the same proofs and repair histories with the same SFT settings, so the extra target tokens are exactly the feedback supervision that our method adds. Our compact format already keeps them short: printing the full proof state at every step would make the targets 2.2 times as long (4,793 against 2,140 tokens on average, Table 1). Most of these extra tokens are also easy ones. Our speculative-decoding measurements show that feedback tokens are more predictable than tactic tokens: n-gram lookup, which only copies earlier text, predicts them better, because they restate hypotheses and goals that already appear in the context. Generation cost and learning signal are not directly comparable, but this repetition suggests that the 3.3 times difference in target length overstates how much new information WM8's targets add over S8's.

Q4: Fairness of the comparison in Section 5.4

Reviewer comment/question. The discussion in Section 5.4 appears to focus on the two baselines' weaker results. Are there other baseline models that would provide a fairer or more informative comparison?

Author response. We thank the reviewer and understand the concern. We compare with Goedel-Prover-V2-8B and DeepSeek-Prover-V2-7B because they are among the strongest and most widely used open provers of this size, not because they perform poorly on our benchmarks. Both learn informal reasoning from much larger teacher models (Section 5.4). WM8 uses no teacher reasoning, yet it matches DeepSeek-Prover-V2-7B on miniF2F (75.61%) and comes within about one point of Goedel-Prover-V2-8B on ProofNet (16.44% against 17.52%, Table 3). This is why we highlight these results.

Q5: Error-prediction targets for WM8-NR

Reviewer comment/question. In Section 4.2, does WM8-NR see error-prediction targets during GRPO, or only through the auxiliary losses used alongside GRPO?

Author response. Only through the auxiliary feedback loss used alongside GRPO. In initial SFT and self-distillation, WM8-NR sees no error targets, because repair examples are excluded from both. During GRPO, the auxiliary feedback loss trains it on Lean's annotations of its own sampled candidates, which for a failing candidate end with Lean's error. Its supervised replay also excludes repair examples, so replay adds no error targets. We will state this explicitly in Section 4.2.

Interestingly, this is enough for self-repair to emerge. After GRPO, WM8-NR retries in 7.9% of its miniF2F responses and 5.4% of its ProofNet responses, although it never saw a repair example and its only error targets come from Lean's feedback (Table 15).

We thank the reviewer again for these thoughtful questions. We hope our answers address the reviewer's concerns, and we are happy to answer any further questions.

Thanks for the detailed answers, which clarified most of my doubts. In particular I appreciated the answers to Q1 and Q2.

Regarding my two points about (i) inference-time and (ii) training-time benefits, which the baselines in Q2 (i) and Q2 (ii) would isolate. The inference-time benefits demonstrated in Q2(i) seem a strong point which I recommend the authors to make more explicitly in the motivation of this work. I see the author's argument about the baseline Q2(ii) suffering the same inference time limitations observed in Q2(i) and, hence, not being a strong competitor. Still, the point of training Q2(ii) is isolating the training-time befits in point (ii), not providing a stronger model.

That being said, the (i) inference-time argument alone seem a valid argument in favor of this method so I recommend strengthening the presentation around it. Accordingly, I will raise my score to a weak acceptance. Still, I believe that further claiming the (ii) training-time benefits require isolating them with the baseline Q2(ii), which I still recommend the authors to include in the final version of the manuscript

Reviewer_ZxS3OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces Lean World Models - the idea that the Lean-trained models can not only be trained to output Lean tactic traces but also predict the intermediate Lean goal and error states. A Qwen3 4B and 8B models are post-trained with this objective and evaluated on miniF2F and ProofNet, showing improved performance over the baseline of models trained without the additional Lean state objective. The manuscript then argues that the proposed techniques are complementary and additive to other post-training techniques, such as using informal reasoning.

Critical strengths
  1. The paper, to my knowledge, proposes a novel data augmentation methodology for training Lean theorem proving models to predict and train on Lean goal and error states. While DeepSeek-Prover-V1.5 already used goal state prediction at inference, this paper, to my knowledge, is the first to extend it to error prediction and in-response repair, and keeping it as a supervised loss during RL post-training. In their methodology, supervision signal requires only one Lean execution per proof, potentially reducing idle GPU-time during repeated Lean executions. Trained on the same proofs, the world-model variants of Qwen3 4B and 8B outperform a baseline without the feedback objective on both miniF2F and ProofNet.
Critical weaknesses
  1. Results do not isolate the methodolical contribution. The manuscript does not report the base model performance, CPT-only or pure SFT (without self-distillation) results on both benchmarks to isolate the gains. As it stands, the gains on the benchmarks are relatively minimal (WM vs S8) and confidence intervals would be very welcome to address this. Lastly, expectedly, the WM models output about 6x more tokens per response than S8, with minimal improvement in performance, hence it would be beneficial to provide at a token-matched performance plot on the benchmark to evaluate the efficiency of each method.

  2. Methodological basis is not convincing. It is unclear why self-predicting errors is helpful and the evidence that it does it weak as it stands. If the model can predict that a tactic result in an error, why does it generate that tactic in the first place? This would require a gap between generating and verifying Lean tactics, which the paper neither argues for nor measures. WM8's F1 score on invalid programs is reported as 0.29 on ProofNet showing inability to predict error states. Therefore, it is unclear if simply repeated attempts, or error state prediction is contributing to performance. A comparison between self-repair and a new, independent attempt, matched for its token budget would clarify if this contribution adds value.

Questions for authors

Please address the critical weaknesses.

2 comments
AuthorsOpenReview ↗

Post 3: Response to Reviewer ZxS3 (1/2)

Reviewer comment/question. The results do not isolate the methodological contribution. The manuscript does not report base-model, continued-pretraining-only, or pure-SFT results on both benchmarks. The gains of WM8 over S8 are relatively small, and confidence intervals would help assess them. Moreover, WM8 produces roughly six times as many tokens per response as S8. Please provide a token-matched performance plot or other budget-matched comparison that evaluates the efficiency of the methods.

Author response. We thank the reviewer for this suggestion, which makes the paper more complete. We have now evaluated the base and continued-pretraining (CPT) checkpoints (Table R5) and will include them in the revision.

Table R5. Pass@32 (%) of the base model, the CPT checkpoint and WM8 after GRPO (for WM8, the mean of three evaluations).

ModelminiF2FProofNet
Qwen3-8B-Base25.42.2
Qwen3-8B after CPT31.87.5
WM8 after GRPO75.5416.35

We also appreciate the question about the SFT checkpoint. The "Before GRPO" rows of Table 3 are measured right after initial SFT, before self-distillation, so they are the pure-SFT results. Self-distillation is a short step that prepares GRPO: one epoch on the model's own verified solutions, sampled for 29,480 problems, compared with 2.76 million examples for initial SFT, at a lower learning rate (2×10⁻⁶ instead of 10⁻⁵, Tables 5 and 7). We add it to make GRPO training cleaner and more stable (Appendix A.3), and we measured that it does not change benchmark performance. We will state clearly in the revision which stages belong to SFT and which to GRPO.

For uncertainty, over three independent evaluations, WM8 after GRPO reaches 75.54 ± 0.43 on miniF2F and 16.35 ± 0.41 on ProofNet, against 70.36 ± 0.66 and 13.93 ± 0.68 for S8 after GRPO. Its leads of 5.2 and 2.4 points are far larger than these standard deviations.

For efficiency, we compare at the same GPU budget rather than the same number of tokens, because tokens differ in cost. With the n-gram speculative decoding that we use for every prover, WM8's feedback tokens are cheaper than its tactic tokens: they restate hypotheses and goals that already appear in the context, which n-gram lookup predicts well. We measure the GPU time that WM8 needs for its 32 samples per problem and give S8 the same average GPU time per problem. S8's samples are much shorter, so in this time it can generate 2.3 to 6 times as many samples (Table R6). WM8 is still ahead of both S8 checkpoints on both benchmarks, and on all 859 problems together, both of its leads have 95% confidence intervals above zero. The note at the end of our general response explains how we measure GPU time and why we use speculative decoding.

Table R6. WM8 after GRPO, with 32 samples per problem, against S8 at the same GPU budget. S8 samples: samples per problem, with the multiple of WM8's 32 in parentheses. WM8 − S8: pass-rate difference in points, with a paired bootstrap 95% confidence interval over problems. miniF2F + ProofNet: all 859 problems (488 + 371) as one set.

S8 checkpointminiF2F: S8 samplesminiF2F: WM8 − S8 [95% CI]ProofNet: S8 samplesProofNet: WM8 − S8 [95% CI]miniF2F + ProofNet: WM8 − S8 [95% CI]
After GRPO193 (6.0×)+2.7 [+0.8, +4.7]133 (4.2×)+1.6 [−0.5, +3.8]+2.2 [+0.9, +3.6]
Before GRPO73 (2.3×)+0.6 [−1.1, +2.4]122 (3.8×)+2.5 [+0.6, +4.6]+1.4 [+0.1, +2.8]
AuthorsOpenReview ↗

Post 4: Response to Reviewer ZxS3 (2/2)

Q2: Why do self-predicted errors help, and how accurate are they?

Reviewer comment/question. The methodological basis for self-predicting errors is not yet convincing. If the model can predict that a tactic will cause an error, why does it generate that tactic in the first place? This appears to require a measurable gap between the model's tactic generation and tactic verification abilities. Please explain and, if possible, measure this gap, especially in light of WM8's reported F1 of 0.29 on invalid ProofNet programs.

Author response. Generating a tactic and recognizing afterwards that it fails are not in conflict. A tactic has nonzero probability and can be sampled, and once it is written, the model can recognize that it fails. The predicted error serves recovery, not prevention.

This mirrors how people write proofs. A tactic can look reasonable when it is chosen and not fail right away, yet lead the proof into a dead end. The prover may only notice this a few steps later, when it can no longer continue without an error. With predicted feedback, WM8 flags the error, keeps the failed attempt and its diagnosis in context, and can start a new attempt in the same response that avoids the same mistake.

What repair needs is to recognize that an attempt fails, and WM8 does this well. On ProofNet, WM8 after GRPO flags 58.65% of its failing attempts with 100% precision (F1 73.93%, Table 14): it never predicts an error for an attempt that Lean accepts. The 0.29 measures something much stricter: exact-match F1 over the feedback blocks of failing programs, where a predicted state or error block counts only if it matches Lean's text exactly, up to whitespace. Reproducing Lean's exact error text is hard, since error messages are diverse and rare in SFT data (Appendix E.4), and ProofNet is much harder than miniF2F. On valid ProofNet programs, WM8's predicted feedback is almost exact (block F1 0.95, Table 13). We will make this distinction clearer in the revision.

Q3: Is repair better than spending the same compute on new attempts?

Reviewer comment/question. It is unclear whether the observed improvement comes from error-state prediction or simply from making repeated attempts. Please compare self-repair with a new independent attempt under the same token budget.

Author response. WM8-NR lets us answer this directly. It follows the same training principles as WM8, predicting feedback and using the same auxiliary losses, but it sees no repair examples in initial SFT, self-distillation or replay (Section 4.2). Retries do emerge on their own during GRPO, but they are much rarer than WM8's (Table 15).

We therefore give both models the same average GPU time per problem, measured as described in the note at the end of our general response. WM8 spends part of this budget on self-repair within its responses, while WM8-NR, whose responses are shorter, uses it for more independent samples: 41 per problem on both benchmarks, against WM8's 32. WM8 still wins (Table R7).

Table R7. WM8 after GRPO (32 samples per problem) against WM8-NR after GRPO at the same GPU budget (41 samples per problem). WM8 − WM8-NR in pass-rate points, with paired bootstrap 95% confidence intervals over problems.

miniF2F [95% CI]ProofNet [95% CI]miniF2F + ProofNet (859 problems) [95% CI]
WM8 − WM8-NR+1.8 [+0.2, +3.5]+1.1 [−0.2, +3.0]+1.5 [+0.3, +2.7]

WM8 leads on both benchmarks, significantly on miniF2F and on all 859 problems together. At the same GPU budget, WM8's 32 samples solve more problems than WM8-NR's 41, so compute spent on WM8's trained self-repair pays off more than extra samples.

We thank the reviewer again for these helpful suggestions. We hope our answers address the reviewer's concerns, and we are happy to answer any further questions.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Post 1: General Response to All Reviewers (1/2)

We thank the reviewers for their careful reading and constructive suggestions. We ran the requested experiments and report them here: repeated evaluations, a comparison at the same GPU budget, and an S8 control with supervised replay. A final note explains how we measure GPU time, why tokens differ in cost, and what verification costs.

1. Repeated evaluations. We evaluate each of our four provers before and after GRPO, without informal reasoning, three times independently, and report means and standard deviations (Table R1).

Table R1. Repeated evaluations: pass@32 (%) and mean generated tokens per response, mean ± standard deviation over three independent evaluations.

ModelminiF2F Pass ↑miniF2F Tok.ProofNet Pass ↑ProofNet Tok.
Before GRPO
WM8 (world model)74.66 ± 0.516281 ± 81.914.74 ± 1.027359 ± 242.5
WM8-NR (no repair)73.70 ± 0.515266 ± 59.513.12 ± 0.416776 ± 70.2
WM4 (world model, 4B)72.41 ± 0.836067 ± 8.914.02 ± 0.276034 ± 34.5
S8 (standard)72.00 ± 0.431995 ± 37.113.39 ± 0.311420 ± 36.3
After GRPO
WM8 (world model)75.54 ± 0.436911 ± 57.716.35 ± 0.418692 ± 13.3
WM8-NR (no repair)73.91 ± 0.435013 ± 53.016.17 ± 0.546035 ± 101.5
WM4 (world model, 4B)74.66 ± 0.475847 ± 43.316.26 ± 0.415776 ± 33.3
S8 (standard)70.36 ± 0.661080 ± 9.113.93 ± 0.68923 ± 34.7

2. Same GPU budget. WM8 after GRPO gets 32 samples per problem. We measure the GPU time these 32 samples take and give each S8 checkpoint the same average GPU time per problem. S8 writes much shorter responses, so in that time it can generate 2.3 to 6 times as many samples per problem as WM8 (Table R2). All three provers run on the same hardware with the same speculative decoding (see the note below).

Table R2. WM8 after GRPO, with 32 samples per problem, against S8 at the same GPU budget. S8 samples: samples per problem, with the multiple of WM8's 32 in parentheses. WM8 − S8: pass-rate difference in points, with a paired bootstrap 95% confidence interval over problems. miniF2F + ProofNet: all 859 problems (488 + 371) as one set.

S8 checkpointminiF2F: S8 samplesminiF2F: WM8 − S8 [95% CI]ProofNet: S8 samplesProofNet: WM8 − S8 [95% CI]miniF2F + ProofNet: WM8 − S8 [95% CI]
After GRPO193 (6.0×)+2.7 [+0.8, +4.7]133 (4.2×)+1.6 [−0.5, +3.8]+2.2 [+0.9, +3.6]
Before GRPO73 (2.3×)+0.6 [−1.1, +2.4]122 (3.8×)+2.5 [+0.6, +4.6]+1.4 [+0.1, +2.8]

WM8 is ahead of both S8 checkpoints on both benchmarks, and on all 859 problems together both of its leads have 95% confidence intervals above zero. Pass rates use the standard unbiased pass@k estimator.

This comparison is lenient toward S8. The budget counts only GPU generation time, but every candidate must also be checked by Lean, and at this budget S8 has 41 to 161 more candidates per problem than WM8 (2.3 to 6 times as many). We do not charge S8 for checking them, although on miniF2F checking adds 17–96% to the cost of an S8 sample and only 5–12% to the cost of a WM8 sample (see the note below).

We have not tried to compress the feedback format further. A more compact format could make each WM8 sample cheaper and improve its results at the same GPU budget, a promising direction for future work.

3. S8 with supervised replay. As requested, we add the supervised replay of WM8's GRPO stage to S8's GRPO stage (Table R3).

Table R3. S8 with and without supervised replay, and WM8, all after GRPO: pass@32 (%), mean ± standard deviation over three evaluations.

Model (after GRPO)miniF2FProofNet
S8 (standard)70.36 ± 0.6613.93 ± 0.68
S8 (standard) + SFT replay71.45 ± 0.3113.93 ± 0.41
WM8 (world model)75.54 ± 0.4316.35 ± 0.41

Replay barely changes S8's pass@32 (+1.1 points on miniF2F, no change on ProofNet), but it makes S8's responses longer (1,310 instead of 1,080 tokens on miniF2F, 962 instead of 923 on ProofNet). WM8 stays 4.1 and 2.4 points ahead of S8 + replay. We will report this control in Section 5.2 of the revision. But this confirms our initial reasoning: we add SFT replay to keep the world-model prover's GRPO training stable, not to raise its benchmark scores. Some of our GRPO runs without replay start writing gibberish, and we have not seen this in runs set up like our main world-model runs, which use replay together with the auxiliary feedback loss (Appendix D.2).

AuthorsOpenReview ↗

Post 2: General Response to All Reviewers (2/2)

Note: GPU time and the cost of tokens.

Measuring GPU time. Each problem runs alone on one H100 with vLLM, as one request for 32 samples (temperature 0.7, top-p 0.95, up to 32k tokens) that are decoded together. A problem's GPU time is the wall-clock time from sending the request until its last sample finishes, including prefill, decoding and speculation overhead. It is measured directly, not estimated from token counts. Table R4 gives the measured cost per sample.

Table R4. Measured GPU time per sample (GPU-seconds) and mean generated tokens per sample, in the GPU-timing runs.

ProverGPU-s per sample, miniF2FGPU-s per sample, ProofNetTokens per sample, miniF2F
WM8 after GRPO1.482.066,055
S8 after GRPO0.250.501,107
S8 before GRPO0.650.541,985

Thirty-two WM8 samples cost an average of 47.4 GPU-s per problem on miniF2F and 66.0 on ProofNet. Dividing these budgets by each S8 checkpoint's cost per sample gives the S8 sample counts in Table R2 (computed from unrounded costs). For example, in 47.4 GPU-s, S8 before GRPO generates 47.4 / 0.65 ≈ 73 samples per miniF2F problem.

Why n-gram speculative decoding. We serve all three provers with n-gram speculative decoding. To deploy them as fast as possible, we compared it with a Qwen3-0.6B draft model that we trained on the same data, and n-gram lookup was clearly faster. It needs no draft model and no training. At each step, it finds the last few tokens earlier in the same sequence and proposes the tokens that followed them there as a draft. The 8B model checks the whole draft in one forward pass and keeps the longest prefix it would have generated itself. Rejection sampling keeps the output distribution unchanged. The draft length grows from 6 tokens while 17–32 samples are still running to 50 tokens when only 1–4 remain. Lean outputs suit this well, because proofs repeat hypothesis names and terms, and a repair attempt copies the previous attempt with small edits. N-gram speculation speeds up WM8 by 4.2× on miniF2F and 5.2× on ProofNet, S8 after GRPO by 3.1× and 3.4×, and S8 before GRPO by 4.7× and 7.7×.

Feedback tokens are cheaper than tactic tokens. Feedback blocks restate hypotheses and goals that already appear earlier in the context, so n-gram lookup predicts them well, and an accepted draft token costs no forward pass of its own. This is one reason why n-gram speculation speeds up WM8 so much. Measured GPU time captures this difference in cost, while a token count does not.

Verification cost. In our setup, each verifier worker runs Lean on one CPU core. Verifying one candidate takes about 8 core-seconds on average. With our 64 workers running in parallel, we complete about one candidate every 0.1–0.25 s. WM8's long outputs are not more expensive to verify, for two independent reasons: only the final Lean code block of a response is checked, and feedback blocks are stripped before verification. WM8's candidates are in fact cheaper to check: on miniF2F, about 7–9 core-seconds against 11–12 for the standard provers. At typical cloud prices, where an H100-hour costs about 50 to 100 times a CPU-core-hour, verification on miniF2F adds about 5–12% to the cost of a WM8 sample (1.48 GPU-s), but 17–37% to the cost of a sample of S8 before GRPO (0.65 GPU-s) and 44–96% to the cost of a sample of S8 after GRPO (0.25 GPU-s). If we matched the total generation and verification cost of 32 WM8 samples per miniF2F problem, both S8 checkpoints would get fewer samples than in Table R2.

We thank all reviewers again for their constructive feedback, which helps us improve the paper. We are happy to answer any further questions during the discussion period.

AuthorsOpenReview ↗

Additional result: budget including verification cost

We thank Reviewer erWN for suggesting this analysis, which makes our efficiency comparison more complete. Table R2 matches each prover's budget in GPU generation time only. Table R10 also charges for Lean verification: each sample costs its GPU-seconds plus its verification CPU-core-seconds divided by R, the number of CPU-core-hours that cost as much as one H100-hour. WM8's lead grows on miniF2F and on all 859 problems, and stays the same on ProofNet. At the same generation time, S8 has 2.3 to 6 times as many candidates to verify on miniF2F as WM8, and its candidates take longer to verify on average (11–12 against 7–9 CPU-core-seconds). Charging for verification therefore cuts S8's sample counts noticeably. Its pass rate is nearly flat beyond about 100 samples (72.4% at 96 samples against 72.7% at 193 on miniF2F), so the samples it loses matter little.

Table R10. WM8 after GRPO (32 samples per problem) against S8 at the same total cost of generation and verification. S8 samples: samples per problem. WM8 − S8: pass-rate difference in points, with paired bootstrap 95% CIs over problems. Pooled: all 859 problems as one set.

S8 checkpointRminiF2F: S8 samplesminiF2F: WM8 − S8 [95% CI]ProofNet: S8 samplesProofNet: WM8 − S8 [95% CI]Pooled: WM8 − S8 [95% CI]
After GRPO100139+2.8 [+1.0, +4.8]125+1.6 [−0.3, +3.8]+2.3 [+0.9, +3.8]
After GRPO50112+2.9 [+1.0, +4.9]118+1.6 [−0.3, +3.8]+2.4 [+0.9, +3.8]
Before GRPO10065+0.8 [−0.9, +2.5]117+2.5 [+0.6, +4.6]+1.5 [+0.2, +2.9]
Before GRPO5060+0.9 [−0.8, +2.7]113+2.5 [+0.6, +4.6]+1.6 [+0.3, +3.0]

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.