- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
-
This paper proposes a three-stage Draft–Translation–Correction framework for the step-level formal reasoning task introduced by ProofGap.
-
The authors construct DSL training data and train the models using SFT and RL. They also propose Paired-view PKPO to optimize the joint success rate of draft-conditioned and no-draft generation.
-
The final system achieves a 35.7% success rate on 4,000 proof gaps, outperforming the Codex baseline (25%).
Critical strengths
- The author conducted many experiments, with ablation studies covering various components, from which we see the engineering contributions.
- The final performance seems not bad, outperforming the Codex baseline.
Critical weaknesses
-
Limited overall novelty. The Draft–Translation–Correction pipeline is a straightforward design similar to existing frameworks such as Draft, Sketch, and Prove (DSP). SFT, RL, and verifier-based correction are also well-established techniques. Overall, this work seems more like an engineering effort tailored to the ProofGap task, without much clear methodological novelty.
-
Paired-view PKPO has some novelty, but lacks a direct comparison with standard PKPO. Therefore, it is unclear whether the performance improvement comes from the proposed paired-view design or simply from RL training.
-
The presentation could be improved. The main methodological contributions are not clearly highlighted, and some technical details are introduced without sufficient intuition. A clearer explanation of the key ideas and a more organized presentation of the experiments would improve readability.
Questions for authors
-
Could the authors provide a direct comparison between PV-PKPO and standard PKPO? How much of the improvement can be attributed to the paired-view credit assignment rather than RL itself?
-
What are the practical advantages of the proposed gap-specific post-training compared with directly applying theorem-level training recipes? Could the authors provide a comparison under the same training budget?
-
Could the proposed methods and training strategies generalize to other formal languages, such as Lean?
4 comments
Q1: Comparison with standard PKPO
Thank you for your question. Regarding the comparison between PV-PKPO and standard PKPO, we are organizing supplementary experiments and will provide the results as soon as the experiments are completed.
Q2: Practical advantages and comparison under the same training budget
Here, we distinguish two ways of connecting step-level and theorem-level proving.
- Building a complete formal proof from the steps of a natural-language solution. This requires an upstream formalization process and composition of the resulting local derivations. It must preserve the solution's semantic steps, whereas an end-to-end theorem prover may generate any valid proof of the final statement. A fair comparison therefore needs matched inputs and output requirements. Our work studies the supplied step-level obligations; constructing a complete, aligned proof is a further research direction.
- Applying a theorem-trained model to step-level goals. This directly concerns the present task, and the Lean comparison provides a reference for that approach. Our scheme concentrates adaptation on solving the supplied gaps in the target DSL: the drafter remains frozen, the writer learns to use and revise its guidance, and the corrector learns from checker feedback.
The practical advantage is a targeted adaptation route that avoids a separate drafter-training stage and teaches the writer to work with a smaller drafting model. Table 8 reports Model 2/3 training over 9,654 source gap identities, with approximately 306.88 NPU-hours in the recorded training loops. For context, the published DeepSeek-Prover-V2 recipe includes expert iteration and recursive proof synthesis, SFT and RL of a 671B model, and distillation into the smaller prover. Its RL stage samples 32 candidates for each of 256 problems per iteration (Ren et al., 2025, Section 2.3). This illustrates the broader training process involved in developing such a theorem prover.
We have not retrained a complete theorem-level pipeline under the same budget. Our experiments evaluate gap-specific adaptation and the application of released theorem provers to local goals. The recorded adaptation resources describe the training we perform.
Q3: Transfer to Lean and other formal languages
The main advantage motivating our design is that it does not require the base model to have learned the target formal language during pretraining. The natural-language drafter can retain its existing abilities, while post-training teaches formal execution and correction through accepted proofs and checker feedback. We believe this separation is useful when adapting models to newly introduced proof languages. This is particularly relevant when new formal proof languages have not accumulated sufficient data for pretraining.
For Lean, it is difficult to establish a comparable condition in which an available pretrained model can reliably be shown to have had no exposure to Lean during pretraining. Starting from an existing Lean prover already supplies that knowledge. Such an experiment would test transfer of the training method to Lean, but would not directly test the advantage we study: acquiring an unfamiliar formal language through gap-specific adaptation.
We therefore expect the design to be useful for other newly introduced formal languages: the natural-language drafting scheme can be reused, while only the language-specific supervision and checker interface need to be adapted. The present experiments establish this approach in ProofGap's DSL and document the resources needed for that adaptation.
Additional response to W2 and Q1: Controlled comparison with standard PKPO
Thank you again for requesting a direct comparison with standard PKPO. We have completed a controlled comparison using five independent training runs per method and the full 4,000-gap test pool.
Controlled settings. Both methods use the same SFT initialization, training examples, draft-conditioned and no-draft views, allocated rollout and optimizer-update budgets, optimizer settings, filtering, and auxiliary replay. Standard PKPO computes credit within each view; PV-PKPO applies the proposed cross-view credit weighting. This coefficient is the only difference between the two training configurations. Both methods are evaluated with the same 5×2 writer allocation—four draft-conditioned views and one no-draft view, with two candidates per view—and Model 3 is disabled.
Repeated runs and checkpoint selection. For each algorithm, we conduct five independent training runs. Within each run, we select the checkpoint with the best score on an independent validation set, then evaluate the selected checkpoint on the full 4,000-gap test pool. Thus, ten selected checkpoints are tested in total. We report the interquartile mean (IQM) of the five test-coverage scores for each method, with 95% confidence intervals (CIs). The CI for the IQM difference is computed with corresponding training seeds paired across methods.
| Metric | PKPO IQM [95% CI] | PV-PKPO IQM [95% CI] | PV-PKPO − PKPO [95% CI] |
|---|---|---|---|
| Gap coverage on all 4,000 test gaps | 28.090% [27.838%, 28.380%] | 30.188% [30.083%, 30.435%] | +2.098 pp [+1.845, +2.300] |
PV-PKPO improves IQM test coverage by 2.098 percentage points, with the reported 95% CI for the improvement entirely above zero. Both methods receive RL training, and the controlled difference is the cross-view credit coefficient. This provides direct evidence for the additional benefit of the proposed paired-view credit assignment over standard PKPO.
Thank you to the authors for the detailed responses. The additional comparison with standard PKPO provides evidence for the effectiveness of the proposed PV-PKPO method. However, overall, I still find the methodological novelty and generalizable insights of this work somewhat limited for ICLR standards. Therefore, I will maintain my original rating.
Thank you for your comments on novelty, experimental attribution, and presentation. We would first like to clarify the learning question of this paper: how can a smaller pretrained model learn to solve natural-language-aligned step-level proof gaps in an unfamiliar formal language, while reusing its existing natural-language reasoning ability? Our contributions include a gap-specific post-training scheme, a process for constructing supervision, and a system that uses a frozen natural-language drafting model and connects translation with feedback-based correction. Within this scheme, PV-PKPO introduces credit assignment for the joint coverage of draft-conditioned and no-draft generation.
W1: Methodological contribution within the staged architecture
We agree that staged proof construction, SFT, RL, and verifier-based correction have important precedents. Draft, Sketch, and Prove (Jiang et al., 2023) demonstrates how informal proofs can guide formal sketches and automated proving. Our methodological contribution concerns the supervision and optimization used to learn reliable local proof construction.
Specifically, our contributions are:
These supervision and optimization designs specify what the formalization models learn within the staged architecture, building on the established components acknowledged above.
W2: Comparison with standard PKPO
Thank you for pointing this out. We are organizing supplementary experiments to address this issue and will provide the results as soon as the experiments are completed.
W3: Clearer intuition and experimental organization
Thank you for this feedback. We clarify our method contributions and the questions addressed by our experiments below.
Method contributions. The learning design has three main elements:
The intuition behind PV-PKPO is that correctness determines whether a candidate is accepted, while the other view's estimated failure probability determines how much a success contributes to joint coverage. This distinction explains why the cross-view factor rewards complementary solutions.
Experimental organization. The existing experiments examine the following aspects of the method: