acceptodds
Under review as a conference paper at ICLR 2027

Learning to Solve Step-Level Proof Gaps via Post-Training

Abstract

ProofGap introduces a new step-level formal reasoning task: solving local proof obligations aligned with semantic steps of natural-language solutions. By separating local proof construction from end-to-end theorem proving, this task provides finer-grained assessment of formal reasoning and motivates dedicated post-training research. We develop gap-specific post-training, aiming to solve these obligations more cost-effectively than transferring theorem-level training recipes to the same task. We also study how post-training can teach models the Domain Specific Language (DSL) introduced with ProofGap, a formal proof language unseen by the base models during pretraining. Our method uses a three-role architecture: natural-language drafting for formal proofs, DSL translation that can revise unreliable drafts, and model correction using verifier diagnostics. The drafting model remains frozen, while the translation and correction models receive task-specific post-training. These roles connect local mathematical intent to executable evidence and target efficient learning of the new language. On 4,000 gaps sampled from the ProofGap benchmark, none of which are solved by direct autosolve, our full system achieves a gap-level success rate of 35.70%, compared with 25.0% for the Codex baseline under the same theorem library and verifier, an improvement of 10.70 percentage points.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
formal reasoning, step-level proof gaps, post-training, large language models, domain-specific languages

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_yGhFOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary
  1. This paper proposes a three-stage Draft–Translation–Correction framework for the step-level formal reasoning task introduced by ProofGap.

  2. The authors construct DSL training data and train the models using SFT and RL. They also propose Paired-view PKPO to optimize the joint success rate of draft-conditioned and no-draft generation.

  3. The final system achieves a 35.7% success rate on 4,000 proof gaps, outperforming the Codex baseline (25%).

Critical strengths
  1. The author conducted many experiments, with ablation studies covering various components, from which we see the engineering contributions.
  2. The final performance seems not bad, outperforming the Codex baseline.
Critical weaknesses
  1. Limited overall novelty. The Draft–Translation–Correction pipeline is a straightforward design similar to existing frameworks such as Draft, Sketch, and Prove (DSP). SFT, RL, and verifier-based correction are also well-established techniques. Overall, this work seems more like an engineering effort tailored to the ProofGap task, without much clear methodological novelty.

  2. Paired-view PKPO has some novelty, but lacks a direct comparison with standard PKPO. Therefore, it is unclear whether the performance improvement comes from the proposed paired-view design or simply from RL training.

  3. The presentation could be improved. The main methodological contributions are not clearly highlighted, and some technical details are introduced without sufficient intuition. A clearer explanation of the key ideas and a more organized presentation of the experiments would improve readability.

Questions for authors
  1. Could the authors provide a direct comparison between PV-PKPO and standard PKPO? How much of the improvement can be attributed to the paired-view credit assignment rather than RL itself?

  2. What are the practical advantages of the proposed gap-specific post-training compared with directly applying theorem-level training recipes? Could the authors provide a comparison under the same training budget?

  3. Could the proposed methods and training strategies generalize to other formal languages, such as Lean?

4 comments

Thank you for your comments on novelty, experimental attribution, and presentation. We would first like to clarify the learning question of this paper: how can a smaller pretrained model learn to solve natural-language-aligned step-level proof gaps in an unfamiliar formal language, while reusing its existing natural-language reasoning ability? Our contributions include a gap-specific post-training scheme, a process for constructing supervision, and a system that uses a frozen natural-language drafting model and connects translation with feedback-based correction. Within this scheme, PV-PKPO introduces credit assignment for the joint coverage of draft-conditioned and no-draft generation.

W1: Methodological contribution within the staged architecture

We agree that staged proof construction, SFT, RL, and verifier-based correction have important precedents. Draft, Sketch, and Prove (Jiang et al., 2023) demonstrates how informal proofs can guide formal sketches and automated proving. Our methodological contribution concerns the supervision and optimization used to learn reliable local proof construction.

Specifically, our contributions are:

  1. A gap-specific post-training scheme with a frozen drafter. We organize learning around supplied local proof obligations. The drafter reuses its natural-language reasoning ability without task-specific fine-tuning, while the writer and corrector receive separate supervision for constructing and repairing derivations in the unfamiliar DSL.
  2. Supervision for formalization under unreliable guidance. We construct draft-conditioned and no-draft examples, including erroneous model-generated drafts selected for repair supervision and paired with reasoning that revises the guidance and produces an accepted derivation. Correction examples additionally pair failed programs and checker diagnostics with accepted replacements. This teaches the formalization models to assess guidance and recover from errors.
  3. Paired-view credit assignment for joint coverage. PV-PKPO couples draft-conditioned and no-draft generation by the same writer. An accepted candidate receives greater marginal credit when the other view is unlikely to solve the gap. The optimization therefore accounts for complementarity between the routes, beyond the correctness of an individual candidate.

These supervision and optimization designs specify what the formalization models learn within the staged architecture, building on the established components acknowledged above.

W2: Comparison with standard PKPO

Thank you for pointing this out. We are organizing supplementary experiments to address this issue and will provide the results as soon as the experiments are completed.

W3: Clearer intuition and experimental organization

Thank you for this feedback. We clarify our method contributions and the questions addressed by our experiments below.

Method contributions. The learning design has three main elements:

  1. The learning organization: gap-specific adaptation of the writer and corrector while keeping the natural-language drafting model frozen.
  2. The supervision construction: accepted derivations, examples that teach revision of erroneous drafts, and diagnostic-conditioned correction targets.
  3. The optimization design: paired-view credit assignment for the joint coverage of draft-conditioned and no-draft generation.

The intuition behind PV-PKPO is that correctness determines whether a candidate is accepted, while the other view's estimated failure probability determines how much a success contributes to joint coverage. This distinction explains why the cross-view factor rewards complementary solutions.

Experimental organization. The existing experiments examine the following aspects of the method:

  1. Supervision construction and robustness: we compare training with and without additional synthetic gap–proof data, and separately with and without erroneous-draft supervision; we evaluate the latter with different drafting models. These experiments examine the role of the training data and the writer's dependence on draft quality.
  2. Writer optimization: we compare the SFT writer with the writer after PV-PKPO training, reporting gap coverage and candidate acceptance separately. This experiment examines the complete RL training configuration.
  3. Feedback-based correction: we compare correction using checker diagnostics with fresh generation and examine the correction configurations. These experiments assess the role of feedback in recovering failed candidates.
  4. System performance and resource use: we compare the assembled system with Codex in the same DSL environment, present the Lean prover results with their respective evaluation settings, and report coverage and token accounting by inference allocation.

Q1: Comparison with standard PKPO

Thank you for your question. Regarding the comparison between PV-PKPO and standard PKPO, we are organizing supplementary experiments and will provide the results as soon as the experiments are completed.

Q2: Practical advantages and comparison under the same training budget

Here, we distinguish two ways of connecting step-level and theorem-level proving.

  1. Building a complete formal proof from the steps of a natural-language solution. This requires an upstream formalization process and composition of the resulting local derivations. It must preserve the solution's semantic steps, whereas an end-to-end theorem prover may generate any valid proof of the final statement. A fair comparison therefore needs matched inputs and output requirements. Our work studies the supplied step-level obligations; constructing a complete, aligned proof is a further research direction.
  2. Applying a theorem-trained model to step-level goals. This directly concerns the present task, and the Lean comparison provides a reference for that approach. Our scheme concentrates adaptation on solving the supplied gaps in the target DSL: the drafter remains frozen, the writer learns to use and revise its guidance, and the corrector learns from checker feedback.

The practical advantage is a targeted adaptation route that avoids a separate drafter-training stage and teaches the writer to work with a smaller drafting model. Table 8 reports Model 2/3 training over 9,654 source gap identities, with approximately 306.88 NPU-hours in the recorded training loops. For context, the published DeepSeek-Prover-V2 recipe includes expert iteration and recursive proof synthesis, SFT and RL of a 671B model, and distillation into the smaller prover. Its RL stage samples 32 candidates for each of 256 problems per iteration (Ren et al., 2025, Section 2.3). This illustrates the broader training process involved in developing such a theorem prover.

We have not retrained a complete theorem-level pipeline under the same budget. Our experiments evaluate gap-specific adaptation and the application of released theorem provers to local goals. The recorded adaptation resources describe the training we perform.

Q3: Transfer to Lean and other formal languages

The main advantage motivating our design is that it does not require the base model to have learned the target formal language during pretraining. The natural-language drafter can retain its existing abilities, while post-training teaches formal execution and correction through accepted proofs and checker feedback. We believe this separation is useful when adapting models to newly introduced proof languages. This is particularly relevant when new formal proof languages have not accumulated sufficient data for pretraining.

For Lean, it is difficult to establish a comparable condition in which an available pretrained model can reliably be shown to have had no exposure to Lean during pretraining. Starting from an existing Lean prover already supplies that knowledge. Such an experiment would test transfer of the training method to Lean, but would not directly test the advantage we study: acquiring an unfamiliar formal language through gap-specific adaptation.

We therefore expect the design to be useful for other newly introduced formal languages: the natural-language drafting scheme can be reused, while only the language-specific supervision and checker interface need to be adapted. The present experiments establish this approach in ProofGap's DSL and document the resources needed for that adaptation.

Additional response to W2 and Q1: Controlled comparison with standard PKPO

Thank you again for requesting a direct comparison with standard PKPO. We have completed a controlled comparison using five independent training runs per method and the full 4,000-gap test pool.

Controlled settings. Both methods use the same SFT initialization, training examples, draft-conditioned and no-draft views, allocated rollout and optimizer-update budgets, optimizer settings, filtering, and auxiliary replay. Standard PKPO computes credit within each view; PV-PKPO applies the proposed cross-view credit weighting. This coefficient is the only difference between the two training configurations. Both methods are evaluated with the same 5×2 writer allocation—four draft-conditioned views and one no-draft view, with two candidates per view—and Model 3 is disabled.

Repeated runs and checkpoint selection. For each algorithm, we conduct five independent training runs. Within each run, we select the checkpoint with the best score on an independent validation set, then evaluate the selected checkpoint on the full 4,000-gap test pool. Thus, ten selected checkpoints are tested in total. We report the interquartile mean (IQM) of the five test-coverage scores for each method, with 95% confidence intervals (CIs). The CI for the IQM difference is computed with corresponding training seeds paired across methods.

MetricPKPO IQM [95% CI]PV-PKPO IQM [95% CI]PV-PKPO − PKPO [95% CI]
Gap coverage on all 4,000 test gaps28.090% [27.838%, 28.380%]30.188% [30.083%, 30.435%]+2.098 pp [+1.845, +2.300]

PV-PKPO improves IQM test coverage by 2.098 percentage points, with the reported 95% CI for the improvement entirely above zero. Both methods receive RL training, and the controlled difference is the cross-view credit coefficient. This provides direct evidence for the additional benefit of the proposed paired-view credit assignment over standard PKPO.

Thank you to the authors for the detailed responses. The additional comparison with standard PKPO provides evidence for the effectiveness of the proposed PV-PKPO method. However, overall, I still find the methodological novelty and generalizable insights of this work somewhat limited for ICLR standards. Therefore, I will maintain my original rating.

Reviewer_f1pQOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies a system for solving step-level proof from ProofGap. The proposed system uses a frozen pretrained model to produce natural-language drafts, a writer trained through sft and paired-view PKPO, and a separately supervised corrector that uses verifier feedback. It achieves 35.7% coverage vs 25% for Codex(gpt-5.6-sol-medium) in the same environment.

Critical strengths
  • Solving proof gaps is a meaningful task with potential downstream applications.
  • Pair-view PKPO is an interesting extension for the specific goals of the paper.
Critical weaknesses
  • The task appears tied to a specific benchmark and its language. Additionally, the paper comparison suggests that an off-the-shelf lean prover is competitive when the gaps are rewritten/expressed in Lean (although the comparison conditions differ.). As a result, one could "question" the "need" for the dedicated post-training recipe.

  • Key design choices are not sufficiently justified. The benefit of the 3 stages is not clearly established. There is no comparison against a single model trained on all the data. Is the split still required if a stronger model is used? What would happen if the authors start with a model that already is good at theorem proving and may require minimal adaptation?

  • Some ablations show limited gains. I recommend to discuss this more at length as it may lead to simplifications in the design choices.

  • The Codex comparison is unclear. The baseline setup is not sufficiently described in the paper. Can the authors provide the entire agent configuration including prompts ?

  • The applicability of the results beyond the benchmark remains open. The paper motivates the task through downstream applications of step level verification and supervision, but does not evaluate those applications. The real world utility thus also remains open.

Overall, my biggest concern about this paper is that is tied to a specific benchmark. Thus, the applicability of the results or the study beyond this particular benchmark remains untested/open. One could imagine the downstream applications as very important and the paper solving an important gap towards it. However, in the current presentation this is not evaluated. I recommend the authors a major rewrite focusing on the applicability of results beyond the benchmark and more so in terms of the downstream applications.

Questions for authors

see above

5 comments

Thank you for your review and for recognizing the value of solving proof gaps. We would first like to clarify our motivation. We study how to post-train smaller models to solve natural-language-aligned step-level proof gaps in a formal language that the base model has not learned during pretraining. The input is an already formalized local context and goal; the model must construct a derivation accepted by the designated checker. A step that appears elementary in a natural-language solution can require several formal inferences, making the construction of this local derivation a meaningful learning problem.

Our contributions include a post-training scheme organized around these local obligations, a process for constructing supervision to learn the proof language introduced by ProofGap, and a system that reuses a frozen natural-language drafter while training translation and feedback-based correction models. The learning design includes supervision for revising erroneous drafts and paired-view optimization of joint proof coverage. ProofGap provides the benchmark and language used to evaluate these contributions. Below, we explain how this motivation relates to the Lean comparison and potential downstream applications.

W1: Benchmark specificity and the value of dedicated post-training

Our method targets a setting in which the base model has not learned the target proof language during pretraining. We aim to reuse the model's natural-language reasoning and concentrate post-training on formal execution and correction so that the model can use this language. This is relevant to newly developed proof languages, which may lack both a large inherited proof corpus and a specialized prover pretrained for that language.

We believe the approach is useful beyond individual DSL commands: the same supervision structure, paired-view credit assignment, and use of checker feedback can be instantiated with another formal language's proof targets and checker. The prompt and operation mappings implement the language interface. The learning design does not require a different optimization algorithm for every DSL. This is why we see the architecture as a useful direction for adapting models to future formal languages.

W2: Three-stage design and a unified model

Thank you for this question. Staged approaches to mathematical proving have precedents such as Draft, Sketch, and Prove (Jiang et al., 2023). We adopt a staged design based on the inputs and learning objectives of each stage, and provide an additional unified-model comparison to examine this choice.

Among the three models in our final system, only Models 2 and 3 are trained. Model 3 handles multi-round refinement, while Model 2's task is writing in the formal language. These are fundamentally different tasks. Model 2 learns to construct DSL from the local goal and draft, whereas Model 3 learns to generate a corrected program from a failed program and checker feedback. We organize these two types of supervision separately to adapt the models to initial generation and feedback-based correction. The paper's comparison between correction and additional generation further examines the role of feedback-based correction. These differences form the basis for our organization of supervision and training by role.

For Models 1 and 2, it is worth noting that we do not train Model 1 in our final system. To examine whether a shared model can replace this division, we are pleased to report findings from an additional joint-SFT comparison of draft generation and DSL generation. We used existing data to train a single model combining Models 1 and 2, which first produces a natural-language draft and then continues by outputting the formal proof. Both configurations use the same base model, checkpoint-selection rule, validation set, and number of test-time attempts. The staged baseline in this comparison uses the SFT-only Model 2 writer, matching the task-specific post-training stage of the unified SFT model. Neither configuration uses RL or Model 3 correction. We obtained the following results in the standard DSL checking environment:

Test setUnified model (joint SFT)Frozen Model 1 + SFT-only Model 2 (4×4)
D₂ (1,000 gaps)657 / 1,000 = 65.7%720 / 1,000 = 72.0%
Full 4,000 gaps1,026 / 4,000 = 25.65%1,137 / 4,000 = 28.43%

In this joint-SFT comparison, the unified model's coverage is lower by 6.3 percentage points on D₂ and 2.775 percentage points on the full test set. This supports using a frozen drafter with a separate writer in the current experimental setting.

Jointly retraining Models 1, 2, and 3 would require reassessing the organization and amount of data, as well as the supervised training procedure. We consider this to entail substantial costs and to be inconsistent with the original aim of our study.

We focus on adapting smaller models to unfamiliar formal languages. Among the general-purpose models of comparable parameter size that we considered, Qwen3-14B offers strong mathematical reasoning and natural-language proof capabilities. We therefore select it as the base model and compare the unified and staged approaches using that same base. For specialized theorem provers such as DeepSeek-Prover-V2-7B, which is designed for Lean, the structural differences between Lean and the new proof DSL mean that Lean proving ability does not directly guarantee proficiency in the DSL. However, if time permits, we will also provide the relevant evaluation results later to increase confidence in this assessment.

W3: Ablations with limited gains and possible simplifications

Thank you for pointing this out. Could you specify which ablation experiments show limited gains and which aspects you would recommend discussing in greater depth?

W4: Codex agent configuration and prompts

We are preparing anonymous configuration materials describing this baseline. We will include the detailed configuration, including the prompts, in an updated supplementary package and upload it again.

W5: Downstream applications and the paper's motivation

We would like to distinguish the motivation for a step-level benchmark from the learning question of this paper.

Current general-purpose large language models can solve step-level problems reasonably well, enabling downstream applications such as checking solutions, selecting reasoning trajectories, and supervising intermediate reasoning. ProofGap therefore provides a benchmark to better evaluate models' ability to solve these problems.

Our direct motivation is to make smaller models more effective at producing checker-accepted proofs for the supplied local obligations. We reuse their natural-language capabilities, keep the drafter frozen, and learn execution and correction in the target proof language. The system reaches 35.70% coverage; its five-round configuration reaches 34.95% with 477,359 reported tokens per gap, compared with Codex's 25.00% and 491,750.

Completing an entire theorem-level solution requires reliable upstream formalization and coverage of the necessary steps. Our system's current success rate is also insufficient to support completing such full solutions. The experiments in this paper evaluate the targeted post-training problem that the benchmark makes measurable.

W6: Applicability beyond the benchmark

Thank you for highlighting the need for broader evaluation. A practical constraint is that we currently do not have a second ready-to-use benchmark providing the same combination of alignment with reliable textbook steps, supplied formal local obligations, and the native executable interface used in this study.

Related resources are informative but have different task constructions. FormalML (Yang et al., 2025) extracts gaps from existing Lean proofs. SAFE/FormalStep (Liu et al., 2025) formalizes steps in model-generated reasoning for verification. Its authors report in Appendix G, Table 5 that GPT-4o-mini judged 80.9% of its formalizations as having good semantic alignment and 18.6% as poor. These are LLM-judge assessments of alignment. Our setting requires reliable source steps and their faithful formalization, so adopting this resource would require further quality validation. We have therefore not used it as an additional evaluation in this study.

We also considered StepProof: Step-by-step Verification of Natural Language Mathematical Proofs (ICLR 2025 submission), which studies sentence-level autoformalization. The data construction in this work provides no assurance of either appropriate sentence segmentation or semantic consistency. Our evaluation requires reliable step alignment, clearly specified formal local obligations, and checking rules. We have therefore not adopted it as an additional benchmark.

Our contribution is to study how to train models for this distinct step-level learning problem. The erroneous-draft supervision ablation and the original-PKPO/PV-PKPO control examine specific learning choices within it. We believe these findings themselves can provide useful knowledge while further task-matched benchmarks and downstream systems are being developed.

As clarified above, the present study evaluates gap-specific post-training. The wider applications motivate the task and provide directions for subsequent work.

Reviewer_UnfHOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper evaluates post-training language models for addressing step-level formal proof gaps on the ProofGap benchmark. In contrast to end-to-end theorem proving, this task requires producing executable derivations for single mathematical obligations given already formalized assumptions and goals.

The authors propose a three-step architecture consisting of a frozen natural-language proof drafter, a DSL proof writer trained through supervised fine-tuning and reinforcement learning, and a verifier-guided correction model. The writer is trained to generate proofs directly without a draft, execute valid drafts, and revise erroneous drafts. The proposed paired-view pass@k policy optimization (PV-PKPO) promotes complementary success between draft-conditioned and no-draft proof generation. The corrector is fine-tuned independently on failed proofs paired with verifier diagnostics and valid replacements.

On 4,000 non-autosolve ProofGap instances, the entire system achieves 35.70% coverage, compared to 25.00% for the Codex baseline using the same DSL checker and theorem library. Component studies report improvement from erroneous-draft supervision, PV-PKPO training, and iterative correction.

Critical strengths
  1. Effective task-specific post-training with verifier-backed evaluation: The paper shows that relatively small amounts of task-specific supervision can train models to produce executable proofs in a DSL absent from their original training setup. The large improvement over the Codex baseline on the same ProofGap environment (35.70% versus 25.00%) gives meaningful evidence that the overall approach works. The use of a formal verifier also gives the evaluation a stronger correctness criterion than natural-language judgments.

  2. Convincing evidence for erroneous-draft supervision and verifier-guided correction: The ablations suggest that it is important to train models on realistic drafting errors rather than simply on natural-language plans. For Qwen3-14B drafts, removal of erroneous-draft supervision leads to a drop of coverage from 68.3% to 37.2% under the reported 4×2 evaluation protocol (Table 3). Furthermore, starting from the same initial failures, the correction model with dynamic retrieval recovers 67 gaps, compared with 47 for fresh writer sampling, while generating fewer proof programs (6,898 versus 7,572; Table 4). These findings give concrete support for two key design decisions apart from the final system-level result.

Critical weaknesses
  1. Inadequate experimental isolation of PV-PKPO's contribution: While the paired view objective is an interesting methodological contribution, there is insufficient experimental validation. The main RL comparison evaluates SFT against the full PV-PKPO training configuration (Table 2), which includes a number of other implementation choices over the proposed cross-view weighting. In particular, the paper has no controlled comparison against standard PKPO with equivalent training data, rollout budgets, optimizer settings, and auxiliary replay. The additional GRPO result in Appendix A.3 has incomplete configuration information, and does not address this concern. Since the proposed objective extends beyond existing pass@k policy optimization by weighting success according to another view's estimated failure probability, isolating the benefit of this methodological extension is vital. The current evidence establishes that the overall RL configuration helps, but does not establish how much of this improvement is due specifically to the new objective.

  2. Evaluation design undermines generalization and efficiency claims: The component experiments use a 1,000-gap subset intentionally enriched for easier problems, in which historical Codex coverage is 58%, compared with 25% on the original 4,000-gap pool (Section 4.1 and Appendix A.3). This same subset is used for component development and evaluation, and raises questions about the generalizability of the reported ablation gains. The efficiency comparison requires qualification as well. While reduced writer budgets achieve favorable reported token-to-coverage trade-offs, these results are achieved through retrospective subsampling. The maximum-performing configuration requires roughly 608k reported tokens per gap, compared with 492k for Codex (Table 5). Moreover, differences in model tokenization, input caching, retrieval, verifier execution, and training or data-generation costs, complicate conclusions about overall computational efficiency. Finally, the comparison against Lean provers uses different formal languages, verifiers, and inference budgets. This provides useful context, but should not be interpreted as a controlled comparison of theorem-proving capabilities.

Questions for authors
  1. Is the improvement specifically attributable to paired-view credit assignment?

Could the authors compare PV-PKPO against standard PKPO, ideally both independently per view and under a shared sampling portfolio, while controlling for initialization, training examples, rollout budgets, optimizer settings, and auxiliary replay? This would directly inform my assessment of the methodological contribution.

  1. Do the component-level improvements generalize beyond the difficulty-enriched development subset?

Could the authors provide corresponding PV-PKPO and erroneous-draft supervision ablations on the full 4,000-gap evaluation pool or an independently held-out subset? Confidence intervals or paired significance estimates would also assist in establishing the robustness of the reported improvements.

  1. How does the approach compare under matched end-to-end inference budgets?

Could the authors provide a cost–coverage comparison incorporating generation, retrieval/reranking, and verifier execution, ideally with wall-clock latency and accelerator resources in addition to token counts? A matched-budget direct DSL writer baseline, sans the drafting stage, would clarify how much value is extracted by the multi-stage architecture over allocating the same resources to direct proof generation.

5 comments

Thank you for your detailed assessment and for recognizing the research value of the paired-view objective. Your questions have helped us identify where our experimental evidence and explanations need to be clearer. We are conducting further experiments on some of the issues you raised and address the questions about evaluation design and resource accounting below.

W1: Experimental isolation of PV-PKPO's contribution

Thank you for pointing this out. We are organizing supplementary experiments to address this issue and will provide the results as soon as the experiments are completed.

W2: Evaluation design and efficiency

Sampling and the purpose of the easier subset

Thank you for pointing this out. We use D₂ here because its problems are easier, which can amplify differences that may not be significant on the full set. Following your suggestion, we will provide results on the full set later.

Meaning of “component development”

The 1,000 evaluation gaps were not used as gradient-training examples. We acknowledge that our earlier use of “component development” was problematic and did not accurately convey its meaning. Here, it referred to participation in the ablation comparisons. We will update the corresponding wording in the paper.

Efficiency comparison

Our intended efficiency comparison concerns reported inference tokens. Notably, the five-round correction configuration already achieves higher coverage than Codex with fewer reported tokens:

ConfigurationCoverageMean reported tokens / gap
Codex25.00%491,750
Our system, five correction rounds34.95%477,359
Our system, union of both correction schedules35.70%608,230

The five-round correction configuration gains 9.95 percentage points over Codex with 2.93% fewer reported tokens, while remaining only 0.75 percentage points below the maximum coverage. Token counts follow Table 5's accounting convention and include drafting, proof generation, and correction.

We acknowledge that differences in model tokenization, input caching, retrieval, verifier execution, and training or data-generation costs make overall computational efficiency more difficult to assess, especially for end-to-end evaluation. We hope the data provided here can serve as an initial reference for computational efficiency.

Inference-budget context for the Lean comparison

Thank you for pointing this out. You are right; the comparison here is mainly provided as background.

Q1: PV-PKPO and paired-view credit assignment

Thank you for your question. Regarding the comparison between PV-PKPO and standard PKPO, we are organizing supplementary experiments and will provide the results as soon as the experiments are completed.

Q2: Component gains outside D₂

Thank you for your question. Regarding component gains outside D₂, we are organizing supplementary experiments and will provide the results as soon as the experiments are completed.

Q3: Additional discussion of the three-stage design

Thank you for your question. Regarding the direct DSL writer baseline without a drafting stage, we are organizing supplementary experiments and will provide the results as soon as the experiments are completed.

Additionally, we are pleased to report findings from a joint-SFT comparison of draft generation and DSL generation, which is related to your question. We used existing data to train a single model combining Models 1 and 2, which first produces a natural-language draft and then continues by outputting the formal proof. Both configurations use the same base model, checkpoint-selection rule, validation set, and number of test-time attempts. The staged baseline in this comparison uses the SFT-only Model 2 writer, matching the task-specific post-training stage of the unified SFT model. Neither configuration uses RL or Model 3 correction. We obtained the following results in the standard DSL checking environment:

Test setUnified model (joint SFT)Frozen Model 1 + SFT-only Model 2 (4×4)
D₂ (1,000 gaps)657 / 1,000 = 65.7%720 / 1,000 = 72.0%
Full 4,000 gaps1,026 / 4,000 = 25.65%1,137 / 4,000 = 28.43%

In this joint-SFT comparison, the unified model's coverage is lower by 6.3 percentage points on D₂ and 2.775 percentage points on the full test set. This supports using a frozen drafter with a separate writer in the current experimental setting.

We also provide an end-to-end runtime supplement. We compute the time per gap as total batch elapsed time divided by the number of evaluated gaps. The pipeline runs on 16 Ascend 910B NPUs, and its timing includes drafting, writer generation, correction, retrieval, and checker execution.

SystemWriter allocationCoverage (%)Input tokens/gapOutput tokens/gapTotal tokens/gapTime/gap (s)
Codex (gpt-5.6-sol, medium)Agent run25.000489,453.252,297.00491,750.2560–300 (estimated from tokens)
Draft + writer5×125.81220,262.4610,726.9430,989.403.96
Draft + writer5×227.89129,889.1815,645.9945,535.175.43
Draft + writer5×429.53349,142.6225,501.3474,643.968.36
Draft + writer10×832.375174,994.8190,505.28265,500.0925.94
Five correction rounds10×834.950338,770.43138,588.58477,359.0137.46
Five rounds + independent single round10×835.700453,019.95155,210.50608,230.4542.92

The pipeline entries are amortized batch times. Codex's 1–5 minute interval is inferred from token quantities.

Additional response to Q3: Direct DSL writing without Model 1

Thank you for suggesting the direct DSL writer baseline. We have completed the additional evaluation without Model 1 mentioned in our previous response and provide the results below.

Both configurations use the same Model 2 RL checkpoint, retrieval and decoding settings, and DSL verification environment. Model 3 correction is disabled in both. The direct baseline generates 16 DSL candidates per gap without a Model 1 draft. The existing Model 1 + Model 2 configuration uses four Model 1 drafts, with four DSL candidates per draft (4×4).

ConfigurationDSL candidates per gapFull 4,000 gapsD₂ (1,000 gaps)
Direct Model 2, no Model 1 draft161,133 / 4,000 = 28.325%716 / 1,000 = 71.60%
Model 1 + Model 24×41,156 / 4,000 = 28.900%725 / 1,000 = 72.50%

On the full 4,000-gap pool, the staged configuration achieves a net gain of 23 solved gaps (+0.575 percentage points) over direct generation. On D₂, the net gain is 9 solved gaps (+0.90 percentage points). These results show higher observed coverage for the drafting–writing configuration under the same allocation of 16 DSL candidates per gap.

Additional response to W2: Clarifying “component development”

Thank you again for your question. We have revised the imprecise wording “component development” in the manuscript to clarify that it refers to comparisons in the component ablation studies. Specifically:

  • Section 4.1 now states: “We use this fixed subset for component ablation comparisons.”
  • Appendix C.4 now states: “The set remains fixed across component ablation comparisons.”

We have also revised related wording in the appendices to consistently refer to D₂ as the component evaluation set and describe its use for component comparisons.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.