- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes VeriLogic for three-way logical reasoning under a Lean4-formalized setting. It represents each response using Positive, Negative, and Unknown paths, with formal proofs for supported positive or negative claims. Its main training component, PSAR-GRPO, supplements a response-level advantage with separately normalized path-level advantages, routing local feedback only to the corresponding token spans. Local rewards combine gold-label-based path verdict correctness, proof verification, and consistency checks. At inference time, the system allows up to three additional verifier-guided repairs before falling back to a prediction from the current reasoning paths. Experiments use three backbone models and processed subsets of six benchmarks. The full system reports macro-average improvements of 1.83, 6.39, and 5.00 percentage points over the strongest available corresponding baselines for Qwen3-8B, Qwen3-4B, and Llama-3.1-8B, respectively. Ablations examine formalization, policy optimization, advantage routing, and iterative refinement.
Critical strengths
- A concrete approach to credit assignment within structured reasoning responses. PSAR-GRPO specifies how to normalize local rewards within each path type and apply the resulting advantages to aligned token spans, while retaining global supervision over the complete response. This is a meaningful technical contribution beyond simply adding another response-level reward: it allows a response with heterogeneous path correctness to receive differentiated updates. The appendix also correctly distinguishes proof failure from non-derivability, explicitly avoiding the invalid inference that two failed proof attempts establish Unknown.
- Encouraging empirical coverage across model families and benchmark sources. The full system improves average accuracy on all three backbones, and Table 1 reports improvements over the available baselines across all six processed benchmark subsets. This breadth makes the method more promising than a single-model or single-dataset demonstration. The component and routing ablations are also useful attempts to investigate the source of improvement, although their interpretation requires the clarifications below.
Critical weaknesses
-
The validity of the transformed evaluation instances is insufficiently established. Appendix B.1 describes model-assisted label normalization and applies verification-based retention to both training and evaluation data: every retained example must pass formalization checks, and True/False examples additionally require an accepted proof, whereas Unknown examples do not. This is a legitimate way to construct a restricted benchmark, but it makes the translation, relabeling, and retention pipeline central to the validity of the evaluation. The supplied PDF does not report the identity of the stronger model, the frequency of label changes, pre-filtering counts and retention rates, or a semantic-fidelity audit. The split description also does not establish the relationship to the original dataset splits or the treatment of shared source contexts.
An accepted proof establishes a proposition under the supplied formal premises; it does not establish that those premises faithfully preserve the original natural-language problem. The paper itself acknowledges this distinction, but does not provide sufficient evidence that the distinction is controlled in dataset construction. In particular, it is unclear how premise consistency and the continued validity of Unknown labels are checked after translation. My concern is not that only VeriLogic receives formalized inputs but that the semantic validity and selection effects of the shared evaluation set remain insufficiently characterized. This limits how confidently the results support the paper’s natural-language logical-reasoning claims. -
The contribution of the proposed training method is not cleanly isolated from inference-time refinement and auxiliary label supervision. Figure 3 reports 92.70% for Formalization + GRPO and 93.67% for Formalization + PSAR-GRPO, a 0.97-point improvement. The full system reaches 96.53%, adding another 2.86 points with refinement. However, the corresponding GRPO + the same verifier-guided refinement control is missing. Moreover, Table 2 reports 96.53% for “Correct routing,” rather than the 93.67% training-only result in Figure 3, without clearly stating the refinement configuration for every routing alternative. Matching structured outputs, rewards, and training budgets does not establish that test-time conditions are also matched. Consequently, the current evidence does not isolate the contribution of the proposed routing mechanism within the full system. There is a related ambiguity about what provides the useful local supervision. Appendix A.2 shows that path correctness is assessed through explicit verdicts against the gold label, while consistency uses verdicts, proof outcomes, and the final answer. It explicitly states that individual natural-language reasoning steps are not independently verified. Thus, the evidence supports a narrower mechanism rather than a general claim of verified intermediate reasoning. A control retaining gold path-label supervision while removing proof-derived feedback would help establish the importance of formal verification itself. The absence of reported run-to-run variability also matters when interpreting the 0.97-point training-only improvement.
Questions for authors
- Please specify the model and protocol used for translation and label normalization, and provide per-dataset retention rates, label-change counts, and class distributions. How are semantic fidelity, premise consistency, and Unknown-label validity checked after translation? Please also clarify the original split/source-instance mapping and whether reference proofs or label-derived information are excluded from evaluation inputs. Representative before/after examples and an independent audit would materially strengthen confidence in the evaluation.
- Does every configuration in Table 2 use the same verifier-guided refinement procedure and budget? Please reconcile its 96.53% “Correct routing” result with the 93.67% PSAR-GRPO result in Figure 3. What are the results for GRPO and PSAR-GRPO both with and without the same refinement procedure, under matched decoding and verifier-call budgets? Please report uncertainty for the key training-method comparison.
- Appendix B.1 states that retained evaluation examples already pass formalization checks, whereas Figure 5 starts from a 58.0% formalization pass rate. What exactly is being formalized in that experiment, by which model, and over which denominator? Are the premise context and target proposition fixed during repair, or can the model modify them? How are the restrictions against additional axioms, sorry, and admit enforced by the verification implementation, beyond their inclusion in the simplified prompt?
- How much improvement remains when local gold path-verdict supervision is retained but all proof-derived reward information is removed, including the verification-dependent portion of consistency? Please provide the numerical reward weights used in these comparisons and report both answer accuracy and verified-proof success under identical inference conditions. This would help distinguish learning from auxiliary label supervision from learning specifically enabled by formal verification.
5 comments
Response to Reviewer q538 – [2/2]
W2 & Q2 & Q4
Under matched inference conditions, how much do routing, refinement, and proof-derived supervision each contribute to answer accuracy and verified-proof success, what reward weights were used, and how much do the training results vary across runs?
(1) Separating training and refinement contributions.
With the same formalization and zero repair, PSAR-GRPO improves answer accuracy and initial proof acceptance over Global GRPO.
Table 3 reports both metrics at zero and three repair rounds. We match decoding settings and verifier-call budgets at each setting. Higher proof acceptance after three rounds is consistent with stronger initial proof construction.
The Correct Routing (Ours) entry in VeriLogic Table 2 used test-time refinement. The other routing controls used zero repair. We will update that entry from 96.53% to 93.67% so that all routing controls are compared without repair.
Table 3. Answer accuracy and proof acceptance (%).
| Method | Acc. (0) | Proof (0) | Acc. (3) | Proof (3) |
|---|---|---|---|---|
| Global GRPO | 92.70 | 25.3 | 93.86 | 58.7 |
| PSAR-GRPO | 93.67 | 42.3 | 96.53 | 76.4 |
Numbers in parentheses are repair rounds.
(2) Isolating proof-derived supervision.
PSAR-GRPO (Gold Label Only) keeps the three-path structure, gold path-verdict supervision, path-specific advantage normalization, and token routing. We remove all proof-derived rewards, including verification-dependent consistency signals. Other settings stay fixed.
Gold labels score verdicts, and Lean4 checks proofs. Under the same zero-repair setting, Gold Label Only achieves 93.47% macro-average answer accuracy and 27.1% proof acceptance.
In the global reward, we weight answer correctness at 0.8 and format correctness at 0.2. The local reward assigns 0.4 to path-verdict correctness, 0.4 to Lean4 proof verification, and 0.2 to path consistency. The local advantage coefficient is 0.1.
(3) Verification of intermediate formal reasoning steps.
Lean4 checks the formal derivation and final conclusion in proofs for the Positive and Negative paths. An invalid inference invalidates the proof. PSAR-GRPO uses verification outcomes as path-specific learning signals for formal reasoning and proof generation. Independent verification of individual natural-language reasoning steps is outside the scope of this work.
(4) Consistency of training improvements.
Without test-time refinement, PSAR-GRPO outperforms the reported Global GRPO baseline at each of the three independent training seeds (Table 4). The small variation across runs suggests that the training-only gain holds beyond a single favorable run.
Table 4. Zero-repair accuracy by training seed.
| Seed | Macro-average (%) | Gain vs. reported GRPO (pp) |
|---|---|---|
| 1273981739 | 93.69 | +0.99 |
| 1837049261 | 93.67 | +0.97 |
| 2065813749 | 93.64 | +0.94 |
Q3
What was formalized in Figure 5, by which model and over which denominator, and how does the verifier keep the premises and target fixed during proof repair and enforce the restrictions on additional axioms,
sorry, andadmit?
(1) Formalization pass rate and dataset filtering.
Figure 5 measures formalization before filtering. DeepSeek-V3.2 translates natural-language problems into Lean4 and uses Lean4 feedback to repair failed formalizations over multiple rounds. The denominator is all 13,304 candidates (9,500 training and 3,804 evaluation). The Lean4 pass rate rises from 58.0% on the first attempt to 99.6% after repair. Figure 5 reports proof acceptance separately.
Appendix B.1 describes later filtering. All retained instances pass formalization checks. Checks on proof validity and suitability of proof strategies for training further reduce retention.
(2) Constraints during iterative refinement.
During formalization repair, the external model may revise the Lean4 representation using verifier feedback. During proof repair, the premise context () and target proposition () stay fixed. The policy can revise only the proof attempt.
(3) Enforcement of proof validity.
During both RL training and inference, we check extracted proofs in code before sending them to Lean4. We reject proofs that contain prohibited constructs such as sorry or admit. These attempts receive negative rewards in training and count as invalid proofs in evaluation.
The verifier also rejects new axiom declarations. It checks proof dependencies against an allowlist of the fixed premises and permitted standard Lean axioms. We will describe the checks in the revised appendix.
Thank you again for your detailed comments. We appreciate the opportunity to clarify these points and would be glad to answer any further questions.
Dear Reviewer,
We hope our response has addressed the concerns raised in the review. As the discussion period is approaching its end, we wanted to check whether there are any remaining questions or points we could further clarify. We would be glad to provide additional detail if helpful.
Best regards, The Authors
Response to Reviewer q538 – [1/2]
Thank you for your detailed feedback! We appreciate your recognition of PSAR-GRPO’s contribution to credit assignment within structured reasoning responses and of the empirical coverage across model families and benchmark sources. We address the weaknesses and questions you raised below.
W1 & Q1
(1) Formalization and label normalization.
We use DeepSeek-V3.2 to formalize problems and normalize their labels. The structured template spells out the entities, predicates, facts, rules, and queries so we can compare them directly with the original problems.
For LogiQA and MultiLogic, we separate refutable False cases from underdetermined Unknown cases to make the three-way labels more consistent. Table 1 reports the class distributions, and Table 2 reports the label reassignments.
During label normalization, DeepSeek-V3.2 uses the original natural-language premises to decide whether the answer is underdetermined. We do not use failed Lean4 proof attempts to assign Unknown.
(2) Dataset construction and fair evaluation.
We apply the same retention criteria to training and evaluation. Every retained instance passes Lean4 formalization checks. True/False instances must also have an accepted proof of the corresponding conclusion.
We evaluate all compared methods on exactly the same retained set. We will clarify preprocessing and retention in Appendix B.1. Table 1 also lists the candidate counts, retention rates, and final evaluation sizes for each dataset.
(3) Semantic fidelity.
During dataset preparation, we checked semantic fidelity by comparing formalizations with the original natural-language problems. The original paper did not explicitly report these human checks.
Of the 180 audited formalizations, 175 (97.2%) were semantically faithful. All 97 with an accepted proof (53.9% of the audit) were faithful, and all five unfaithful samples lacked an accepted proof. The human checks provide empirical evidence of semantic alignment beyond Lean4 verification.
(4) Split separation and evaluation inputs.
Where official splits exist, we keep the dataset authors' original training and evaluation partitions. For datasets without official splits, the problems are constructed independently, with no shared contexts or templates.
We split these datasets at the source-problem level into disjoint training and evaluation subsets. Instances that share a source context stay in the same partition.
At evaluation time, the model receives only the formalized premises and two unproved targets. The targets are the query and its negation. We exclude reference labels and completed proofs to prevent direct answer leakage. The representative example below illustrates this setup.
Table 1. Dataset composition and retention.
Table 2. Label reassignments.
Representative formalization example
Question: Is Penelope adventurous? Reference label: False (excluded from the model input).
Lean4 premises (excerpt)
Proof targets:
Adventurous Penelopeand¬ Adventurous Penelope.