acceptodds
Under review as a conference paper at ICLR 2027

ReaLogic-FT: Progressive Neuro-Symbolic Post-Training for Reasoner-Verifiable Logical Reasoning

Abstract

Large language models often answer logical reasoning questions correctly, yet the reasoning they display is free-form text: no machine can confirm that each step follows from the premises or that the conclusion is supported. Making such reasoning verifiable faces two obstacles. Facts, rules, and inferences must be expressed in a formal language the model does not natively speak, and training on reference proofs ties it to a single derivation per problem, penalizing equally valid alternatives while giving no signal about logical soundness. We present **ReaLogic-FT**, a progressive post-training framework that removes both. The model first learns to translate natural-language statements into the description logic , decidable yet expressive enough for the theories we retain, so that a reasoner can conclusively check every claimed inference. It then learns to compose such assertions into structured proofs ending in a verdict of *True*, *False*, or *Unknown*. A final reinforcement learning stage drops the reference proofs entirely: an off-the-shelf reasoner checks each candidate derivation step by step, and any proof grounded in the given theory and logically valid is rewarded. We build verifier-checkable versions of ProofWriter and ProntoQA, projecting each problem onto the supported logic and adding queries that can be neither proved nor refuted. Proof supervision raises joint answer-and-proof correctness by over 40 percentage points relative to few-shot prompting. HermiT-guided reinforcement learning then matches or improves the strong SFT initialization on every reported metric and outperforms its semantic-reward ablation in mean performance; these differences are small and not statistically significant, with the clearest gains on deeper ProofWriter instances. Every accepted proof step can be independently audited by a symbolic reasoner against the given formal theory.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Large Language Models, LLM Post-Training, Neuro-Symbolic Reasoning, Verifier-Guided Reinforcement Learning, Description Logic

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_CCHxOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses an important problem in logical reasoning: an LLM may produce the correct answer while its reasoning process remains difficult to verify. To address this issue, the paper introduces ReaLogic-FT, a progressive post-training framework for generating formally verifiable reasoning traces. Experiments on ProofWriter and ProntoQA show that structured proof supervision improves both answer accuracy and proof quality, while the additional gains from verifier-guided reinforcement learning are relatively modest.

Critical strengths
  1. The paper targets an important problem: LLM reasoning needs to be verifiable.
  2. The methodology is written clearly and rigorously, with good reproducibility.
Critical weaknesses
  1. The experiments are relatively weak. The evaluation is limited to two well-defined, highly structured logical reasoning benchmarks and includes only a small set of baselines. Moreover, the gains from verifier-guided RL are marginal, which raises questions about the effectiveness and necessity of this component. Overall, the empirical significance and generalizability of the proposed method remain unclear.
  2. The novelty is also not sufficiently established. Prior work, such as LogicReward and VeriCoT, already uses formal verifiers together with reinforcement learning to improve intermediate reasoning. It is therefore unclear what new insight ReaLogic-FT provides beyond adopting a different formal representation and incorporating verifier feedback into GRPO. Furthermore, if the main novelty lies in generating structured proofs that can be formally verified, it is unclear why directly generating executable formal representations, such as Lean or Prolog, would not provide a simpler or stronger alternative. The paper would benefit from a clearer justification of why its specific formal representation and training design are necessary, and from additional experiments comparing against prior approaches such as LogicReward and VeriCoT, as well as alternatives that directly generate executable formal proofs, such as Lean.
Questions for authors

Could you add experiments on more benchmarks, such as Multi-LogiEval, FOLIO, and LogiQA, and include comparisons with LogicReward, VeriCoT, and Lean? It would also be helpful to provide a clearer justification for the paper’s positioning relative to these approaches.

Reference • Multi-LogiEval: Towards Evaluating Multi-Step Logical Reasoning Ability of Large Language Models — EMNLP 2024. • FOLIO: Natural Language Reasoning with First-Order Logic — EMNLP 2024. • LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning — IJCAI 2020.

3 comments
AuthorsOpenReview ↗

Response to Reviewer CCHx (Part 1 of 2)

Response to Reviewer CCHx (Part 1 of 2)

Thank you for the detailed comments on novelty and on the choice of formalism. They helped us make both explicit in the revision.

W1. "The experiments are relatively weak. The evaluation is limited to two well-defined, highly structured logical reasoning benchmarks and includes only a small set of baselines. Moreover, the gains from verifier-guided RL are marginal, which raises questions about the effectiveness and necessity of this component. Overall, the empirical significance and generalizability of the proposed method remain unclear."

  • Baselines → three external systems with the same backbone (T2). ReaLogic-FT is the only one above 90% on both datasets.
  • Generalizability → three backbones for the supervised stages, two for RL (T1, T4).
  • Effectiveness of RL → on problems that require a proof, fully verified outputs rise by +2.6+2.6 to +8.4+8.4 points for two backbones (T3), with Holm-significant aggregate gains in the settings marked in T4.
  • Necessity → the supervised stages provide most of the improvement, and we present them as the core of the method. The RL stage adds the gain in T3, which the supervised model alone does not reach. Our updated ablation (Table 20) indicates that this gain is obtained already with the format and answer rewards on reasoner-screened prompts, so we now describe the RL stage in these terms.
  • Benchmarks → please see our answer to your question below.

W2 (first part). "Prior work, such as LogicReward and VeriCoT, already uses formal verifiers together with reinforcement learning to improve intermediate reasoning. It is therefore unclear what new insight ReaLogic-FT provides beyond adopting a different formal representation and incorporating verifier feedback into GRPO."

Answer: the difference lies in what the trained model outputs and in what the verifier can certify. We agree that these are closely related works, and Sec. 1 now discusses them explicitly.

T6. Positioning (to the best of our reading of each paper)

Training signalVerifierWho writes the formal objectsUnknown
LogicRewardoffline SFT + DPOIsabellea separate LLM formalizes the model's natural-language stepslabel match
VeriCoToffline SFT + DPOZ3a separate LLM formalizes steps and supplies premisesabsent from its ProofWriter subset
PRoSFIonline GRPOLean / Z3 / Prover9formal premises are given in the promptuncertain instances excluded
ReaLogic-FTonline Dr. GRPOHermiTthe trained model itself, from natural languagecertified by the reasoner
  • (i) The formal proof is the model's own output. In LogicReward and VeriCoT, the trained model writes natural language and another LLM translates it for the verifier. Here an 8B model emits the formal query and proof itself, so every step can be checked at inference time without an auxiliary LLM.
  • (ii) Unknown is a certified verdict. In a decidable logic the reasoner establishes that neither the query nor its negation follows; the other methods exclude such instances or match a label.
  • Insight → proof supervision in a decidable formal language closes most of the gap between correct answers and verifiable proofs for 8B models (T1), and reasoner-based evaluation exposes errors and benchmark label artifacts that accuracy hides (T5).
AuthorsOpenReview ↗

Response to Reviewer CCHx (Part 2 of 2)

Response to Reviewer CCHx (Part 2 of 2)

W2 (second part). "... it is unclear why directly generating executable formal representations, such as Lean or Prolog, would not provide a simpler or stronger alternative. The paper would benefit from a clearer justification of why its specific formal representation and training design are necessary ..."

Answer: our proofs are executable formal objects (every step is an OWL assertion that HermiT checks). We chose a description logic because of the Unknown class.

  • Lean → failing to prove a statement and its negation does not show that neither holds, so Unknown cannot be certified.
  • Prolog-style engines → negation as failure gives a closed-world reading, and forward rules with explicit negation miss classical consequences such as contrapositives. This is precisely the source of the 37 label mismatches in ProofWriter (T5).
  • ALC with HermiT → decidable, sound, complete, and terminating, so the three-way verdict is returned directly.
  • Published evidence at this model scale → PRoSFI reports that a 7B model trained to emit Lean directly obtains low verified reward, and Leang et al. (ACL 2025) need iterative autoformalization to raise the Lean execution rate. For this reason, and given our compute, we did not run a Lean baseline.
  • Training design → proof supervision is essential (T1: at best 46.88%46.88\% and 38.42%38.42\% G-Joint without it, 89.24%89.24\% and 90.84%90.84\% with it); assertion-level alignment helps where formalization is hard (iLVy, Q1); RL then improves proofs on the problems that require one (T3).

Q. "Could you add experiments on more benchmarks, such as Multi-LogiEval, FOLIO, and LogiQA, and include comparisons with LogicReward, VeriCoT, and Lean? It would also be helpful to provide a clearer justification for the paper's positioning relative to these approaches."

We appreciate these suggestions. We could not add them within this period, and beyond compute there is a specific reason for each: the method needs, per problem, a formalization against which every step can be checked, and a three-way label.

RequestedStatus
FOLIOsatisfies both conditions and is our next benchmark; it first requires identifying which of its theories fall within ALC
Multi-LogiEvalbinary labels without Unknown, and it includes non-monotonic reasoning, which a monotonic logic does not express
LogiQAfour-option reading comprehension that relies on implicit and commonsense premises; solver-based methods struggle there in general (Aristotle reports 31.2%31.2\% against 65.6%65.6\% for chain-of-thought)
LogicReward, VeriCoTtheir training data are built with GPT-4o and Claude 3.5 Sonnet, so a faithful reproduction with an 8B model is not possible, and their published numbers use different samples and label sets
Leanplease see W2 (second part)
  • Positioning → T6, Sec. 1, and App. G.
  • Presentation → following your rating, we revised the text so that every abbreviation and symbol is defined at first use and each metric is defined before the results.

Dear Reviewer CCHx,

Thank you again for checking T2 against the original papers. Your question led us to add the comparison of settings and the breakdowns by label and depth (App. F.5), which we think makes the baseline comparison much easier to trust. We would be glad to clarify anything further, on T2 or on the positioning against LogicReward and VeriCoT. If the revision addresses your concerns, we would be grateful if your final assessment could take it into account. Either way, we sincerely appreciate the care you put into this review!

Reviewer_iLVyOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Authors present a 3-stage LLM fine-tuning recipe for logical reasoning. Stage 1 uses SFT to convert natural language to OWL ALC (formal description language usable by reasoning engines such as HermiT). Stage 2 uses SFT to produce structured proofs. Stage 3 uses RL to remove dependence on reference proofs using HermiT as a verifier to supply reward.

The paper fine-tunes LoRA adapters for Qwen3-8B and targets ProofWriter and ProntoQA datasets. Stage 2 is shown to raise correctness substantially over few-shot baselines but the RL stage yields small aggregate mean improvements. Authors additionally explore an ablation over the reward formulation.

Critical strengths

The paper provides a simple recipe that shows strong improvement on the selected tasks. Performance after each stage is evaluated and reported allowing for granular understanding of which pieces have what contribution.

Moreover the verifier provides strong grounding on the process level, disincentivizing unfaithful reasoning or incorrect reasoning that happen to produce correct outcomes. This audit is useful for both evaluation of proofs as well as a robust reward signal to optimize against.

Critical weaknesses
  1. The final RL stage of the recipe which is purported to mitigate modal collapse towards the reference solution is not empirically supported by their experiments. While improvement is substantial in the SFT phase, RL seems to contribute much smaller performance gains with 0 inside confidence intervals.

  2. Scope seems limited with only one base model tested against 2 narrowly scoped logical reasoning benchmark and no external benchmarks besides intra-stage checkpoints for their own method. It's unclear if this method's efficacy holds across models, if this method generalizes to do more general reasoning problems, or how it compares to other methods.

Questions for authors
  1. Have you considered compute-matched methods that don't use this staged approach? (e.g. using the same amount of compute to do RL from the initial checkpoint, or only doing the structured proof SFT without assertion level alignment first)
3 comments
AuthorsOpenReview ↗

Response to Reviewer iLVy

Response to Reviewer iLVy

Thank you for the thoughtful review; your first point in particular led us to measure something we had only argued for.

W1. "The final RL stage of the recipe which is purported to mitigate modal collapse towards the reference solution is not empirically supported by their experiments. While improvement is substantial in the SFT phase, RL seems to contribute much smaller performance gains with 0 inside confidence intervals."

On the size and significance of the RL gains → with a larger budget and a second backbone, the gains are now significant after Holm correction in the settings marked in T4, and on problems that require a proof the fully verified rate rises by +8.36+8.36, +2.58+2.58, and +6.88+6.88 points (T3).

On collapse toward the reference → we agree that this needed direct evidence, so we measured how often a verified proof differs from the reference:

Verified proofs that differ from the referenceSFTSFT + RL
ProofWriter19 / 324 (5.86%5.86\%)24 / 369 (6.50%6.50\%)
ProntoQA24 / 1,993 (1.20%1.20\%)26 / 2,051 (1.27%1.27\%)
  • → ProntoQA problems have a single proof by construction, and alternative verified proofs are rare on ProofWriter as well, so there is little room for them, and RL leaves their share about the same.
  • → We have therefore reworded the paper. The RL reward is reference-free by construction, and its measured benefit is more verified proofs (T3), which is what the revised abstract and Sec. 4.3 now state.

W2. "Scope seems limited with only one base model tested against 2 narrowly scoped logical reasoning benchmark and no external benchmarks besides intra-stage checkpoints for their own method. It's unclear if this method's efficacy holds across models, if this method generalizes to do more general reasoning problems, or how it compares to other methods."

  • Across models → yes. Both new backbones exceed Qwen3-8B after the supervised stages (T1), and RL with Llama reproduces the ProofWriter gains (T3, T4).
  • Against other methods → first on ProntoQA, second on ProofWriter, and the only system above 90% on both (T2).
  • More general reasoning problems → the method targets rule-based reasoning that a decidable logic can express, since this is what makes every step checkable. Extending it to human-written theories is our next step, starting with FOLIO (App. G).

Q1. "Have you considered compute-matched methods that don't use this staged approach? (e.g. using the same amount of compute to do RL from the initial checkpoint, or only doing the structured proof SFT without assertion level alignment first)"

Your second example → we ran it (Table 24). This is the initialization ablation of the submitted configuration with Qwen3-8B and one seed (42). Both variants train Structured Proof SFT with identical data and settings; one starts from the Assertion-Level SFT adapter, the other from a fresh LoRA adapter on the base model. Both are evaluated on the same fixed test sets (2,500 ProofWriter and 2,936 ProntoQA problems) with greedy decoding, batch size 8, and at most 1,024 generated tokens:

Effect of skipping assertion-level alignment (Δ\Delta in pp)Acc.Macro-F1JointStep-F1
ProofWriter+0.56+0.56+0.59+0.59+0.68+0.68−0.67-0.67
ProntoQA−1.12-1.12−1.11-1.11−1.40-1.40−0.25-0.25
  • → The first stage helps on ProntoQA, where formalization is harder (zero-shot exact match 57.86%57.86\% vs. 83.02%83.02\%) and proof data are about 11 times scarcer; on ProofWriter the two variants are within one point.
  • → The comparison is close to compute-matched, because the first stage accounts for only 2.4%2.4\% of the supervised GPU-hours in that configuration.

Your first example → we could not run it in this period, for two reasons:

  • Compute → one RL run costs 22 to 63 GPU-hours per dataset on the cards we could rent.
  • Expected signal → without proof supervision, the base model produces no valid structured proof on any of the 5,436 test problems (G-Joint 0.000.00, Table 2), so the candidates in a group would initially differ almost only in their answers and give RL little to learn from about proofs. This is the motivation for placing RL after the supervised stages.

Thank you for the thorough rebuttal. The new results address my main concerns, with added backbones and external baselines strengthening the evaluation. I am raising my score to borderline/weak accept.

I still have 2 reservations:

  1. the updated ablation indicates the RL gains come from format and answer rewards on reasoner-screened prompts rather than from the step-level verifier reward.
  2. your diversity measurement shows that alternative proofs are rare and that RL does substantially move this needle, so the reference-collapse motivation is undersupported. The abstract and title should reflect this, presenting RL as improving verified-proof rates, not as verifier-guided exploration of alternative derivations. A compute-matched SFT-only control would further strengthen an argument for RL's positive effect in this pipeline.

Thank you for raising your score, and for stating the two remaining reservations so precisely. We agree with both. We have revised the paper accordingly (changes in blue), and we tested each point directly on the archived outputs, without new training (App. F.6).

1. Source of the RL gains → agreed, and now shown on the problems that require a proof.

Full-Sem. (%), Qwen3-8B, problems that require a proofSFTRL w/o Sem. (no step-level reward)RL, full rewardw/o Sem. −- SFTFull −- w/o Sem.
ProofWriter (n=538n = 538)60.2260.2269.5269.5268.5968.59+9.29+9.29 [6.69,11.90][6.69, 11.90]−0.93-0.93 [−2.42,0.37][-2.42, 0.37]
ProntoQA (n=2,250n = 2{,}250)88.5888.5891.1191.1191.1691.16+2.53+2.53 [1.82,3.29][1.82, 3.29]+0.04+0.04 [−0.40,0.49][-0.40, 0.49]

(Differences in points, with unadjusted 95% paired bootstrap intervals.)

  • → The whole gain is obtained without the step-level reward. The gain of the w/o Sem. model grows with difficulty: from +1.18+1.18 points at depth 1 to +13.02+13.02 at depth 5 on ProofWriter, and from +0.67+0.67 at one hop to +4.44+4.44 at five hops on ProntoQA.
  • → What the screened prompts look like. After matching on difficulty and label, the eight SFT candidates already receive different format and answer rewards on 95.71%95.71\% of the selected ProofWriter prompts, against 42.41%42.41\% of the eligible prompts that were not selected (67.28%67.28\% against 5.74%5.74\% on ProntoQA). Reasoner-based screening thus retains mostly prompts on which the format and answer rewards already distinguish the candidates, which is consistent with your reading.
  • → Paper: the abstract states that the gains do not depend on the step-level reward, and the contribution list (Sec. 1) and Sec. 4.3 attribute them to the format and answer rewards on reasoner-screened prompts.

2. Reference collapse → agreed; we have removed this motivation. Two further analyses confirm your reading and show why the benchmarks cannot support it.

Verified proofs whose derivation differs from the referenceSFTSFT + RL
ProofWriter (of 538538)5577
ProntoQA (of 2,2502{,}250)0000
  • Stricter criterion. The counts in our earlier reply used Step-F1 <1< 1, which does not separate a different derivation from extra steps or a different rendering of a step. Here we compare the dependency structure of rules and premises that leads to the conclusion.
  • Same problems. Among the problems verified under both models (319319 and 1,9891{,}989), no proof changes its structure. The two additional ProofWriter cases are problems that only the RL model verifies.
  • Little room. A search that is independent of the model outputs finds a second, structurally different verified proof for only 44 of 6464 sampled ProofWriter problems and 00 of 8080 sampled ProntoQA problems (problems that require a proof, stratified by difficulty and label; these are lower bounds, since the search covers explicit subclass rules only).
  • → Paper. Abstract, Sec. 1, and Sec. 3.4 no longer state that reference proofs penalize equally valid alternatives. The RL stage is described as using no reference proofs, and its measured effect as a higher rate of fully verified proofs. What remains is that overlap with one reference does not measure correctness, which concerns evaluation.
  • → Title. It names a property of the model's output (every step can be checked by a reasoner), not the RL mechanism, so we have kept it. We would be glad to adjust it if you feel it still suggests exploration.

3. Compute-matched SFT-only control → agreed; it is the right control, and we could not complete it in this period. The design is to continue Structured Proof SFT on the same 2,9122{,}912 screened ProofWriter prompts for the 13.6413.64 GPU-hours that RL uses up to the evaluated update, with the same development-based selection. We have listed it in the limitations (Sec. 5) and will include it in the next revision.

Reviewer_r2XDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ReaLogic-FT, a progressive neuro-symbolic post-training framework combining assertion-level SFT, structured proof SFT, and HermiT-guided reinforcement learning. Experiments on ProofWriter and ProntoQA demonstrate improvements in generating formally verifiable logical reasoning chains.

Critical strengths
  1. The progressive training framework provides a clear approach to learning formally verifiable reasoning.
  2. Integrating HermiT enables rigorous step-level verification of generated proofs.
  3. The framework explicitly handles three-way logical entailment, including UNKNOWN cases.
  4. Structured proof supervision substantially improves answer and proof correctness.
Critical weaknesses
  1. The improvements from verifier-guided RL are small and statistically insignificant.
  2. Comparisons with strong neuro-symbolic reasoning and self-training baselines are insufficient.
  3. Evaluation is limited to synthetic benchmarks with restricted logical expressiveness.
  4. Experiments rely on a single LLM backbone, limiting evidence of generalizability.
Questions for authors
  1. Can larger RL training budgets yield statistically significant improvements?
  2. How does ReaLogic-FT compare with Logic-LM, LINC, and recent verifier-guided methods?
  3. Can the framework generalize to human-written reasoning tasks or other LLM backbones?
  4. How do the 37 identified ontology-label mismatches affect evaluation reliability?
2 comments
AuthorsOpenReview ↗

Response to Reviewer r2XD (Part 1 of 2)

Response to Reviewer r2XD (Part 1 of 2)

Thank you for the clear and specific questions. We answer them in order.

W1. "The improvements from verifier-guided RL are small and statistically insignificant." Q1. "Can larger RL training budgets yield statistically significant improvements?"

Answer: yes. We raised the budget from 80 / 400 prompts to 2,048 groups of eight candidates and added a second backbone.

  • → Significant after Holm correction (T4): Qwen3-8B on ProntoQA in G-Joint and Full-Sem.; Llama-3.1-8B on ProofWriter in Acc., Step-F1, and U-F1.
  • → Qwen3-8B on ProofWriter: Acc. +1.08+1.08 and Step-F1 +3.89+3.89 with unadjusted intervals excluding zero (Holm p=0.078p = 0.078); both become significant once the 37 instances discussed in Q4 are set aside (T5).
  • → On problems that require a proof (T3): +8.36+8.36, +2.58+2.58, and +6.88+6.88 points of fully verified outputs, with intervals far from zero.

W2. "Comparisons with strong neuro-symbolic reasoning and self-training baselines are insufficient." Q2. "How does ReaLogic-FT compare with Logic-LM, LINC, and recent verifier-guided methods?"

Answer: with the same backbone, ReaLogic-FT is first on ProntoQA and second on ProofWriter among Logic-LM, SymbCoT, and Aristotle (T2).

  • ProntoQA → 96.36%96.36\%, 12.4712.47 points above the best baseline (SymbCoT).
  • ProofWriter → 92.48%92.48\%, 4.604.60 points below Logic-LM. Logic-LM hands deduction to Pyke, whose rule semantics are those that generated the ProofWriter labels, so a faithful translation reproduces the label. Its bottleneck is translation: on ProntoQA, 654 of its 729 errors occur after the translated program ran normally.
  • Beyond the answer → the baselines return a verdict; ReaLogic-FT also returns a proof, which HermiT verifies in full for 87.44%87.44\% and 91.59%91.59\% of the test problems.
  • LINC, verifier-guided methods, self-training → we were not able to run these in this period; the specific reason for each is given in the last table of the general response, and App. F lists how the three baselines were adapted.

W3. "Evaluation is limited to synthetic benchmarks with restricted logical expressiveness." Q3. "Can the framework generalize to human-written reasoning tasks or other LLM backbones?"

  • Other backbones → yes, shown in T1 and T4.
  • Human-written tasks → this is the natural next step, and we do not claim it yet. The method needs a formalization of each theory against which every step can be checked. ProofWriter and ProntoQA provide one, and they are the benchmarks used by Logic-LM, SymbCoT, Aristotle, and LogicReward, which makes the comparison in T2 possible.
  • On expressiveness → ALC includes negation, disjunction, and existential and universal restrictions. We chose it because it is decidable, which is what allows an Unknown verdict to be certified.
  • Plan → FOLIO is human-written with first-order annotations and three-way labels; extending to it requires identifying which of its theories ALC can express (App. G).

W4. "Experiments rely on a single LLM backbone, limiting evidence of generalizability."

Answer: addressed. The supervised stages now cover three backbones (T1) and RL covers two (T3, T4). All three reach 100%100\% exact match after Assertion-Level SFT, and both new backbones exceed Qwen3-8B on every metric after Structured Proof SFT.

AuthorsOpenReview ↗

Response to Reviewer r2XD (Part 2 of 2)

Response to Reviewer r2XD (Part 2 of 2)

Q4. "How do the 37 identified ontology-label mismatches affect evaluation reliability?"

Answer: they do not change any conclusion, and we have identified their cause.

  • All 37 carry the benchmark label Unknown, while the query (20) or its negation (17) is classically entailed. An independent enumeration of classical models confirms all 37.
  • Example → "Harry is young", "young people are not white", "round people are white". Classically, Harry is not round. Forward application of the rules never derives this, so the benchmark says Unknown.
  • FoVer (Pei et al., TACL 2025) independently reports such instances and traces them to ProofWriter's forward-only construction. The verifier is therefore detecting a property of the benchmark labels.
  • Scoring is conservative → accuracy uses the benchmark label, and these instances receive no Full-Sem. credit.

T5. ProofWriter, Qwen3-8B, SFT → SFT + RL, with and without the 37 instances

MetricAll 2,500: Δ\DeltaHolm ppWithout the 37: Δ\Delta95% CIHolm pp
Acc.+1.08+1.080.0780.078+1.26+1.26[0.37,2.15][0.37, 2.15]0.028\mathbf{0.028}
G-Joint−0.28-0.281.0001.000−0.12-0.12[−0.93,0.73][-0.93, 0.73]1.0001.000
Step-F1+3.89+3.890.0780.078+4.20+4.20[1.00,7.40][1.00, 7.40]0.039\mathbf{0.039}
U-F1+0.78+0.780.2020.202+0.97+0.97[0.13,1.83][0.13, 1.83]0.0700.070
Full-Sem.−0.16-0.161.0001.000−0.16-0.16[−0.97,0.69][-0.97, 0.69]1.0001.000

→ Every difference keeps its sign, and the Acc. and Step-F1 gains become significant.

Comments on the paper

5 comments
AuthorsOpenReview ↗

General Response to All Reviewers (Part 2 of 2)

General Response to All Reviewers (Part 2 of 2)

T4. Aggregate RL effect, SFT → SFT + RL (Δ\Delta in pp; bold = significant after Holm correction over the five metrics)

SettingAcc.G-JointStep-F1U-F1Full-Sem.
Qwen3-8B, ProntoQA+0.61+0.61+0.89\mathbf{+0.89} (p=0.043p = 0.043)+0.47+0.47+0.81+0.81+0.92\mathbf{+0.92} (p=0.041p = 0.041)
Qwen3-8B, ProofWriter+1.08+1.08 (p=0.078p = 0.078)−0.28-0.28+3.89+3.89 (p=0.078p = 0.078)+0.78+0.78−0.16-0.16
Llama-3.1-8B, ProofWriter+1.04\mathbf{+1.04} (p=0.014p = 0.014)+0.52+0.52+4.59\mathbf{+4.59} (p=0.008p = 0.008)+0.89\mathbf{+0.89} (p=0.020p = 0.020)+0.60+0.60
Llama-3.1-8B, ProntoQA0.000.00−0.07-0.07+0.04+0.04−0.15-0.15−0.07-0.07

→ The ProofWriter Step-F1 gain is reproduced in all four RL runs (2 backbones × 2 batch sizes): +3.89+3.89, +5.26+5.26, +4.59+4.59, +3.71+3.71, with every 95% interval excluding zero. → Llama on ProntoQA starts at ≥98.4%\geq 98.4\% on every metric, which leaves little room for further gains.

Requested experiments that we could not complete in this period

RequestedReasonWhat we provide
LogicReward, VeriCoTboth construct their training data with closed-source models (GPT-4o; Claude 3.5 Sonnet). Substituting an 8B model would no longer be the published method, and their reported numbers use different test samples and label setspositioning, T6
LeanUnknown cannot be certified in a proof assistant, and published results show that direct Lean generation is difficult at this model scaleCCHx, W2
LINCsame translate-then-solve route as Logic-LM, with majority voting addedLogic-LM row, T2
FOLIO, Multi-LogiEval, LogiQAeach requires either a formalization within a decidable logic or a three-way label that it does not provideCCHx, Q
RL from the base model; self-trainingeach needs a further 22 to 63 GPU-hours per run and dataset, which we could not obtainstage ablation, iLVy Q1
AuthorsOpenReview ↗

General Response to All Reviewers (Part 1 of 2)

General Response to All Reviewers (Part 1 of 2)

We sincerely thank all reviewers for their careful and constructive comments, and we apologize for the late reply. The 80 GB GPU used for the submission was no longer available to us, and with the whole community preparing CVPR submissions in this period, rental GPUs were extremely scarce: we could secure only eight 24 GB RTX 3090 cards. On this hardware one ProofWriter SFT run takes about 58 GPU-hours and one RL run 22 to 63 GPU-hours, so we ran the requested experiments in priority order and replied as soon as they had finished. Unless stated otherwise, the results below come from a complete re-run of the three stages on this hardware, and every comparison is made within one configuration; the only exception is the initialization ablation in our reply to Reviewer iLVy (Q1), which is retained from the submitted configuration. The revised PDF marks all changes in blue.

Your concerns → what we added

Concern (raised by)New evidenceWhere
RL gains small, not significant (all)larger RL budget, a second backbone, analysis by problem typeT3, T4; Sec. 4.3
No external baselines (all)Logic-LM, SymbCoT, Aristotle with the same backboneT2; Table 3
Single backbone (r2XD, iLVy)supervised stages on 3 backbones, RL on 2T1, T4
Novelty; why not Lean / Prolog (CCHx)explicit positioningT6; Sec. 1, App. G
37 label mismatches (r2XD)cause identified, sensitivity analysisT5; App. D.1

T1. Supervised stages on three backbones (test sets, %; Acc. / G-Joint / Step-F1)

ModelProofWriterProntoQA
Qwen3-8B, best prompting condition71.4471.44 / 46.8846.88 / 42.1942.1954.9454.94 / 38.4238.42 / 40.8940.89
Qwen3-8B, Structured Proof SFT91.4091.40 / 89.2489.24 / 57.9557.9595.7495.74 / 90.8490.84 / 92.3592.35
Llama-3.1-8B-Instruct, Structured Proof SFT94.6094.60 / 93.4093.40 / 70.9370.9399.6999.69 / 98.4098.40 / 98.5498.54
Mistral-7B-Instruct-v0.3, Structured Proof SFT97.5297.52 / 96.8496.84 / 83.7383.7399.8399.83 / 99.3299.32 / 99.4899.48

→ For Qwen3-8B, grounded joint correctness (G-Joint) rises over the best prompting condition by +42.36+42.36 [40.08,44.60][40.08, 44.60] points on ProofWriter and +52.42+52.42 [50.44,54.39][50.44, 54.39] points on ProntoQA (95% paired intervals; Holm p=0.0032p = 0.0032). G-Joint applies the grounding check to every row. The ProntoQA gain is similar to the +52.28+52.28-point Joint gain reported in the submission.

T2. External baselines (same Qwen3-8B backbone, same test instances; Acc. / Unknown-F1, %)

SystemProofWriterProntoQA
Logic-LM97.08\mathbf{97.08} / 97.21\mathbf{97.21}75.1775.17 / 65.6365.63
SymbCoT90.3290.32 / 90.6890.6883.8983.89 / 69.6669.66
Aristotle78.1278.12 / 86.0986.0953.9953.99 / 53.0253.02
ReaLogic-FT92.4892.48 / 93.1693.1696.36\mathbf{96.36} / 94.25\mathbf{94.25}

→ ReaLogic-FT is the only system above 90% on both datasets, in a single greedy pass and without a solver at inference time.

T3. Effect of RL on problems whose proof requires at least one inference step (share of outputs that pass full reasoner verification, Full-Sem., %; unadjusted 95% paired intervals)

SettingSFTSFT + RLΔ\Delta (pp)95% CInewly pass / newly fail
Qwen3-8B, ProofWriter (n=538n = 538)60.2260.2268.59\mathbf{68.59}+8.36+8.36[5.76,10.97][5.76, 10.97]50 / 5
Qwen3-8B, ProntoQA (n=2,250n = 2{,}250)88.5888.5891.16\mathbf{91.16}+2.58+2.58[1.91,3.29][1.91, 3.29]62 / 4
Llama-3.1-8B, ProofWriter (n=538n = 538)73.2373.2380.11\mathbf{80.11}+6.88+6.88[4.65,9.11][4.65, 9.11]39 / 2
Llama-3.1-8B, ProntoQA (n=2,250n = 2{,}250)97.9197.9197.9197.910.000.00[−0.22,0.22][-0.22, 0.22]3 / 3

→ RL improves verified proofs where a proof is actually required, for two backbones. Most ProofWriter test problems (1,962 of 2,500) need no inference step, and on those RL shifts some Unknown verdicts to definite ones. This is why the aggregate proof metrics in T4 understate the effect on proof construction.

I thank the authors for their efforts in running the additional experiments. However, I find the results in T2 a bit strange as it seems that better results are obtained in ProofWriter than ProntoQA for Logic-LM, SymbCoT and Aristotle but in their original paper, the results seem to the opposite?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.