acceptodds
Under review as a conference paper at ICLR 2027

NF-Bridge: Faithfully Bridging Natural and Formal Languages for Mathematical Reasoning

Abstract

Natural-language and formal-language reasoning and proving constitute two important paths for advancing mathematical theory: the former supports flexible mathematical expression, while the latter provides rigorous machine verification. However, faithfully converting between these two proof forms and effectively combining their respective strengths remains a central challenge. To address this problem, we propose NF-Bridge, a faithfulness-centered framework that connects natural-language mathematical reasoning with formal verification. NF-Bridge first converts a candidate natural-language solution into a semantic proposition DAG, which explicitly represents answer-relevant mathematical assertions and their dependency structure, while abstracting away rhetorical and repetitive content without discarding key mathematical commitments. The entire DAG is then formalized into Lean representations. Compilation-based verification and semantic-fidelity evaluation are subsequently used to assess, respectively, the syntactic and type-level validity of the formalization and its consistency with the original natural-language argument. We further demonstrate that NF-Bridge not only supports faithful autoformalization of natural-language proofs, but also enables the construction of formal-verification-grounded reward signals for improving mathematical reasoning. Across AIME24, AIME25, MATH-500, and BRUMO25, the verifier-based reward derived from NF-Bridge outperforms multiple learned reward-model baselines. As additional evaluations of its underlying proof-autoformalization capability, NF-Bridge also achieves strong results on ProofFlowBench and RobustPABench.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Proof autoformalization, Reward modeling, Lean 4

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_YeNnOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper tackles the problem of autoformalization and test-time selection of samples for both informal and formal math domains. The general idea relies on building a graph, defining persisting objects/statements, and operating on them. The work shows experimental advantage of the proposed NF-Bridge method.

Critical strengths

Overall, the paper presents a straightforward idea for scaling inference and bridging natural language and Lean.

Strengths

  • Useful construction. The authors use both lean and NL to construct aligned DAGs, which are subsequently used for TTL/reasoning.
  • Wide array of benchmarks. This work includes many baselines and different scenarios.
Critical weaknesses

Weaknesses

  1. Missing baselines. The authors pose a question in the introduction: How can formal verification be translated into effective signals for TTS?, where TTS is test-time selection. And then Table 2 reports how the proposed method is better than other ORM/PRM models. But there already was the work on this: Safe [1]. And what is even more jarring is that they cite this work without elaborating on it in Related works or main section. Including a citation for it just being shown in the References section is already questionable, but omitting the work that directly proposes an answer from your introduction section is purposefully misleading. [1] proposed simple fine-tuning using generated Lean steps and combining them with ORM/PRM models for best test-time selection.

Moreover, this work is eerily similar to Hermes [2], where the abstracts bring up similar problems, proof of Q\mathcal{Q} and ¬Q\neg\mathcal{Q}, which I believe was proposed there (correct me if I'm wrong, I am unsure on this point), and the general idea of interleaving and combining natural language with formal language verification. I understand that the settings are slightly different, but it seems like a lot of re-used parts from that agent combined with DAG (which is not novel too) and wrapping as a new method. Overall, reading this work was giving me a serious déjà vu and the presentation of this work leans me more towards negative assessment, especially given that authors do not explicitly compare NF-Bridge with Safe, a rather straightforward baseline. Given that Safe was cited but not acknowledged anywhere and many re-used parts from another work without citation, I have rather serious concerns regarding this work.

  1. Unreliable evaluations. Moreover, I would argue that the auto formalization experiments and results are not adequate enough. First, the work lacks any back translation mechanism that is a staple established by [3] and discussed in many auto formalization works. The omission of it is an oversight. Second, the work makes confident claims as ~50% boost in proof auto formalization compared to some strong models. And who evaluated this? Another LLM. For a work that tries to answer a notoriously difficult and intricate question of auto formalization, this work lacks any kind of human ablation. And manual inspection is the golden standard, we cannot rely on semantic faithfulness made by LLM when evaluating another set of LLMs.

Moreover, I have a separate problem with Table 1, where first for models were not built for proof autoformalziation. This comparison is misleading and unfair. And in general it is hard to believe these massive improvements on datasets like MATH-500/miniF2F that are pretty much solved. I would like to see some confidence intervals to verify how stable this method is. And pick suitable strong baseline models for this task, instead of opting for models that were designed for theorem proving and not proof autoformalziation.

  1. Novelty and writing. In short: the writing and presentation of this work are abysmal.

The following are their most jarring examples: the narrative jumps around too much; Table 3 is never discussed; Tables 4 and 6 do not reveal which benchmarks were used; unnecessary and confusing introduction with Holistic-DAG/Uni-DAG that are not standard in literature and so weakly connected to the overall message; misleading and messy abstract that seemingly describes an agent but the text talks about fine-tuning and GRPO; figures use comic sans for crying out loud; and many, many more mistakes and inconsistencies I found but there is not enough space to lay them out here.

The writing is so messy, I had to read it thrice, and even then I am confused on the message and what authors wanted to say. None of the moving parts make sense; the introduction is talking about DAG, and then jumps to method, no related works in the main section. Descriptions and analysis are lacking. Somehow, authors could not put any clear, complete idea onto 9 pages. I would highly recommend authors to scrap everything and write from scratch, and stick to 1-2 main results with everything else in the Appendix. As it stands right now, this work is borderline unreadable.

And I think because writing is so messy, the novelty is buried. I am already not convinced by the results and baselines choices, but the abstract and introduction themselves are quite poorly written. The contributions mention improved autoformalization and Lean-powered TTS, but the results are weak, lack human inspection, and some baselines do not even appear. Overall, the paper should be re-worked and I recommend authors spend more time on writing instead of releasing an incoherent manuscript that is so jarringly difficult to follow.


References

[1] Liu et al. 2025. Safe: Enhancing Mathematical Reasoning in Large Language Models via Retrospective Step-aware Formal Verification. ACL

[2] Ospanov et al. 2026. HERMES: Towards Efficient and Verifiable Mathematical Reasoning in LLMs. ICML

[3] Gao et al. 2025. Herald: A Natural Language Annotated Lean 4 Dataset. ICLR

Questions for authors

Questions

  1. What is the novelty compared to other DAG methods, Safe, Hermes?

  2. Why Safe was cited but not acknowledged? Why is it not a baseline?

  3. What is the purpose of Figure 1 in the narrative of this work? How is it even connected to the proposed method?

Reviewer_7yFbOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

NF-Bridge formalizes a natural-language solution into Lean in three stages:

  1. Extract the premises and freeze "shared Lean terms" (objects plus a target term), checked by the compiler and an LLM judge.
  2. Build a natural-language proposition DAG and a paired Lean DAG, with compiler feedback and an LLM semantic judge.
  3. For each node in dependency order, try to prove the thoemre and its exact negation with a tool-using prover

Node states get aggregated into a score used to rank Qwen3-8B candidates on AIME24/25, BRUMO25 and MATH-500. The paper also reports semantic faithfulness on ProofFlowBench and miniF2F, and FullyCorrect on RobustPABench global perturbations.

Critical strengths

Proved / disproved / unresolved at node level. Proving the exact outer negation gives a real refutation certificate, which a failed proof search cannot.

Engineering. There is a trained node-level prover with verifier-grounded GRPO, a full pipeline evaluated on four task families, and complete prompts.

Critical weaknesses

Not enough novelty. The method is very similar to the ProofFlow loop and other DAG based pipelines. NL DAG, per-node Lean lemmas with prescribed dependencies, verifier feedback, repeat until each node goes through, then dependency-aware proving. I believe freezing definitions before formalizing proofs is in M2F, and also in Goedel-Architect and AMBER (both uncited). Prove-or-disprove per node is in Safe and LeanReasoner. What seems new is the implicit-premise extraction.

Would a generic agent do the same? The paper does not compare against a coding agent (Codex, OpenCode) given Lean and search tools and a /goal like "first build an informal blueprint, then formalize it faithfully". Evidence from milestone formalization projects such as FLT is that less orchestration works better. Good tools and a good communication protocol beat a detailed, carefully tuned pipeline. That simple workflow already does well on frontier benchmarks like LeanEval. So why is this pipeline better? Or does the benefit only hold for small open models, where maybe careful pipelines are still needed? The paper should answer this directly.

Results are not adjusted for effort. Table 1 and Table 2 report no tokens or cost, so the rows cannot be meaningfully compared. The comparison is against one forward pass of an 8B reward model.

Test-time selection is within noise. NF-Bridge ties the best reward model on AIME24, AIME25 and BRUMO25, and beats it by one problem out of 500 on MATH-500. Majority vote ties it on AIME25 and beats it on BRUMO25, and there are no confidence intervals on 30-problem sets where one problem is 3.33 points.

Can we trust the judge? Every faithfulness number comes from an LLM judge with no human check, and the pipeline is refined against Qwen judges in the loop (+9 SF points in Table 5) and then scored by a Qwen judge.

Questions for authors

How does NF-Bridge compare with a generic agent (Codex or OpenCode with Lean and search tools) told to build a blueprint and then formalize it faithfully? Please run this with the same budget, and with both a frontier model and the 27B model. Can you report tokens and cost for every row in Tables 1-3? Can you report bootstrap confidence intervals and results on several candidate pools?

Reviewer_6aXuOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a framework for faithful natural-language-to-Lean proof autoformalization and verification-grounded test-time candidate selection. The method consists of three stages. First, it extracts implicit modeling premises and constructs shared Lean definitions that are frozen throughout the pipeline. Second, it decomposes natural-language reasoning into a proposition DAG and constructs a corresponding Lean DAG, iteratively refining both using compilation and semantic-fidelity feedback. Third, it performs dependency-aware proof and exact negation searches for individual propositions. The resulting verification outcomes are aggregated into a reward for ranking candidate mathematical solutions. The paper evaluates autoformalization on ProofFlowBench, miniF2F, and RobustPABench, and candidate selection on AIME24, AIME25, MATH-500, and BRUMO25. It also trains a specialized 9B verifier using SFT and GRPO on node-level verification trajectories.

Critical strengths
  1. Meaningful improvements in faithful proof autoformalization: The paper demonstrates substantial improvements over previous approaches in semantic faithfulness and joint correctness, particularly on miniF2F. The improvements on RobustPABench's global perturbations are also large and consistent. The preliminary study comparing Holistic-DAG against Uni-DAG is particularly informative: freezing shared mathematical definitions improves independently judged faithfulness from 235/1,000 to 576/1,000 examples. This provides direct motivation for an important design choice.

  2. A principled distinction between verified mathematical evidence and inconclusive proof search: The use of positive and exact-negation proof searches is well motivated. Unlike systems that interpret failure to generate a proof as evidence of incorrectness, NF-Bridge explicitly distinguishes proved, disproved, unresolved, and blocked states. The decision to freeze formal propositions before proof search also helps prevent the prover from silently changing the statement it is supposed to establish. These are valuable design choices for auditable mathematical reasoning.

Critical weaknesses
  1. The practical significance of the test-time selection contribution is insufficiently established: Table 2 shows that NF-Bridge is competitive with learned reward models, but its improvements are small. On MATH-500, the improvement over the strongest learned reward-model result is 0.20 percentage points, or one problem. More importantly, aggregating the four benchmarks gives 503/590 correct selections for NF-Bridge versus 502/590 for majority voting. The additional verification evidence therefore produces only a marginal observed improvement over a much simpler selector on this candidate pool. Besides that, since NF-Bridge requires autoformalization, semantic checking, and potentially expensive proof searches, an end-to-end accuracy vs. computation comparison is needed to establish the practical usefulness of the reward mechanism.

  2. The contribution of individual mechanisms is not sufficiently isolated in the end-to-end results: The paper provides useful ablations of semantic judges and verifier-ranking signals. However, the overall pipeline combines frozen semantic terms, paired DAGs, iterative refinement, specialized prover training, dual verification, and heuristic direction rejection. It remains unclear how much each component contributes to final autoformalization success and downstream selection accuracy. In particular, an end-to-end comparison with and without frozen shared terms under identical model and inference budgets would help establish the causal contribution of the proposed semantic interface. Comparing the pretrained 9B model, the SFT-only checkpoint, and the SFT+GRPO checkpoint would clarify the benefit of the training.

Questions for authors
  1. How accurately do the independent semantic fidelity judges agree with expert mathematical annotations, particularly on incorrect or subtly altered formalizations?

  2. How sensitive is the candidate ranking score to alternative faithful DAG decompositions of the same solution?

  3. What is the end-to-end accuracy/compute tradeoff relative to majority voting and learned reward models on identical candidate pools?

  4. How much do frozen shared terms contribute to downstream joint correctness and candidate selection when the remaining pipeline is controlled?

Reviewer_XgdwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a method to translate natural language proofs into formalized Lean proofs. A lean prover is built based on the translation technique. Specifically, a shared group of definitions, and a DAG of mathematical claims are provided by a strong LLM. After that, the DAG is refined to pass formal verification and a semantic judge. At last, a formal prover solves each claim in the DAG. In cases where there are multiple candidate natural language proofs, the proposed technique can also improve test time scaling by selecting the best proof based on the proportion of proved nodes in the DAG. Empirical results study the translation faithfulness (judged by LLM), the proof success, and candidate selection.

Critical strengths

The faithfulness of translated natural language proofs are significantly improved. On both miniF2F and ProofFlowBench, the semantic faithfulness is improved by more than 20%. The ablation also demonstrates that the shared layer of frozen definitions is the key to faithfulness.

Critical weaknesses

The improvement on end-to-end proof success rate is less impressive than the faithfulness. For example, on ProofFlowBench, the proof acceptance does not significantly improve over the strong baseline ToMap-Gemini. The performance on NL candidate solution selection also compares to reward model. It is unclear whether the improvement in faithfulness translates to improvement in end-to-end prover capability. Additionally, the semantic faithfulness is scored by a LLM Judge, which depends on variations in the judge model capability, and is therefore a less objective metric compared to the end-to-end proof acceptance rate.

Questions for authors

Corresponding to the weakness, can author provide more evidence about the significance of end-to-end proof success?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.