acceptodds
Under review as a conference paper at ICLR 2027

Let Solvers Reason and Models Understand: Proof-Obligation-Guided Bridging

Abstract

Neuro-symbolic approaches aim to combine the flexible language understanding of neural models with the rigorous reasoning of symbolic solvers. These approaches typically use a large language model (LLM) to translate the problem expressed in natural language (NL) into formal representations and a symbolic solver to reason over these representations. However, even when individual NL statements are faithfully formalized, their semantically related expressions may still be mapped to symbolically unrelated predicates, leading to failures in downstream symbolic reasoning. To overcome this challenge, we propose Proof-Obligation-Guided Bridging (POGB). During autoformalization, POGB maintains a grounding table that records the NL semantic grounding for each formal element. When the solver encounters an unresolved proof obligation, POGB identifies candidate semantic bridges and uses an LLM to evaluate them against the recorded grounding information, thereby recovering semantic relations missing from the formal representations. This design keeps all logical reasoning under the solver's control while restricting the LLM's role to autoformalization and the evaluation of candidate semantic bridges. By letting solvers reason and models understand, POGB leverages their complementary strengths to advance neuro-symbolic integration. Theoretical analysis proves the soundness of POGB, and extensive experiments on diverse logical reasoning benchmarks demonstrate its superior performance.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
neuro-symbolism, formal representation, logical reasoning, semantic bridging, autoformalization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_fhW7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The main contribution of this paper is to improve the robustness of combining LLMs with formal reasoning systems in neuro-symbolic systems, by having the LLM judge whether an entailment relation (including commonsense relations) holds between semantically related predicates after formalization. The method works backward: starting from the goal to be proved, it traces back to where the proof gets stuck and has the LLM judge and resolve those points. The authors report performance on five logical reasoning benchmarks with four LLMs, and ablate the effectiveness of the goal-directed backward search, as well as of the the grounding table (type information plus natural-language templates).

Critical strengths
  • The problem the paper addresses is real and clearly described. It is a problem the neuro-symbolic field genuinely needs to solve, and the proposed approach is easy to follow.
  • The ablation studies demonstrate the effectiveness of the grounding table and the goal-directed backward search, with clear accuracy gains.
Critical weaknesses
  • The abstract and introduction frame the contribution around the observation that semantically related expressions may be mapped to symbolically unrelated predicates. This problem, however, is already the target of several existing works and does not by itself distinguish this paper. SymBa (Lee & Hwang, NAACL 2025) likewise uses a solver-controlled backward proof and invokes the LLM only when the proof stalls. LLM-TRes (Toroghi et al., EMNLP 2024) and LLM-TH (Toroghi et al., ICLR 2025) use an entailment model during proof search to connect symbolically mismatched predicates, and LLM-TH further uses type information to prune the search space. ARGOS (Cotnareanu et al., ICLR 2026) augments the solver with commonsense supplied by the LLM. SymBa and LLM-TRes are cited, but the paper does not explain how it differs from them; LLM-TH and ARGOS are not cited at all. Consequently, it is unclear to readers what the actual novelty of this paper is, and the contribution needs to be articulated more precisely. Moreover, the baselines are all fairly dated; could more recent ones be included?

  • The cost and reliability of the LLM Gate are not sufficiently analyzed. Regarding cost, I suggest reporting the average number of Gate calls and the number of bridges used per problem. Regarding reliability, additional evidence would be valuable, such as the precision of accepted bridges and the proportion of gold-UNKNOWN instances that are incorrectly proved.

  • Certain figures are difficult to follow. In Figure 2, for example, symbols such as b1\mathfrak{b}_1 and b2\mathfrak{b}_2 cannot be understood from the figure alone. I encourage the authors to check and improve these figures.

Questions for authors
  1. Could the authors include some more recent baselines? The baselines in the main table seem fairly dated to me.
  2. Could the authors provide further analysis of the cost and of the reliability of the LLM Gate?
  3. Could the authors state more clearly how this work differs from related works such as SymBa, LLM-TRes/LLM-TH, and ARGOS, so as to make their own contributions clearer?
Reviewer_snP3OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the issue of missing semantic relationships in neuro-symbolic reasoning. Even when an autoformalization process faithfully translates individual natural language statements, it may map semantically related expressions to predicates that are logically disconnected, thereby preventing the symbolic solver from completing the intended proof.

To tackle this, the paper proposes a framework called Proof-Obligation-Guided Bridging (POGB). This framework employs a Grounding Table to record the type information and natural-language semantic grounding of each formal element. When an unresolved proof obligation arises during proof search, POGB constructs candidate semantic bridges from type-compatible premise instantiations, ranks and filters them using semantic similarity and natural language inference (NLI), and employs logical abduction to derive residual proof obligations. These bridges are then evaluated using an LLM-based Gate, while a symbolic solver handles the remaining proof obligations and verifies the final formalized proof.

Experiments conducted across four different LLMs demonstrate that this approach improves average answer accuracy on four benchmark datasets and enhances the average Verified Proof Rate (VPR) across five benchmarks. Ablation studies further analyze the individual contributions of semantic grounding, semantic bridging, and the Chain-of-Thought (CoT) fallback mechanism.

Critical strengths
  1. The paper clearly explains why locally faithful formal translations may still fail to preserve the logical relationships necessary for composing proofs. Figure 1 provides an intuitive illustration of this problem, using examples of predicate mismatches and missing commonsense knowledge.

  2. Leveraging unresolved proof obligations to identify local semantic gaps is a reasonable design. In this framework, the Grounding Table provides semantic context and type constraints, while the symbolic solver governs the proof search and verification process.

  3. POGB consistently improves average accuracy across all four LLMs tested, with particularly notable performance gains on the ProverQA task. The VPR evaluation further assesses whether the generated formal proofs pass machine verification and support the ground-truth labels. Furthermore, ablation studies and case analyses provide additional insight into the operational mechanisms of the framework.

Critical weaknesses
  1. POGB incorporates Gate-accepted semantic bridges as additional premises in the formal proof. This approach may carry risks similar to those found in existing LLM-based proof-repair methods, namely the introduction of assumptions that lack sufficient justification. Moreover, the paper does not independently evaluate the additional contribution of the LLM-based Gate beyond the NLI filtering stage.

  2. POGB employs recursive search, candidate filtering, and potentially multiple LLM Gate evaluations; however, the paper lacks a comprehensive comparison of computational costs against the baseline methods. While each baseline follows its own standard reasoning workflow, their actual computational budgets may differ significantly. Without comparative experiments conducted under a fixed computational budget, it is difficult to determine whether performance improvements stem from the proposed design or from the additional inference-time computation.

  3. The ablation study in Table 3 compares GRD + LLM-CMP with GRD + BDG. However, this comparison replaces the overall bridging procedure with LLM-based completion, rather than isolating proof-obligation guidance as a single experimental variable. Consequently, it remains unclear whether the performance gains are truly attributable to Proof-Obligation Guidance or to other variations in the bridge-generation process. More rigorous, controlled ablation experiments would help clarify the independent contribution of this specific mechanism.

Questions for authors
  1. How do the results—in terms of accuracy, VPR, and the precision of accepted semantic bridges—differ when using NLI alone versus a combination of NLI and the LLM Gate? Furthermore, does this local semantic evaluation effectively reduce the introduction of poorly justified assumptions compared to existing LLM-based proof-repair methods?

  2. Could the authors report the average number of LLM calls, token consumption, inference latency, and candidate bridges per instance for POGB and the baselines?

  3. What are the specific implementation details of LLM-CMP? Could the authors provide a controlled comparison against pre-proof bridging or unguided bridging (i.e., bridging without proof-obligation guidance) to better isolate the contribution of proof-obligation guidance?

  4. Could the authors provide confidence intervals or paired significance tests for the primary experimental comparisons, and assess the stochastic variability in the inference results by conducting repeated experimental runs?

Reviewer_iE9rOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper proposes Proof-Obligation-Guided Bridging (POGB) to address failures in symbolic reasoning caused by missing semantic relations during natural-language autoformalization. POGB uses a grounding table to preserve the natural-language meanings of formal symbols. During proof search, an LLM evaluates local candidate semantic bridges, while a symbolic solver performs the subsequent derivations. Experiments across several logical reasoning benchmarks and four LLMs show improvements in both accuracy and verified proof rate.

Critical strengths
  1. Clear motivation and a well-designed approach. The paper addresses missing semantic relations after autoformalization, a practical limitation of existing neuro-symbolic frameworks. Combining the grounding table with proof-time bridging provides a sensible separation between local semantic judgments and symbolic reasoning.

  2. Convincing empirical results. POGB improves performance across four LLMs and multiple benchmarks. The ablation studies support the contributions of its main components, while the verified proof rate evaluation provides further evidence of its value.

Critical weaknesses
  1. Limited direct evaluation of bridge reliability. Appendix E discusses Gate overacceptance, but the paper does not systematically assess the semantic validity of accepted bridges. Moreover, the soundness guarantee applies to formal proofs under the added bridges; it does not establish that those bridges faithfully reflect the original natural-language premises.

  2. Incomplete comparisons and efficiency evaluation. The baselines primarily cover earlier approaches, without direct comparisons against more recent relevant methods such as SymBa (Lee & Hwang, NAACL 2025) and CLOVER (Ryu et al., ICLR 2025). Although the paper reports some relative changes in search overhead, a fuller account of inference cost and budget-matched comparisons would strengthen the evaluation. Moreover, Table 3 shows that without the CoT fallback the formal pipeline (GRD + BDG) reaches 81.86% on FOLIO, below plain CoT (83.82%), so the FOLIO gain of the full system depends substantially on falling back to CoT; the main results do not make this clear.

Questions for authors
  1. Could the authors manually assess a sample of accepted bridges and report their semantic validity? Does the system check whether the added bridges are logically consistent with the original premises?

  2. Could the authors provide comparisons with recent methods under comparable settings, or clarify the key methodological differences? Reporting inference time and LLM call costs would also be helpful. Please also report, for each benchmark, how often the final answer comes from the CoT fallback rather than from a verified proof.

  3. Algorithm 1 may return FAILED when a search limit is reached, while Section 3.1 maps failure to prove both a proposition and its negation to UNKNOWN. How does the system distinguish incomplete search from cases where neither conclusion follows under open-world semantics?

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response from the Authors

We sincerely thank the reviewers for their valuable feedback and constructive suggestions. In response to their insightful comments, we have carefully revised the manuscript, with particular emphasis on technical details, experimental evaluations, and the corresponding discussions and analyses. The main revisions are summarized as follows:

  • We have explicitly clarified the distinctions between existing approaches and our proposed method in the Introduction and Related Work sections. The fundamental difference lies in how solver feedback guides the LLM and which tasks are delegated to it (Reviewer iE9r, W2 & Q2; Reviewer fhW7, W1 & Q3).
  • We have extended and strengthened our experimental evaluations by adding more recent baselines in the Extended Main Results section. The results demonstrate that POGB achieves the highest accuracy across all 16 LLM–dataset combinations (Reviewer iE9r, W2 & Q2; Reviewer fhW7, W1 & Q1).
  • We have conducted a comprehensive efficiency evaluation and cost analysis in the Bridge Reliability and Computational Efficiency section. The results demonstrate that POGB requires fewer LLM calls than most baselines while maintaining reasonable token consumption and inference time (Reviewer iE9r, W2 & Q2; Reviewer snP3, W2 & Q2; Reviewer fhW7, W2 & Q2).
  • We have manually evaluated the accepted bridges and analyzed their reliability in the Bridge Reliability and Computational Efficiency section. The results demonstrate high bridge reliability and a substantially higher rate of correct proofs (Reviewer iE9r, W1 & Q1; Reviewer snP3, W1 & Q1; Reviewer fhW7, W2 & Q2).
  • We have added an ablation experiment comparing NLI-only filtering with NLI filtering followed by the LLM-based Gate evaluation (Reviewer snP3, W1 & Q1). We have also clarified the implementation details of LLM-CMP and introduced an unguided-bridging ablation study. The results demonstrate that proof-obligation guidance substantially improves accuracy while reducing the number of LLM calls (Reviewer snP3, W3 & Q3).
  • We have explicitly reported the CoT fallback rates in the Main Results section and further clarified the advantages of the formal pipeline over CoT in the Ablation Studies section (Reviewer iE9r, W2 & Q2).
  • We have provided confidence intervals and assessed stochastic variability through five independent experimental runs (Reviewer snP3, Q4).
  • We have improved the presentation of Figure 2 (Reviewer fhW7, W3), explicitly shown the logical consistency check in Algorithm 1 (Reviewer iE9r, Q3), and addressed other presentation-related issues.

We look forward to further discussion with the reviewers and would be happy to address any additional questions or comments to further improve the manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.