acceptodds
Under review as a conference paper at ICLR 2027

Grounding Is Not Enough: Testing Evidence Sufficiency in LLM Reasoning over Knowledge Graphs

Abstract

Grounded large language model (LLM) systems are expected to justify predictions with evidence, making answer correctness alone insufficient. Knowledge graph question answering (KGQA) provides a concrete setting for testing this requirement since returned graph evidence can be checked against the conditions needed to support a prediction. Yet existing KGQA evaluations largely emphasize answer correctness, leaving two questions insufficiently examined: whether the returned evidence sufficiently supports the prediction, and whether systems return Unknown when the available knowledge cannot support an answer. We introduce an executable evaluation protocol that derives constraints from structured queries, verifies whether the returned evidence satisfies those constraints, and uses controlled graph edits to test system behavior after answer-related evidence is removed. We apply this protocol on WebQSP, CWQ, and 2WikiMultiHopQA. Across existing grounded LLM-based KGQA systems, the required supporting evidence is unavailable in up to 76.77% of cases, while systems correctly return Unknown in only 0.00–30.53% of cases where support is unavailable. After the removal of answer-critical evidence, 53.98% of initially correct predictions still persist, whereas only 18.31% transition to Unknown. These gaps persist even with Codex (GPT-6), suggesting that stronger language models alone do not eliminate the problem. Motivated by these findings, we introduce Grounding with Requirement-Aware Shared-binding Preservation (Grasp) as a reference mitigation that preserves question constraints, reducing the EvidenceGap to 27.47%, improving the correct Unknown up to 88.01%. Overall, our results reveal a key limitation of answer-based evaluation for grounded LLMs: it does not capture whether returned evidence sufficiently supports an answer or whether models abstain when such support is unavailable.

Primary area:
datasets and benchmarks
Keywords:
Grounded LLMs, Evidence Sufficiency, Knowledge Graph Question Answering

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_yiQCOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether grounded KGQA systems provide sufficient evidence for their answers, rather than simply producing the correct answer. It proposes an evaluation protocol based on query constraints and support-removal interventions, and evaluates it on WebQSP, CWQ, and 2WikiMultiHopQA. The paper also introduces GRASP to improve evidence grounding and abstention.

Critical strengths
  1. The paper is easy to understand. The main concepts, including evidence sufficiency, binding consistency, and support removal, are explained with clear examples.

  2. The motivation is clear. The paper makes a reasonable case that answer accuracy alone is not enough when a system is expected to answer only from a given knowledge source.

  3. The evaluation design is concrete. Query-derived constraints and controlled graph edits provide a direct way to test evidence sufficiency and behavior after support is removed.

Critical weaknesses
  1. The main observation is close to findings already reported in Sufficient Context (Joren et al., 2025), FaithEval (Ming et al., 2025), and When Silence Is Golden (Zhou et al., 2026). These works have already shown that LLMs may answer when the available evidence is insufficient. The paper should clarify more clearly what new insight is obtained from studying this problem in the KGQA setting.

  2. The design of EVIDENCEGAP is not well justified. It is defined as 1 − E-HIT/HIT@1, but does not account for cases where the model answers from its own parametric knowledge. In other words, a correct answer may still be counted as an evidence gap simply because the relevant fact is not present in the returned evidence. This is especially important for general-domain datasets such as CWQ and WebQSP, since models such as Qwen2.5-7B, Llama-3.1-8B, and GPT-6 may already have learned some of the facts in these datasets. The authors should discuss the evidence boundary of parametric knowledge and provide suitable controls.

  3. The experimental setup is not fully clear, and the fairness of the comparisons is difficult to assess. For example, Table 4 does not provide enough information about the Codex/GPT-6 configuration or prompts. It is also unclear why RoG, VoG, and SubgraphRAG are not included as baselines, although they are discussed in Table 1. In addition, most open-source experiments use 7–9B models. Including models around 13B, as used by methods such as RoG and GCR, would make the comparison more balanced. Given these limitations, the current evidence is not sufficient to fully support the conclusions of RQ1.

  4. The support criterion relies heavily on gold SPARQL queries or annotated evidence. This may judge semantically valid evidence as insufficient when it uses a different schema or relation path. The semantic-relaxation analysis is helpful, but may not fully resolve this issue.

Questions for authors

See above.

Reviewer_s2ztOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper investigates the problem that LLMs may generate correct answers without sufficient supporting evidence. To examine this issue, it derives constraints from gold SPARQL queries or annotated evidence triples, verifies whether the returned evidence satisfies those constraints, and uses controlled graph edits to test system behavior after answer-related evidence is removed. Further, it also introduces Grasp as a reference mitigation approach that preserves question constraints, aiming to improve evidence grounding during answer generation.

Critical strengths

The research question is important and realistic. The executable protocal and the support-removal interventions are well-designed.

Critical weaknesses
  1. As the evaluated datasets CWQ, WebQSP, and Wikidata, primarily contain commonsense factual knowledge, LLMs may be able to generate correct answers by their internal parametric knowledge rather than grounding their responses in retrieved evidence. It would be useful to evaluate with domain-specific KGs that general-purpose LLMs are less familiar with, to enhance reliance on evidence grounding.

  2. The sufficiency judgment is relative to one gold query or one set of annotated triples, so evidence that supports the answer through a different but valid path (for example, an alternative relation chain or a different CVT route) counts as insufficient -- this is a fundamental issue as real-world KG reasoning, including CWQ etc., usually contains alternative reasoning paths to the answer.

  3. The GRASP comparison for RQ3 is less fair, as GRASP uses GPT-OSS-120B both to induce the specification and to answer, while CoT and DoG rely only on 7-9B models.

Questions for authors

It is unclear whether CoT, DoG and GCR* were given an explicit UNKNOWN option and an instruction to abstain, whereas the Codex prompt does require UNKNOWN. If the baselines were not given this option, the near-zero CAR in Table 5 and much of RQ2 are artifacts of the prompts rather than findings about the systems.

Reviewer_SnxbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces evidence-bounded KGQA as a formal setting in which a system must not only predict a correct answer but also return evidence that jointly satisfies all question constraints under binding-consistent assignments, and must return UNKNOWN when sufficient support is unavailable. The experiment results demonstrate the effectiveness of the approach.

Critical strengths
  1. It defines a well-motivated and precisely formalized problem.
  2. It introduces an executable evaluation protocol for grounded LLM systems to assess whether returned evidence is sufficient and whether systems return UNKNOWN when sufficient support is unavailable.
Critical weaknesses
  1. The S/I partition inherits all annotation noise, and the paper acknowledges this only in Limitations.
  2. The GRASP results are promising but the method is a reference mitigation, not a solution — and the paper is appropriately candid about this, perhaps too much so.
  3. The evaluation is confined to KGQA, and the transferability to open-domain RAG is asserted rather than demonstrated.
  4. The support-removal analysis is restricted to CWQ and to instances initially answered correctly with sufficient evidence by all three systems. 5.The CoT baseline is a weak foil for the "grounded systems" claim.
Questions for authors

Refer to the weaknesses mentioned above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.