acceptodds
Withdrawn from ICLR 2027

Grounded Latent Reasoning for Knowledge Graph Question Answering

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Knowledge graph question answering (KGQA) aims to identify the entities and relations in a knowledge graph needed to answer a given question. Recent approaches use large language models (LLMs) to generate textual relation paths, but this introduces a discrete vocabulary bottleneck because the relations generated by an LLM do not necessarily match the relation vocabulary of the KG. We propose grounded latent reasoning (LATENTRAG), a framework that performs intermediate reasoning in continuous latent space and directly grounds each latent reasoning step in the KG. Given a question, an LLM first generates a sequence of continuous latent thoughts without decoding intermediate reasoning steps into discrete tokens. Each latent thought is then decoded into scores over the KG’s actual relations, which guide entity propagation through the graph and produce grounded multi-hop reasoning paths. The retrieved paths are subsequently provided to a downstream reasoning module to generate the final answer. This design allows the LLM to reason without committing to textual relation names while ensuring that each graph reasoning step follows an actual KG relation. Experiments on different datasets show the effectiveness of the proposed method.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Knowledge Graph Question Answering

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_x4w1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LatentRAG, a framework that integrates continuous latent reasoning with knowledge graph question answering (KGQA). Instead of generating discrete textual relation paths, the method autoregressively produces latent hidden states and projects them into scores over the KG's relation vocabulary. These scores are used to construct weighted graph transitions and perform multi-hop entity propagation. The model subsequently reconstructs explicit reasoning paths, which are provided to a separate reader LLM for final answer generation.

Critical strengths
  1. An interesting approach to bridging latent reasoning and structured knowledge.

The paper addresses the mismatch between textual relations generated by LLMs and the actual relation vocabulary of a knowledge graph. Mapping latent states directly to KG relation scores is a reasonable way to avoid exact textual relation matching. Moreover, restricting entity propagation to existing KG edges provides explicit structural grounding.

  1. Promising empirical results.

The method achieves strong Hits@1 results on WebQSP and CWQ. The ablation studies also provide evidence that latent reasoning contributes to performance. The framework offers an interesting direction for connecting continuous neural reasoning with explicit graph-based retrieval.

Critical weaknesses

1. Insufficient methodological clarity and justification.

The latent reasoning process does not explicitly incorporate intermediate KG states, raising questions about whether it performs genuinely graph-conditioned reasoning. Moreover, the paper lacks important implementation details, including how multiple relations between the same entity pair are handled, how the reasoning depth is determined, and how per-hop relation supervision is constructed over a large relation vocabulary.

2. Insufficient experimental validation of the core contribution.

The comparisons mainly involve older baselines, with limited coverage of recent KGQA methods. More importantly, the improvement from latent reasoning is relatively small on CWQ (+0.7 Hits@1), while the final reader contributes substantially to overall performance. The reader LLM is not clearly specified, making it difficult to isolate the benefits of the proposed latent reasoning mechanism and assess the fairness of the comparisons.

Questions for authors

Q1. Relation mapping and scalability.
Eq. (5) predicts scores over the entire KG relation vocabulary at every reasoning step, although each question may involve only a small subset of relations. How is the global relation vocabulary constructed, particularly when using question-specific KG subgraphs such as those provided by RoG? Why is it necessary to score all relations rather than only the currently reachable ones? How does this design scale to larger or dynamically changing relation vocabularies?

Q2. Relation supervision and training loss.
How are the per-hop multi-hot relation labels in Eq. (10) constructed? Given the large number of relations and relatively few positive labels per question, how is class imbalance handled? Moreover, when multiple valid reasoning paths exist, how does the method avoid incorrectly treating unannotated but valid relations as negative examples?

Q3. Graph transition matrix construction.
How is the transition matrix in Eq. (6) implemented for large-scale KGs? Is it constructed dynamically for each question-specific subgraph or over the entire KG? Additionally, when multiple relations connect the same pair of entities, how are their relation scores aggregated into a single matrix entry?

Q4. Reader LLM and contribution of latent reasoning.
Which specific LLM is used as the final reader in Section 3.4, and is it fine-tuned? Since the reader substantially improves performance, while the gain from latent reasoning on CWQ is relatively small, how can the authors demonstrate that the improvements primarily originate from the proposed latent reasoning mechanism rather than the reader's own knowledge or reasoning capability?

Q5. Comparison with recent methods.
Why does Table 1 mainly compare against methods published before 2025? Can the authors include more recent KGQA baselines under comparable backbone, KG, and reader settings to provide stronger evidence of the proposed method's effectiveness?

Reviewer_fax2OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes grounded latent reasoning (LATENTRAG) that performs intermediate reasoning in continuous latent space and directly decodes each latent thought into scores over the KG’s actual relation space. The relation scores are used to propagate entity distributions through the graph, keeping the reasoning process grounded in the KG without generating exact textual relation names.

Critical strengths

The paper is easy to follow.

Critical weaknesses

Limited novelty since most of the method is taken from the existing work.

Only two datasets are used for evaluation.

The ablation study ‘without latent reasoning’ is not clearly defined and the effect is small. Hit@1: 85.2 on WebQSP and 62.7 on CWQ are very close to the KG-CoT.

Lack of comparisons with more recent baselines (such as GCR, and Think-on-Graph etc.).

Notation issues and typos. 1) Some notations used are not defined. 2) some notations are defined but not used throughout the paper (such as M).

Questions for authors

What is the novelty or key contribution compared to prior differentiable KG reasoners?

What is the value of T for each dataset? How is it chosen? Is it fixed or adjusted for different questions?

What are pep_e and p^e\hat{p}_e in Eq. (9)?

What does ‘without latent reasoning’ mean in Table 7? Please provide more details on this definition.

Reviewer_fcdTOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper addresses the "discrete vocabulary bottleneck" in LLM-based knowledge graph question answering (KGQA): methods such as RoG ask an LLM to generate textual relation paths, but the generated relation names may not match the relation vocabulary of the KG. The proposed method, LatentRAG (grounded latent reasoning), has the LLM (Llama-2-7B with LoRA) generate a sequence of continuous latent thoughts in the style of Coconut, instead of textual relations. Each latent thought is decoded into scores over the KG's actual relations. These scores weight the existing edges of the KG and are used to propagate an entity distribution from the topic entity, hop by hop. Following KG-CoT, the per-hop distributions are combined with question-dependent weights, and the model is trained end to end with an answer-level loss and an auxiliary per-hop relation loss. Explicit reasoning paths are then reconstructed for the top-scoring candidate entities and passed to a separate LLM reader that generates the final answer.

Experiments on WebQSP and CWQ (Freebase) compare LatentRAG with embedding-based, retrieval-based, LLM-only, and LLM+KG baselines. LatentRAG achieves 88.1 Hits@1 / 74.9 F1 on WebQSP and 63.4 Hits@1 / 56.8 F1 on CWQ, the highest Hits@1 among the listed methods on both datasets. The paper also reports ablations on the final reader, the number of latent embeddings per hop, and the numbers of candidate entities and retained paths. It further includes an appendix comparison with and without latent reasoning, an inference-time comparison with RoG, and case studies.

Critical strengths

1. Grounded and interpretable reasoning. Each latent thought is decoded into scores over the KG's actual relations, and the entity distribution is propagated only along existing edges. As a result, every retrieved path is guaranteed to exist in the KG, and no unmatched relation names are generated. The retrieval module is differentiable and trained end to end, yet it still produces explicit relation paths that can be inspected. The case studies (Table 4) illustrate that, with the same backbone, the retrieved paths are more relevant than those of RoG.

2. Strong overall performance with efficient inference. The full system achieves the highest Hits@1 among the listed methods on both WebQSP (88.1) and CWQ (63.4). Inference is also efficient: the paper reports about 10× faster inference than RoG (0.049 s vs. 0.498 s per question). In addition, the ablation on the number of latent embeddings per hop shows that a single latent embedding per hop is already sufficient.

Critical weaknesses

1. The motivating problem has largely been addressed by prior work. The paper aims to solve the mismatch between relations generated by an LLM and the actual relation vocabulary of the KG (e.g., Freebase). However, this mismatch mainly arises in methods that generate textual relation paths, such as RoG. Many earlier methods, including NSM, TransferNet, and KG-CoT, avoid it by scoring the KG's actual relations directly, and the paper itself acknowledges this for KG-CoT in the related work. The lower accuracy of NSM and TransferNet is mainly due to the absence of an LLM, not to any vocabulary mismatch: once an LLM reader is added, as in KG-CoT, accuracy becomes high. The paper's own Table 2 supports this: without the final LLM reasoning module, the proposed method obtains only 66.0 Hits@1 on WebQSP and 35.0 on CWQ, lower than both NSM and TransferNet.

2. Limited method part. The proposed method essentially follows the KG-CoT pipeline (relation scoring, entity propagation over the KG, hop aggregation, path reconstruction, and an LLM reader) and adds latent reasoning to produce the per-hop representations. Since KG-CoT already avoids the relation-name mismatch, this is an incremental modification of an existing pipeline, or an alternative way to obtain per-hop representations, rather than a solution to an unsolved problem. In my view, this level of novelty does not meet the bar for ICLR.

3. The ablation shows only a small effect of latent reasoning. The only comparison with and without latent reasoning is Table 7 in Appendix C, which is not referenced in the main text and does not describe the variant without latent reasoning. The gains are small: +2.9 Hits@1 on WebQSP and only +0.7 on CWQ. With a single run and no reported variance, the CWQ gain may lie within run-to-run noise. Moreover, the variant without latent reasoning already outperforms all baselines in Table 1 in Hits@1 on both datasets (85.2 on WebQSP and 62.7 on CWQ). Since this variant presumably corresponds to a KG-CoT-style pipeline, most of the improvement over prior work appears to come from components that are not new, rather than from latent reasoning.

Questions for authors

see weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.