Grounded Latent Reasoning for Knowledge Graph Question Answering

Knowledge graph question answering (KGQA) aims to identify the entities and relations in a knowledge graph needed to answer a given question. Recent approaches use large language models (LLMs) to generate textual relation paths, but this introduces a discrete vocabulary bottleneck because the relations generated by an LLM do not necessarily match the relation vocabulary of the KG. We propose grounded latent reasoning (LATENTRAG), a framework that performs intermediate reasoning in continuous latent space and directly grounds each latent reasoning step in the KG. Given a question, an LLM first generates a sequence of continuous latent thoughts without decoding intermediate reasoning steps into discrete tokens. Each latent thought is then decoded into scores over the KG’s actual relations, which guide entity propagation through the graph and produce grounded multi-hop reasoning paths. The retrieved paths are subsequently provided to a downstream reasoning module to generate the final answer. This design allows the LLM to reason without committing to textual relation names while ensuring that each graph reasoning step follows an actual KG relation. Experiments on different datasets show the effectiveness of the proposed method.