acceptodds
Under review as a conference paper at ICLR 2027

BridgeKG-RAG: Bridging Wording Gaps in Knowledge-Graph RAG

Abstract

Knowledge-graph retrieval-augmented generation (KG-RAG) answers questions by linking phrases in the question to entities in a graph of document-grounded facts and retrieving evidence from their neighborhoods. This works when the question uses the same or similar names under which the relevant evidence is organized. However, users may instead mention a near-synonym, a loosely related term, or even a different kind of entity, while the evidence needed to answer is organized under an entity the question never names. We call this the wording gap. The entity mentioned in the question may be a valid graph entity, yet retrieval from its neighborhood can still miss the answer-bearing entity. Neither string similarity nor query rewriting necessarily resolves this mismatch, because the two entities may refer to different things, while traversal over factual edges cannot recover a connection that is absent from the graph. We propose BridgeKG-RAG, which augments the factual graph with typed bridge edges that capture wording relationships between entities. These bridges are used only to expand retrieval starting points and are never treated as answer evidence. Given a question, BridgeKG-RAG selects a small number of bridges to traverse, adds the reached entities to the retrieval set, and then retrieves document-grounded evidence from the factual graph. If the question already names the answer-bearing entity, retrieval remains unchanged. On GapQA, a wording-gap benchmark constructed from Doc2Dial, MultiDoc2Dial, and PrivacyQA, our proposed BridgeKG-RAG substantially improves evidence recall and answer quality.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Knowledge-Graph RAG, Retrieval-Augmented Generation, Wording Gap

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7vboOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on solving the wording gap problem in GraphRAG, which the entity mentioned in the query may be different in format from the entity in the KG. The authors propose BridgeKG-RAG, which augments the factual graph with typed bridge edges that capture wording relationships between entities. BridgeKG-RAG contains two stages. In the Bridge Construction stage, the framework first a pool of candidate entity pairs based on name similarity and document co-occurrence, and represents each pair with multi-dimensional similarity vector that estimates the similarity between entity names, contexts, role types, neighbors of the entities, relation overlaps, and their co-existence within the same section or documents. It then applies K-Means algorithm to cluster these vectors, and utilizes an LLM to interpret each cluster centroid to either reuse a type from a shared catalog (e.g., alias, abbreviation) or to define a new corpus-specific wording type. The resulting bridge types form a corpus skill. Finally, an LLM labels each candidate pair using its similarity vector and the skill as soft guidance, outputting a bridge type, direction, and confidence score, those with no meaningful relation or with a confidence score lower than the threshold will be discarded. In the traversal and retrieval stage, the system first determines whether the entities mentioned in the original query already include the one where the relevant evidence is organized. If yes, the LLM directly resolve the query with RAG, otherwise, otherwise, it follows the top-k highest-scoring bridges of predicted types to expand the query entities, retrieves query-relevant facts from the expanded neighborhood, and generates the answer grounded solely in their source document spans.

Critical strengths
  1. The paper is clearly written and in general easy to follow.

  2. This paper identifies and resolves the limitation of existing GraphRAG system, which the entity mentions in the query may be vague or differ from the one within the KG. The solution has its practical value.

  3. The number of bridge types within Eb\mathcal{E}^b is not fixed, which the type inventory can adapt to each corpus, resulting in personalized types. This allows us to strike a balance between a fixed type inventory and unconstrained per-pair definition.

  4. Different from a single dense similarity score, the multi-dimensional vector keeps lexical, contextual, graph-structural, and document-structural signals disentangled and interpretable. This allows an LLM to interpret cluster prototypes as wording types, and captures relations such as abbreviations or nicknames that may be distant in embedding space but share neighbors or source documents.

Critical weaknesses
  1. The traversal score c is only used to select the top-k bridges. Once an entity enters S(q), it is treated equally with other entities during retrieval. It would be interesting to see whether propagating c into fact ranking, for example, down-weighting facts reached through low-confidence bridges, could further reduce off-topic evidence.

  2. The proposed method is not compared with more advanced knowledge graph RAG methods, such as PoG [1], ToG2 [2], HippoRAG2 [3], GFM-RAG [4], and G-Reasoner [5]. Therefore, it is unclear whether those advanced solutions also suffer from the same issue.

  3. The proposed method and baselines are only evaluated on the GapQA dataset constructed in the same work and proposed in the same paper. It is unclear which LLM is used to construct the wording-gap variant for the answer-bearing entity and for candidate question rewrite. The paper also lacks sufficient quality control measures for the newly constructed dataset. Therefore, based on existing clues, one is not able to conclude that the proposed method has desirable generality.

[1] Chen et al., Plan-on-Graph: Self-Correcting Adaptive Planning of Large Language Model on Knowledge Graphs (Neurips 2024)

[2] Ma et al., Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation (ICLR 2025)

[3] Gutiérrez et al., From RAG to Memory: Non-Parametric Continual Learning for Large Language Models (ICML 2025)

[4] Luo et al., GFM-RAG: Graph Foundation Model for Retrieval Augmented Generation (Neurips 2025)

[5] Luo et al., G-reasoner: Foundation Models for Unified Reasoning over Graph-structured Knowledge (ICLR 2026)

Questions for authors

The soundness issue needs to be resolved (please refer to W2 and W3).

6 comments
AuthorsOpenReview ↗

Response 1

We sincerely thank you for your insightful feedback. We have added a confidence weighting experiment, compared with stronger graph retrieval baselines, and provided further evidence of generalizability and evaluation reliability.

The traversal score c is only used to select the top-k bridges. Once an entity enters S(q), it is treated equally with other entities during retrieval. It would be interesting to see whether propagating c into fact ranking, for example, down-weighting facts reached through low-confidence bridges, could further reduce off-topic evidence.

Thank you for the insightful suggestion to propagate bridge confidence into evidence ranking. Currently, confidence c determines which bridges are traversed under the top-k budget. Once their destination entities enter S(q), the factual evidence is selected by query relevance, not multiplied by bridge confidence. We agree that propagating c into evidence ranking is a meaningful alternative, but note that confidence measures the plausibility of a wording transition, not necessarily the relevance of every fact near the reached entity.

We therefore test "Confidence-weighted Fact Ranking": the relevance score of a fact reached through a bridge is multiplied by its bridge confidence, while facts retrieved directly from question-linked entities retain their original scores. The retrieval and answer-generation pipeline is otherwise unchanged.

The following table presents results.

MethodDoc RecallDoc F1Doc Acc.Multi RecallMulti F1Multi Acc.Privacy RecallPrivacy F1Privacy Acc.
BridgeKG-RAG0.66400.43090.67400.64600.42640.66000.74710.38310.4864
Confidence-weighted Fact Ranking0.66000.43160.67600.64200.42710.66200.74320.38400.4903

Confidence-weighted fact ranking yields broadly comparable overall performance, with small trade-offs among the evaluation metrics. It may reduce off-topic evidence from uncertain bridges while also suppressing relevant facts reached through less confident wording transitions. We will report this variant as an optional extension.

AuthorsOpenReview ↗

Response 2

The proposed method is not compared with more advanced knowledge graph RAG methods, such as PoG [1], ToG2 [2], HippoRAG2 [3], GFM-RAG [4], and G-Reasoner [5]. Therefore, it is unclear whether those advanced solutions also suffer from the same issue.

Thank you for identifying these important advanced KG-RAG baselines. We evaluate PoG, ToG-2, HippoRAG 2, GFM-RAG, and G-Reasoner, together with HippoRAG and LinearRAG, on the same GapQA splits. Following Appendix B, the evaluation uses evidence-span recall, token-level F1, and LLM-scored answer accuracy. BridgeKG-RAG uses GPT-4o and the default bridge traversal budget of k=5, with fixed query-linked entity starting points for applicable graph comparisons. The baselines retain their method-specific implementations and native retrieval budgets, since bridge traversals, graph hops, and retrieved passages are not identical units.

The following table presents the results under this common benchmark and answer evaluation protocol.

MethodDoc RecallDoc F1Doc Acc.Multi RecallMulti F1Multi Acc.Privacy RecallPrivacy F1Privacy Acc.
HippoRAG0.3440.3860.5140.2640.4070.5720.4840.3450.383
LinearRAG0.3920.4060.5640.5120.4220.6440.6140.3570.462
PoG0.4320.4080.6340.3940.4200.6540.6420.3530.473
ToG-20.3520.4200.5040.3340.4020.5840.5020.3340.392
HippoRAG 20.4440.4130.5320.5340.4230.6520.4430.3560.453
GFM-RAG0.2940.3800.5240.2840.4180.6040.5130.3210.334
G-Reasoner0.2840.3720.4340.2820.3760.5540.2430.2920.283
BridgeKG-RAG0.6640.4310.6740.6460.4260.6600.7470.3830.486

The added baselines cover graph propagation, multi-hop exploration, association retrieval, and graph-based reasoning. BridgeKG-RAG achieves the best overall results across the reported evaluation metrics. These findings support modeling wording relationships explicitly when the entity named in a question differs from the entity organizing the relevant evidence.

Typed bridges directly address that absent wording relationship and serve only to change retrieval starting points. The final answer still uses source-grounded factual spans. The comparison thus supports a distinct modeling objective rather than claiming that advanced graph reasoning is intrinsically ineffective.

AuthorsOpenReview ↗

Response 3

The proposed method and baselines are only evaluated on the GapQA dataset constructed in the same work and proposed in the same paper. It is unclear which LLM is used to construct the wording-gap variant for the answer-bearing entity and for candidate question rewrite. The paper also lacks sufficient quality control measures for the newly constructed dataset. Therefore, based on existing clues, one is not able to conclude that the proposed method has desirable generality.

Thank you for raising these important concerns regarding the construction quality of GapQA and the generalizability of our method. We appreciate the opportunity to clarify the benchmark construction and validation process and to provide additional experimental evidence supporting the effectiveness of BridgeKG-RAG beyond the constructed benchmark.

First, GapQA follows a human-driven construction and validation process that is independent of BridgeKG-RAG. As described in Appendix C, human annotators identify the original answer-bearing entity, manually select a plausible alternative entity that a user might naturally mention, and formulate a question that preserves the original information need while introducing a wording gap. GPT-4o is used only to improve the fluency and naturalness of the manually constructed questions. It does not determine entity substitutions or change the intended meaning. Each rewritten question is then independently reviewed by multiple human annotators to verify its naturalness, semantic consistency, answer preservation, and evidence sufficiency. Only samples receiving unanimous approval are retained.

Moreover, GapQA questions are finalized before bridge construction. No predicted bridge edges, bridge types, confidence scores, or traversal outcomes are used during question generation or filtering. This ensures that the benchmark construction process is independent of the proposed method's bridge predictions and retrieval decisions.

Second, we conduct an additional evaluation to examine the linguistic realism and semantic reliability of GapQA. Using private, anonymized real-user questions as references, we evaluate three complementary dimensions. Linguistic style similarity measures whether GapQA questions resemble natural user expressions. Ambiguity style similarity examines whether indirect references and underspecified terminology resemble those found in real queries. Information intent recoverability assesses whether rewritten questions preserve sufficient information to recover their original information needs. The first two dimensions are evaluated against domain-matched real-user questions, while the third measures consistency between the original and rewritten questions.

DatasetLinguistic Style (%)Ambiguity Style (%)Intent Recoverability (%)Overall (%)
GapQA-Doc94.291.894.693.5
GapQA-Multi93.692.494.293.4
GapQA-Privacy92.890.693.392.2
Macro Average93.591.694.093.0

The results show consistently strong performance across all three GapQA subsets in terms of linguistic realism, ambiguity style similarity, and intent recoverability. Together with the human validation process, these findings further support the quality and reliability of GapQA.

Third, we provide an independent human evaluation of answer correctness, as reported in Appendix E.2. We randomly sample 100 questions from each GapQA subset and ask human evaluators to assess generated answers without knowing which method produced them. This blinded evaluation provides an independent assessment of answer quality without relying on GPT-4o as the evaluator.

DatasetKG-RAG (no bridging)BridgeKG-RAG
GapQA-Doc50%66%
GapQA-Multi52%68%
GapQA-Privacy38%47%

The results show that BridgeKG-RAG consistently outperforms KG-RAG across all three datasets under independent human evaluation. These findings demonstrate that the improvements observed in automatic evaluations are also reflected in human judgments of answer correctness, further strengthening the reliability of our experimental results.

Reviewer_aGzaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes BridgeKG-RAG to address the wording gap in knowledge-graph retrieval, where the entities mentioned in a question differ from those associated with the relevant evidence. The method introduces typed bridge edges that connect related entities based on their names, contexts, semantic roles, and graph structures. These bridges are constructed using K-means clustering and LLM-based labeling, then used to expand retrieval starting points while keeping answer generation grounded in the original factual graph. Experiments on GapQA generated by authors, a benchmark derived from Doc2Dial, MultiDoc2Dial, and PrivacyQA, show substantial improvements in evidence recall and answer accuracy over the evaluated baselines. However, the results rely mainly on synthetically rewritten questions, leaving the method's effectiveness on real-world queries and its advantages over existing semantic retrieval approaches less clear.

Critical strengths

1. Clear problem motivation and a practical retrieval design.

The paper identifies a relevant limitation of KG-RAG: the entity mentioned in a user's question may differ from the entity under which the answer is stored. The proposed typed bridges offer a reasonable way to address this mismatch. Importantly, bridge edges are used only to guide retrieval, while answers remain grounded in the original document evidence. This separation is a useful aspect of the approach.

2. Strong improvements in evidence retrieval.

BridgeKG-RAG achieves substantial gains in evidence recall and answer accuracy across all three GapQA subsets. The ablation studies show that both personalized bridge types and type-guided traversal contribute to these improvements. The additional human evaluation also supports the main findings, providing further evidence that the proposed approach is effective for the wording-gap scenarios studied in the paper.

Critical weaknesses

1. Limited generalizability of the experimental results.

The evaluation relies primarily on GapQA, a benchmark constructed by rewriting existing questions to introduce wording gaps. While BridgeKG-RAG performs well on this benchmark, its improvements on the original, unmodified questions are much smaller. The paper also lacks an independent evaluation of whether the rewritten questions preserve their original meaning and remain natural and unambiguous. As a result, it is unclear how well the method would generalize to naturally occurring user queries.

2. Insufficient comparison experiments with related approaches.

The proposed framework mainly combines existing techniques, including similarity-based entity pairing, K-means clustering, LLM-based relation labeling, and graph-based retrieval expansion. Although the distinction between bridge edges and factual edges is useful, the paper does not sufficiently establish its advantages over recent semantic bridging methods, such as LinearRAG, or simpler LLM-based entity expansion approaches. In addition, the lack of direct evaluations of bridge quality and the clustering component makes it difficult to determine which parts of the proposed framework are necessary for its performance gains.

Questions for authors
  1. Validity and generalizability of GapQA (Clarification and additional evidence request). How did the authors verify that the rewritten questions preserve the original information needs and remain natural and unambiguous? Given the much smaller improvements on the original datasets, could the authors provide results on naturally occurring wording-gap questions? This would help establish whether the gains extend beyond the constructed benchmark.

  2. Effectiveness of bridge construction (Clarification and additional evidence request). Could the authors report the accuracy of the generated bridge edges and their predicted relation types? Additionally, how would the method perform if bridge types were assigned directly by an LLM without the K-means clustering stage? These results would help determine whether the proposed type discovery mechanism provides a meaningful benefit.

  3. More comparison experiments with related semantic retrieval methods. Most of the baseline experiments cited in the paper from 2024 or earlier, the authors should compare with more recent semantic-based retrieval methods like LinearRAG (ICLR 2026) and LLM-based Direct Entity Expansion(Is it possible to have a large language model directly identify potentially relevant entities and establish connections based on supporting evidence? How effective is this approach?)

5 comments
AuthorsOpenReview ↗

Response 1

We sincerely thank you for your thoughtful and constructive comments. Your suggestions have helped us strengthen the evidence for generalizability, bridge quality, clustering, and comparisons with recent retrieval methods.

Validity and generalizability of GapQA (Clarification and additional evidence request). How did the authors verify that the rewritten questions preserve the original information needs and remain natural and unambiguous? Given the much smaller improvements on the original datasets, could the authors provide results on naturally occurring wording-gap questions? This would help establish whether the gains extend beyond the constructed benchmark.

Thank you for emphasizing the importance of benchmark validity and generalizability. We would like to clarify how GapQA preserves the original information needs and maintains natural question formulations. We also provide additional experimental evidence from a separate private, anonymized internal dataset containing naturally occurring wording-gap questions.

GapQA Construction and Validation. GapQA follows a human-driven construction and validation process. Human annotators identify the original answer-bearing entity, manually select a plausible alternative entity, and formulate a question that preserves the original information need while introducing a wording gap. GPT-4o is used only to improve the fluency and naturalness of the manually constructed questions, without changing the intended meaning or entity substitution. Each rewritten question is independently reviewed by multiple human annotators for naturalness, semantic consistency, answer preservation, and evidence sufficiency. Only samples receiving unanimous approval are retained.

To further assess the realism and semantic reliability of GapQA, we conduct an additional GPT-4o evaluation using domain-matched, anonymized real-user questions as references. The evaluation considers linguistic style similarity, ambiguity style similarity, and information intent recoverability. The first two dimensions assess whether GapQA resembles natural user expressions and indirect references, while the third examines whether the original information need remains recoverable after rewriting.

DatasetLinguistic Style (%)Ambiguity Style (%)Intent Recoverability (%)Overall (%)
GapQA-Doc94.291.894.693.5
GapQA-Multi93.692.494.293.4
GapQA-Privacy92.890.693.392.2
Macro Average93.591.694.093.0

The results show consistently strong linguistic realism, ambiguity-style similarity, and intent recoverability across all three GapQA subsets. Together with the human validation process, these findings support the quality of GapQA and indicate that the introduced wording gaps preserve meaningful and natural information needs.

Generalizability to Naturally Occurring Wording Gaps. We additionally evaluate naturally occurring wording-gap questions from a private, anonymized internal dataset containing 1,000 customer inquiries collected from real service interactions. We select inquiries previously flagged as ambiguous or underspecified and identify cases where the user-mentioned entity differs from the entity associated with the supporting evidence. KG-RAG without bridging and BridgeKG-RAG are evaluated on the same questions under consistent settings.

MethodRecallF1Accuracy
KG-RAG (no bridging)0.3420.3940.516
BridgeKG-RAG0.6180.4290.652

BridgeKG-RAG achieves better overall performance across all reported metrics on the internal customer inquiries. These findings provide additional supporting evidence of its applicability to naturally occurring wording gaps in real user questions.

Evaluation on Original Datasets. We also report results on the original, unmodified Doc2Dial, MultiDoc2Dial, and PrivacyQA datasets, as presented in Table 7 of Appendix E. These evaluations use the original questions without introducing additional wording gaps.

DatasetMethodRecallF1Accuracy
Doc2DialKG-RAG (no bridging)0.63800.32840.6320
BridgeKG-RAG0.65200.33120.6400
MultiDoc2DialKG-RAG (no bridging)0.69600.33000.6540
BridgeKG-RAG0.70800.33310.6620
PrivacyQAKG-RAG (no bridging)0.34630.23340.3346
BridgeKG-RAG0.36110.23860.3424
AuthorsOpenReview ↗

Response 1 (continue)

BridgeKG-RAG performs better across all reported metrics on the original datasets. The smaller gains are consistent with these datasets including questions that do not require a change in retrieval starting point. The internal customer-query experiment complements this broader evaluation.

Taken together, the human validation process and additional realism evaluation support the validity of GapQA, while the experiments on naturally occurring wording-gap questions and original unmodified datasets provide complementary supporting evidence for generalizability beyond the constructed benchmark. We sincerely appreciate your thoughtful feedback and hope these additional results and clarifications address your concerns.

AuthorsOpenReview ↗

Response 2

Effectiveness of bridge construction (Clarification and additional evidence request). Could the authors report the accuracy of the generated bridge edges and their predicted relation types? Additionally, how would the method perform if bridge types were assigned directly by an LLM without the K-means clustering stage? These results would help determine whether the proposed type discovery mechanism provides a meaningful benefit.

Thank you for suggesting both a bridge quality assessment and an ablation without K-means. We provide both a bridge-label quality evaluation and the requested controlled ablation of corpus-adaptive type discovery. The results test two separate claims: whether predicted bridges are plausible, and whether clustering improves retrieval beyond direct GPT-4o pair labeling.

For bridge quality, GPT-4o is shown candidate entity pairs, their context, and the bridge type definitions without seeing the predicted labels or confidence scores. It judges whether each edge encodes a plausible wording relationship, which gives edge precision, and whether the predicted type is appropriate for valid edges, which gives type accuracy. The judge is blinded to the method's predictions, but it is not model independent because GPT-4o is also used during bridge construction. We therefore report these numbers as automatic bridge quality estimates rather than human ground truth.

Independent evaluation. We also report the blinded human answer audit already conducted in Appendix E.2 of the paper. Human evaluators assessed answer correctness on 100 sampled questions from each GapQA split without knowing which system generated an answer. BridgeKG-RAG achieved 66% versus 50% for KG-RAG on GapQA-Doc, 68% versus 52% on GapQA-Multi, and 47% versus 38% on GapQA-Privacy. These human judgments independently support the downstream answer quality improvements, and they do not rely on GPT-4o as the answer judge. This audit does not directly assess bridge edge validity or relation type labeling, so it should not be interpreted as an independent human verification of the edge precision and type accuracy figures. We will make this distinction explicit in the revised paper. The following table reports the GPT-4o evaluation results for bridge edge precision and relation type accuracy.

DatasetEdge Precision (%)Type Accuracy (%)
GapQA-Doc89.486.2
GapQA-Multi87.884.6
GapQA-Privacy91.288.1
Macro Average89.586.3

For the "w/o K-means" control, we remove cluster discovery, corpus-specific type descriptions, and prototypes, and ask GPT-4o to assign bridge types directly to each candidate pair. Confidence filtering, query-aware traversal, budget, evidence retrieval, and answer generation are unchanged.

The following table provides results for this ablation, alongside the results already reported in Table 2.

MethodDoc RecallDoc F1Doc Acc.Multi RecallMulti F1Multi Acc.Privacy RecallPrivacy F1Privacy Acc.
w/o K-means0.55800.41950.63200.54200.41680.62600.67320.36840.4514
BridgeKG-RAG0.66400.43090.67400.64600.42640.66000.74710.38310.4864

An explanation is that isolated LLM pair labels are less stable when the names of related entities differ sharply. Clustering organizes repeated combinations of context, role, graph neighborhood, and document signals into interpretable prototypes. Consider the paper’s library book → lost-item replacement fee example: lexical similarity alone is low, whereas an "item-to-rule" wording type can identify a useful retrieval transition. Crucially, K-means supplies reusable corpus-level patterns rather than imposing a cluster label as the bridge type. Pair-specific LLM decisions can still reject a bridge, assign direction, and score confidence. This combines cross-pair consistency with local flexibility, explaining why clustering is more than a preprocessing convenience.

Reviewer_yprCOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper addresses the "wording gap" in KG-RAG: the user names an entity that is related to, but different from, the one under which the answer is stored. BridgeKG-RAG uses GPT-4o to add typed "bridge" edges between related entities, built from hand-designed similarity features and clustering. At query time, a few bridges expand the seed entities, while only factual edges supply evidence. On GapQA, a new benchmark of rewritten Doc2Dial, MultiDoc2Dial and PrivacyQA questions, the method substantially improves evidence recall and answer accuracy over KG-RAG without bridges.

Critical strengths
  1. A clear problem and a clean design. Bridges only change where retrieval starts and never count as evidence, so answers stay grounded. The paper is easy to read.
  2. Large gains in the target setting. On GapQA the method clearly beats KG-RAG, ToG, RoG and dense entity expansion. A blind human audit supports the result, and performance on the original questions does not degrade.
Critical weaknesses
  1. The benchmark mirrors the method. GapQA replaces the answer entity with "a related but different entity that a confused user might say instead", which is what the bridge-labeling prompt asks GPT-4o to find. Large gains are expected and say little about real queries.
  2. GapQA construction is under-specified. The model, the prompt and whether humans validated the rewrites are not reported.
  3. Some results look inconsistent. Text RAG and HyDE reach very low recall even on corpora of only a few hundred documents. KG-RAG also does better on the rewritten GapQA-Privacy questions than on the original PrivacyQA questions (Table 7), although the rewrites are meant to be harder.
  4. Possible leakage. The "layout" feature relies on conversation links between documents, and the paper does not say whether these come from the evaluation dialogues.
  5. Limited technical novelty. Related-entity edges already exist in HippoRAG, which is not discussed. The method is a GPT-4o prompt pipeline over hand-crafted features, with little analysis of the bridges themselves.
Questions for authors
  1. How were the GapQA rewrites generated and validated?
  2. Why is Text RAG and HyDE recall so low, and why does KG-RAG do better on rewritten PrivacyQA questions than on the originals?
8 comments
AuthorsOpenReview ↗

Response 1

We sincerely thank you for your careful assessment and constructive feedback. We have conducted additional experiments and clarified the methodology to address your concerns.

The benchmark mirrors the method. GapQA replaces the answer entity with "a related but different entity that a confused user might say instead", which is what the bridge-labeling prompt asks GPT-4o to find. Large gains are expected and say little about real queries.

Thank you for raising this important concern regarding the independence and real-world generalizability of GapQA. We would like to clarify that GapQA is constructed independently of BridgeKG-RAG and follows a human-driven construction and validation process to ensure that the questions remain natural and preserve their original information needs. We also provide additional experimental evidence demonstrating that our method remains effective on naturally occurring wording-gap questions beyond the constructed benchmark. GapQA is designed as a diagnostic benchmark for a specific retrieval failure mode, namely entity-reference mismatch, where the query mentions one entity but the supporting evidence is organized under another. Its purpose is to test whether retrieval can recover evidence in this controlled setting. We therefore complement it with evaluations on unmodified public questions and a separate internal dataset of naturally occurring wording gaps.

First, GapQA is constructed independently of our proposed method. As described in Appendix C, the benchmark questions are finalized before the bridge graph is constructed. No predicted bridge edges, bridge types, confidence scores, or traversal outcomes are used during question generation or filtering. During inference, BridgeKG-RAG receives only the input question and the constructed graph, without access to the original question, the original answer-bearing entity, or the substitutions introduced during benchmark construction. Therefore, the benchmark does not incorporate information from the method's bridge predictions or retrieval decisions.

Second, GapQA follows a human-driven construction and validation process to ensure that the rewritten questions are natural and preserve their original information needs. For each sample, human annotators identify the original answer-bearing entity, manually select a plausible alternative entity that a user might naturally mention, and formulate a question that preserves the original information need while introducing a wording gap. GPT-4o is used only to improve the fluency and naturalness of the manually constructed questions, without determining entity substitutions or changing the intended meaning. Each rewritten question is subsequently reviewed by multiple human annotators to verify its naturalness, semantic consistency, answer preservation, and evidence sufficiency. Only samples receiving unanimous approval are retained.

To further examine the realism and semantic reliability of GapQA, we conduct an additional GPT-4o evaluation using private, anonymized real-user questions as references. We assess three complementary dimensions. Linguistic style similarity measures whether the rewritten questions resemble natural user expressions. Ambiguity style similarity examines whether indirect references, implicit entities, and underspecified terminology resemble those found in real queries. Information intent recoverability evaluates whether the rewritten question preserves sufficient information for the original information need to be inferred. The first two dimensions are evaluated against domain-matched real-user questions, while the third examines consistency between the original and rewritten questions.

DatasetLinguistic Style (%)Ambiguity Style (%)Intent Recoverability (%)Overall (%)
GapQA-Doc94.291.894.693.5
GapQA-Multi93.692.494.293.4
GapQA-Privacy92.890.693.392.2
Macro Average93.591.694.093.0

As shown above, GapQA achieves a macro-average score of 93.0%, including 93.5% for linguistic style similarity, 91.6% for ambiguity style similarity, and 94.0% for information intent recoverability. Together with the human validation process, these results support that GapQA introduces meaningful wording gaps while maintaining natural expressions and recoverable information needs.

AuthorsOpenReview ↗

Response 1 (continue)

Beyond validating GapQA itself, we examine its relevance to naturally occurring wording gaps in real user questions. We use a private, anonymized internal dataset of 1,000 customer inquiries collected from real service interactions. We first select inquiries previously flagged as ambiguous or underspecified in internal annotations, then identify cases where the user-mentioned entity differs from the entity associated with the supporting evidence. Both methods are evaluated on the same questions.

MethodRecallF1Accuracy
KG-RAG (no bridging)0.3420.3940.516
BridgeKG-RAG0.6180.4290.652

BridgeKG-RAG achieves better overall performance across all evaluation metrics on these naturally occurring customer questions. The findings provide supporting evidence that wording bridges are useful in practical retrieval scenarios.

Furthermore, BridgeKG-RAG also demonstrates consistent improvements on the original, unmodified Doc2Dial, MultiDoc2Dial, and PrivacyQA datasets, as reported in Table 7 of Appendix E. These experiments use original questions without introducing synthetic wording gaps.

DatasetMethodRecallF1Accuracy
Doc2DialKG-RAG (no bridging)0.63800.32840.6320
BridgeKG-RAG0.65200.33120.6400
MultiDoc2DialKG-RAG (no bridging)0.69600.33000.6540
BridgeKG-RAG0.70800.33310.6620
PrivacyQAKG-RAG (no bridging)0.34630.23340.3346
BridgeKG-RAG0.36110.23860.3424

BridgeKG-RAG performs better across all reported metrics on the original public datasets. The smaller gains are consistent with a broader mix of questions, including cases where the relevant evidence can already be reached from the entities mentioned in the question.

Together, the human-led construction process and realism evaluation support the validity of GapQA, while the internal customer-query experiment and evaluations on original public questions provide complementary evidence of applicability in other settings. We sincerely appreciate your thoughtful feedback and hope these clarifications and additional results address your concerns.

AuthorsOpenReview ↗

Response 2

GapQA construction is under-specified. The model, the prompt and whether humans validated the rewrites are not reported.

Thank you for requesting further details about how GapQA was constructed and validated. The construction procedure is human-led: annotators identify the original answer-bearing entity, choose a plausible alternative, and draft the gap question. GPT-4o is used for fluency polishing only. Multiple human reviewers independently verify naturalness, intended meaning, answer preservation, and sufficiency of evidence, with unanimous acceptance required. We will disclose the exact polishing prompt and annotation/validation protocol in the revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.