acceptodds
Under review as a conference paper at ICLR 2027

PATHSMOOTHING: Mitigating Semantic Drift and Hub Noise via Path Evidence in KG-based RAG

Abstract

Knowledge graph-based retrieval-augmented generation (KG-based RAG) connects information across text passages to support answer generation. Recent studies have shown that the main limitation of KG-based RAG lies in retrieving excessive irrelevant information, researchers have proposed optimization methods centered on path filtering. However, existing methods focus on structural factors when conducting retrieval, while ignoring the semantic factors, which results in the introduction of noise nodes. To overcome this, we propose PathSmoothing that uses personalized PageRank to aggregate the structural evidence scores of entities. It enumerates context paths containing entities in the subgraph, calculates the semantic relevance of the paths, and then smooths these scores by combining the original semantic scores of these triples. PathSmoothing filters triples and retrieves their source text as evidence for answer generation. Experiments on six datasets show that PathSmoothing achieves an average win rate of 54.13% against PathRAG across four evaluation dimensions on domain-specific tasks, and increases the average answer matching score from 50.24 to 52.02 on multi-hop question answering datasets. PathSmoothing also improves LightRAG and PathRAG when applied as a plugin to them. Our code and datasets are available at supplementary material.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Knowledge Graph, Retrieval-Augmented Generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_uEsVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PathSmoothing, a triple-level reranking method for knowledge graph based RAG (KG-RAG) that mitigates two common issues: semantic drift during graph traversal and noise introduced by structurally important but query-irrelevant hub nodes. Instead of scoring triples in isolation or relying solely on structural propagation, the method evaluates each candidate triple in the context of the paths it resides in within the query’s local subgraph. It uses Personalized PageRank to compute structural importance weights of entities, enumerates short paths containing each triple, calculates path-query semantic similarity, and combines the triple’s own semantic score with a weighted average of path semantic scores using PPR weights. A cliff-detection strategy selects the final set of triples, whose source text passages are then retrieved for answer generation.

Critical strengths
  1. Novel path-aware triple scoring that refines relevance using surrounding path context, effectively addressing both semantic drift and hub noise in a unified framework.
  2. Plug-and-play compatibility: works as a post-retrieval filter without modifying upstream graph construction or downstream answer generation, with consistent improvements demonstrated for both LightRAG and PathRAG.
  3. Comprehensive empirical validation across six diverse datasets, four strong baselines, multiple evaluation dimensions, ablation studies, hyperparameter sensitivity analysis, and cross-model verification.
Critical weaknesses
  1. It is an optimization of existing KG-RAG retrieval rather than a paradigm change, with moderate absolute performance gains.
  2. Increased computational overhead: path enumeration and embedding add extra computation cost, and the paper does not quantify the latency impact across different graph scales.
  3. While path semantics and structural weights are validated, there is less analysis of path selection strategies, PPR parameter settings, and the cliff detection threshold mechanism.
  4. Performance is highly dependent on text embedding model quality, and robustness under low-quality or domain-specific embedding models is unclear.
Questions for authors
  1. Can you provide latency comparisons of PathSmoothing versus baselines at different graph scales? How does overhead scale with the number of triples and path length?
  2. Has the method been tested on graph construction pipelines other than LightRAG, such as GraphRAG’s community structure? Is it equally effective on graphs built with different extraction models?
  3. The paper uses a fixed budget of 20 paths per triple and 600 total paths. How sensitive is performance to these parameters, and is there a point of diminishing returns?
  4. Why are the gains larger on domain-specific datasets than on multi-hop QA datasets? What types of queries benefit most from path smoothing?
1 comment

I have not received any response from the author up to now.

Reviewer_8RZpOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

PathSmoothing is a training-free triple filter for KG-based RAG. It scores each candidate triple as a convex combination of its query similarity and a PPR-weighted average of the query similarities of paths that contain it, then keeps triples using a score-gap rule. With DeepSeek-V4-Flash, it is roughly on par with or slightly better than PathRAG on UltraDomain and SQuALITY and gives marginal gains on HotpotQA and MuSiQue. It also works as a plug-in for LightRAG and PathRAG.

Critical strengths
  1. Simple and plug-and-play. The method needs no training and improves both LightRAG and PathRAG when inserted.

  2. Intuitive idea with low overhead. Rescoring a triple by the paths around it is a sensible way to keep bridging triples that look unrelated to the query, as the case study illustrates. The smoothing step uses only embeddings and PPR, with no extra LLM calls, and online token use stays close to PathRAG's (Table 3).

Critical weaknesses
  1. Limited novelty. A weighted average of cosine scores plus a cutoff is a standard score-propagation heuristic. The claim that prior work ignores semantics is overstated (HippoRAG 2, SubgraphRAG).
  2. Small gains. The win rate against PathRAG is close to a tie, and the HotpotQA LLM score does not change.
  3. Weak evaluation. A single LLM judge, which is also the generator, scores a small number of questions per dataset, with no significance tests.
  4. Missing evidence and baselines. There are no retrieval metrics and no analysis of semantic drift or hub nodes, the two problems in the title. Dense retrieval with a reranker and HippoRAG 2 are not compared.
  5. Needs proofreading. λt\lambda_t differs between Section 4.1 and Table 7, and Section 3.2 states two different path budgets. There are also many grammatical errors (e.g., lines 45, 106–107 and 113–114).
Questions for authors
  1. Can you report retrieval metrics for the retained triples, such as supporting-fact recall and precision?
  2. How does the method compare with HippoRAG 2 and with dense retrieval plus a reranker?
2 comments

Thank you for your question.

Our current experiments assess triple retention through downstream QA performance, scoring ablations, and retrieval cases.

In Table 2 of the paper, PathSmoothing improves AMS over PathRAG from 59.24 to 60.23 on HotpotQA and from 41.24 to 43.80 on MuSiQue. This indicates that the retained evidence, mapped back to source text, supports more accurate answer matching.

Section 4.3 provides more targeted evidence: on MuSiQue, AMS falls from 43.80 to 39.21 when path semantics are removed and only original triple similarity is used, and to 40.92 with uniform path weights. These results support the contributions of both path semantics and structural weighting to answer quality.

Appendix C illustrates the mechanism. For “When did the manufacturer of SS.11 terminate its independent existence?”, PathRAG follows missile-service-year evidence and answers 1960. PathSmoothing instead emphasizes the manufacturer's merger relation and correctly answers 1970. This shows how path context distinguishes topically related evidence from the relation needed to answer the question. Together, these results and cases support the downstream utility of the retained evidence and the proposed mechanism.

Table 5 compares PathSmoothing with RAG and HippoRAG 2 on Legal and Computer Science across the four answer-quality dimensions, using the manually adjusted baseline scores.

Table 5. Comparison on Legal and Computer Science using manually adjusted baseline scores. PS. denotes PathSmoothing, Comp. denotes Comprehensiveness, and Empo. denotes Empowerment. PS. reports the percentage of questions where it scores strictly higher than the corresponding baseline. Baseline columns report the complementary percentage, including ties. An asterisk marks the higher value in each pair, which was underlined in the original table.

            Legal                                    Computer Science
            ---------------------------------------  ---------------------------------------
Metric      RAG      PS.      HippoRAG 2  PS.         RAG      PS.      HippoRAG 2  PS.
----------  -------  -------  ----------  -------     -------  -------  ----------  -------
Comp.       33.33%   66.67%*  36.78%      63.22%*     25.93%   74.07%*  40.74%      59.26%*
Diversity   51.72%*  48.28%   55.17%*     44.83%      41.98%   58.02%*  39.51%      60.49%*
Empo.       28.74%   71.26%*  44.83%      55.17%*     28.40%   71.60%*  45.68%      54.32%*
Accuracy    27.59%   72.41%*  28.74%      71.26%*     28.40%   71.60%*  39.51%      60.49%*

For multi-hop QA, Table 6 compares mean LLM-based Accuracy scores on MuSiQue, where PathSmoothing scores 57.30, HippoRAG 2 scores 56.14, and RAG scores 40.08; the latter two scores are manually adjusted.

Table 6. Mean LLM-based Accuracy scores on MuSiQue (0–100 scale). PS. denotes PathSmoothing. RAG and HippoRAG 2 scores are manually adjusted. An asterisk marks the highest score, which was bold in the original table.

Method      MuSiQue LLM
----------  -----------
RAG         40.08
HippoRAG 2  56.14
PS.         57.30*

Thanks the authors for the reply. While part of my concerns have been resolved, I will keep my current score for the remaining ones.

Reviewer_so3xOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authos proposed PathSmoothing, a training-free triple reranker for KG-based RAG. It combines a triple's query similarity with the semantic similarities of paths containing it, weighted by sums of personalized PageRank node scores. Score-gap filtering selects triples, whose source passages are trimmed for generation. Experiments cover three UltraDomain domains, SQuALITY, HotpotQA and MuSiQue. The reported results outperform baselines on several datasets. Component ablations and integrations into LightRAG/PathRAG provide additional evidence for the reranker.

Critical strengths
  1. A localized technical contribution supported by a meaningful component comparison. On MuSiQue, full PathSmoothing obtains 43.80 AMS versus 39.21 for triple-only scoring and 40.92 for uniform weighting of the same selected paths (sec 4.3, Fig. 3). Holding the selected paths fixed is a sensible way to test their aggregation weights. These comparisons provide direct evidence that path semantics and weighting matter within this pipeline.

  2. The plugin results support practical reuse. Adding the module increases MuSiQue AMS from 35.04 to 38.30 for LightRAG and from 41.24 to 46.04 for PathRAG. The latter 4.80-point gain is appreciably larger than the standalone improvement over PathRAG. Together with the long-form results, this supports the potential value of a relatively small modification to an existing system.

Critical weaknesses
  1. The evaluation does not yet establish how much of the improvement is reliable, answer-correctness improvement. Against PathRAG, average AMS increases by 1.78 points, while the semantic LLM score increases by approximately 0.37; HotpotQA's LLM scores are identical (Table 2). AMS awards 100 for exact equality but 50 for containment, so equally correct short and explanatory answers can receive different scores. Appendix H supplies a semantic rubric that permits paraphrases, and the plugin/ablation gains are meaningful counterevidence: the manuscript does not prove that formatting causes the improvement. However, evaluated sample counts, exact splits, exclusion of tuning examples, score-to-win tie handling, and uncertainty are not specified in sec4.1 or Appendix A. Without those details and a format-robust check of existing outputs, it is difficult to quantify the retrieval contribution, particularly for the smaller headline gains.

  2. A canonical executed configuration is missing. Sec 3.2 says both that at most 600 paths are retained and that selection stops at 2,000. Table 7 usefully distinguishes 2,000 candidate triples from 600 path embeddings, making a transcription error a plausible explanation, but the text should identify the actual counting and reuse convention. The main setup uses lambda_t=0.6, whereas Table 7 gives 0.50. Candidates are described as edges adjacent to localized entities in sec 3.2, versus triples from the expanded subgraph in Algo 1. These affect eligible evidence and smoothing, not merely presentation. The broad lightweight characterization also exceeds the available cost evidence: online LLM tokens are reported, but path embedding and retrieval-time overhead are not measured, a limitation Appendix G acknowledges.

  3. limited and old baselines, more recent basslines should be compared, especially there are so many work published last year.

Questions for authors
  1. Evaluation clarification: Please provide sample counts and split identifiers, the relationship between the 80 HotpotQA tuning examples and evaluated questions, and the rule converting judge scores into wins, including ties. What do paired uncertainty estimates from the saved outputs show for the PathRAG standalone and plugin comparisons?

  2. Focused validation: In the plugin and component comparisons, are the answer generator, instructions, passage trimming and context budgets unchanged? Can the existing outputs be rescored using normalized answer-only EM/F1 or a common semantic-correctness rule? This would clarify answer-format effects without requiring a new application benchmark or wholesale rerunning of every baseline.

  3. Implementation clarification: Which lambda_t, candidate definition and path-selection procedure produced the reported tables? Does the 600 limit count distinct embedded paths or path–triple memberships, and how are shared paths reused? Please confirm whether 2,000 is a triple budget rather than the path stopping condition. Measured embedding/retrieval overhead would additionally establish the intended cost claim; otherwise that claim should be narrowed.

2 comments

Thank you for pointing out the missing evaluation-protocol details.

All reported evaluations use official splits, with no hyperparameter tuning on the evaluation sets.

Table 1. Datasets, splits, and evaluation sample counts.

Dataset                       Split used              N    Purpose
----------------------------  ----------------------  ---  ---------------------------
UltraDomain-Agriculture       test                    81   Domain QA
UltraDomain-Computer Science  test                    81   Domain QA
UltraDomain-Legal             test                    87   Domain QA
SQuALITY                      test                    150  Long-document summarization
HotpotQA                      dev (distractor)        303  Multi-hop QA
MuSiQue                       dev                     137  Multi-hop QA
UltraDomain-Art               80 examples from test   80   Tuning only, not reported
HotpotQA                      80 examples from train  80   Tuning only, not reported

Here, N denotes the number of evaluation or tuning examples. The 80 HotpotQA tuning examples come from the train split and are disjoint from the dev evaluation questions. They are used only to select hyperparameters such as λₜ, the absolute threshold θ(abs), and the relative threshold θ(rel), and do not contribute to reported results. The 80 Art tuning examples likewise do not overlap with the Agriculture, Computer Science, Legal, or SQuALITY evaluation questions.

For each question and evaluation dimension, PS. wins only if its score strictly exceeds the baseline score. Ties and lower scores count as non-wins. Thus, the PS. win rate equals W / N, where W is the number of strict wins and N is the number of evaluated questions. The baseline column reports 1 minus the PS. win rate, including ties.

We performed paired bootstrap resampling (10,000 iterations) and paired uncertainty estimation on saved per-question scores.

Table 2. Strict win rates (%) of PathSmoothing against each baseline with individual 95% Wilson confidence intervals. Only strictly higher scores count as wins; ties remain in the denominator and do not count as wins.

Metric             PathRAG               HippoRAG              LightRAG              GraphRAG
-----------------  --------------------  --------------------  --------------------  --------------------
Agri (N=81)
Comprehensiveness  54.32 [43.52, 64.73]  54.32 [43.52, 64.73]  58.02 [47.15, 68.17]  67.90 [57.12, 77.06]
Diversity          53.09 [42.33, 63.57]  49.38 [38.77, 60.05]  54.32 [43.52, 64.73]  69.14 [58.40, 78.14]
Empowerment        55.56 [44.73, 65.88]  55.56 [44.73, 65.88]  50.62 [39.95, 61.23]  66.67 [55.85, 75.97]
Accuracy           54.32 [43.52, 64.73]  53.09 [42.33, 63.57]  60.49 [49.61, 70.43]  61.73 [50.84, 71.55]

Legal (N=87)
Comprehensiveness  56.32 [45.85, 66.26]  58.62 [48.12, 68.39]  56.32 [45.85, 66.26]  68.97 [58.62, 77.71]
Diversity          55.17 [44.72, 65.18]  59.77 [49.26, 69.45]  54.02 [43.60, 64.10]  68.97 [58.62, 77.71]
Empowerment        52.87 [42.49, 63.02]  58.62 [48.12, 68.39]  56.32 [45.85, 66.26]  67.82 [57.43, 76.70]
Accuracy           51.72 [41.38, 61.93]  52.87 [42.49, 63.02]  57.47 [46.98, 67.33]  65.52 [55.06, 74.66]

Thank you for this constructive suggestion and identifying this ambiguity.

The answer generator, instructions, passage trimming, and context budgets were unchanged across the plugin and component comparisons:

  • Generator and instructions: All methods and variants used DeepSeek-V4-Flash with identical generation prompts and instructions.
  • Trimming and budgets: We fixed the chunk segmentation budget at 1,200 tokens, retained the evidence paragraph and its immediate neighbors, and used answer_max_tokens = 1024.
  • PathRAG+PS vs. PathRAG: PS. is a post-retrieval filter that changes only the candidate triples passed downstream. Text chunk retrieval, paragraph trimming, and context assembly remain identical. Filtering may naturally shorten the context under the same budget rules; we do not pad it.

AMS addresses answer-format differences by awarding 100 for an exact match, 50 for a non-exact match where either answer is a substring of the other, and 0 otherwise. Here, the two answers are the generated and reference answers.

AMS therefore extends EM with partial credit for containment-based wording differences, including additional explanations or qualifications. Unlike token-overlap F1, it uses explicit matching categories rather than proportional token overlap. This suits our mix of concise and explanatory answers without making scores depend on token overlap throughout an explanation.

AMS remains a string-matching metric: containment does not guarantee full semantic correctness, and meaning-preserving paraphrases may be missed. We additionally report reference-based LLM scores to assess semantic correctness beyond string matching.

The current main implementation uses the following configuration:

  • Smoothing coefficient: λₜ = 0.5 for all candidate triples. Pₜ denotes the paths actually allocated to triple t; scoring follows Equation (9).
  • Candidates: Distinct edges t = (s, r, o) in the subgraph reconstructed after entity-level local PPR selection. If there are more than 2,000, we retain the top 2,000 ranked by the larger of PPR(s) and PPR(o). Every retained candidate receives an original similarity score. Thus, 2,000 is the candidate-triple budget, not a path-selection stopping condition.
  • Path selection: Candidate paths are ranked by their weights wₚ, calculated as the sum of PPR(v) over all nodes v in path p. We retain the top 600, or all paths if fewer are available, and expand only this retained set. For each candidate triple, eligible paths containing it may be extended by at most one edge on each side; either or both sides may remain unextended. The resulting paths are ranked by wₚ, and at most 20 are retained per triple.

For the runtime comparison, Table 4 reports average time per question for PathSmoothing and the baselines on Computer Science, Legal, and MuSiQue.

Table 4. Average runtime per question on each dataset (seconds).

Dataset  Ours   PathRAG  HippoRAG  LightRAG
-------  -----  -------  --------  --------
Cs       28.26  17.05    7.75      20.81
Legal    27.29  33.27    10.09     24.04
Musique  54.10  42.63    14.36     41.50

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.