acceptodds
Under review as a conference paper at ICLR 2027

Perturbation Attribution Fragmentation under Alias-Redundant Evidence in Knowledge-Graph RAG

Abstract

When several retrieved graph paths express the same answer bearing fact, a language model reader can lose one and still answer correctly. Perturbation attribution splits credit across these paths, making the top ranked path a poor guide to which evidence must be removed to change the answer. We study this effect in knowledge graph retrieval augmented generation, or KG RAG, across three corpora and four open readers, with an eight question confirmation on a 72B reader. Controlled sweeps vary the number of alias paths for one fact while holding the question, reader, and distractors fixed. As this number grows, the top path's share of the group's credit decreases toward an equal split, and the top ranked path changes across perturbation seeds. A deletion study uses 25 WebQSP questions with Qwen 1.5B. With four or eight equivalent paths, removing the top ranked path changes a correct answer to an incorrect one in only 4% of cases; removing the whole group, a larger intervention, does so in 92 to 96%. Held out surrogate fidelity signals controlled redundancy but does not identify a uniquely responsible path. Jointly masking known groups yields high group level fidelity. In natural retrieval, 669 of 1,628 WebQSP rows contain multiple answer bearing paths, which need not be semantically interchangeable, and the tested grouping detectors differ widely in accuracy. These findings support reporting path attributions together with their grouping rule and deletion unit.

Primary area:
interpretability and explainable AI
Keywords:
knowledge graph retrieval augmented generation, perturbation attribution, evidence redundancy, attribution fragmentation, semantic path grouping, surrogate fidelity, explanation evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_rCu3OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how perturbation-based attribution methods (LIME/SHAP-style) assign importance to retrieved paths in knowledge-graph RAG when several paths express the same fact. Through controlled experiments that add alternative expressions of one fact, the authors show that the importance is split across these paths and that removing the top-ranked path alone rarely changes the answer. They propose treating such paths as one group when computing attributions, and they recommend reporting the grouping rule together with the path scores.

Critical strengths
  1. The paper asks a well-motivated question: how path-level attribution should be read when several retrieved paths carry the same information. This is relevant for anyone who uses such scores to explain KG-RAG answers.
  2. The experiments are carefully controlled. Only the number of equivalent paths is varied, while the question, reader and other evidence stay fixed. A control condition that adds irrelevant facts about the same entity helps rule out simple context dilution.
  3. The authors are careful about what their results do and do not show, and the discussion of limitations is candid.
Critical weaknesses
  1. The practical implications could be made clearer. The main finding is that equivalent paths end up sharing importance, which is largely what one would expect; the analysis in Section 3.1 also predicts it. I was left wondering how this insight could be used in practice, for example to improve answer accuracy, reduce cost, or decide which evidence to keep. Showing even one such downstream benefit would make the contribution much more convincing.
  2. The paper cites eight references, and its discussion of both attribution methods and graph-based RAG is quite brief. Several relevant lines of work exist: how attribution behaves when inputs are redundant or correlated, how to compute importance for groups of inputs rather than individual ones, and how to attribute RAG outputs to retrieved context by ablating parts of it and fitting a surrogate model. Positioning the paper against these lines of work would make its contribution easier to assess.
  3. The novelty of alias canonicalization is unclear. Treating equivalent pieces of knowledge as a single unit is a common idea. For example, synonymous entities are often linked or merged when knowledge graphs are built for RAG, and similar entries are consolidated in agent memory. A discussion of how the proposed grouping differs from these practices would help clarify what is new.
  4. The scope and scale of the experiments are limited. The setting focuses on cases where several paths supporting the same answer are retrieved together, and the equivalent paths in the controlled study are constructed by the authors. The experiments also use a fairly small number of questions per setting and mostly small readers, so it is hard to judge how well the findings generalize.
  5. I found the paper as a whole difficult to follow. Starting from the abstract and introduction, the writing moves quickly between many specific points without a clear, connected storyline. The overall structure also differs from a typical CS paper; for example, Section 3 combines the method with experimental details. In addition, there is no concrete example of the actual data or the reader prompt. A clearer narrative, a worked example and a cleaner separation between method and experiments would help readers a lot.
Questions for authors
  1. Could the authors comment on how practitioners might use these findings in a KG-RAG system, for example for evidence selection, efficiency or citation? A small example would be very helpful.
  2. How does alias canonicalization relate to existing ways of merging equivalent knowledge, such as entity deduplication in graph-based RAG or group-level attribution?
  3. Could the authors share the reader prompt, the format of the path text, and one real example from the data?
  4. How many questions were filtered out by the selection criteria, and do the authors expect the findings to hold on larger question sets?
Reviewer_RwmROpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies what happens to perturbation based attribution in KG-RAG, when several paths in the graph express the same fact. In this case, the top rated path become not important, since removing it changes nothing, as the other copies of information still remain in the KG. To show this, the authors take one fact and verbalize it for k times, they measure how much credit the top path keeps, whether the top path changes across seeds. As k grows, the top path’s share drops toward 1/k, and the top path changes across seeds. Deleting the top path only costs 4%, deleting all equivalent paths costs over 90%.

Critical strengths

None

Critical weaknesses

The paper is not finished and presentation is poor. Sections 4’s subsections have no numbers. Nearly every result sentence is followed by a caveat that retracts part of it, so the reader cannot tell what the paper actually claims. There are only 9 references.

The research question and conclusion of the paper is trivial. The proposed treating the group as one unit, is group Shapley, which is decades old and not cited.

Questions for authors

see weakeness above

Reviewer_YTsrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates attribution fragmentation arising from redundant evidence paths in knowledge graph retrieval-augmented generation (KG-RAG). The authors provide a theoretical analysis of how attribution scores are distributed across equivalent evidence paths and conduct experiments using perturbation-based attribution methods across multiple datasets and language models. The results show that increased evidence redundancy leads to more dispersed attribution scores and less stable path rankings. Removing the highest-ranked path often has little impact on the generated answer, whereas removing the entire group of equivalent paths can substantially affect the output. These findings highlight the importance of group-level attribution for understanding evidence dependence in KG-RAG.

Critical strengths

S1. The paper investigates attribution fragmentation caused by redundant evidence in KG-RAG, highlighting the limitations of path-level attribution scores in reflecting the model's actual dependence on evidence.

S2. By varying the number of equivalent evidence paths and introducing irrelevant evidence as a control, the authors effectively distinguish the effects of evidence redundancy from those of simply increasing the amount of context.

Critical weaknesses

W1. The paper's main finding is that attribution scores become fragmented across multiple evidence paths expressing the same fact. Although the authors provide theoretical analysis and empirical validation, they do not sufficiently clarify what new theoretical insights their work offers beyond existing studies on attribution under feature redundancy. Furthermore, the proposed approach of jointly masking equivalent evidence paths is relatively straightforward and offers limited methodological novelty.

W2. The paper primarily uses LIME-style Ridge and KernelSHAP-style methods to investigate attribution fragmentation, but lacks sufficient comparisons with complete KG-RAG attribution methods. Although the authors discuss KG-SMILE, they do not reproduce or evaluate its full attribution pipeline.

Questions for authors

Q1. The attribution fragmentation observed in this paper appears to be a direct consequence of the symmetry property of Shapley-based attribution under redundant features. Could the authors identify a theoretical finding or an empirical phenomenon specific to KG-RAG that cannot be explained by existing attribution theory?

Q2. The paper shows that removing an entire evidence group is more effective than removing the highest-ranked individual path. However, these interventions have different deletion budgets. Under an equal budget of removed paths, does the proposed grouping strategy still outperform Top-k attribution, random selection, or existing KG-RAG attribution methods such as KG-SMILE? Without such comparisons, how can the authors demonstrate that the improvement comes from more meaningful attribution rather than simply removing more evidence?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.