acceptodds
Under review as a conference paper at ICLR 2027

From Detection to Mitigation: Neuron Attribution for Contextual Hallucination in RAG

Abstract

Retrieval-augmented language models may ignore externally provided evidence when it conflicts with parametric knowledge, leading to contextual hallucination. We introduce NADM (Neuron Attribution for Detection and Mitigation), a unified framework that uses response-conditioned neuron attribution as a shared internal signal for contextual hallucination detection and mitigation. NADM represents a generated response by computing gradient-based attribution from internal neurons to the logits of its realized tokens, and trains a simple linear classifier on the resulting attribution patterns to distinguish context-faithful from contextually hallucinated generation. Across multiple datasets and model scales, the NADM detector achieves strong discrimination between context-faithful and contextually hallucinated generation. We further examine the causal role of the key neurons identified by the detector and find that, although they participate in context–parametric competition, transferring their activations alone reproduces only a small fraction of the model's context-versus-parametric preference difference. This motivates using attribution as a readout rather than directly manipulating the selected neurons. At inference time, NADM samples alternative responses, scores them with the same frozen detector, and selects alternatives with stronger evidence of contextual faithfulness. This yields an inference-time intervention that requires no update to the base-model parameters and does not assume that direct manipulation of the selected neurons is sufficient to reproduce the desired source preference. Together, NADM connects detection and mitigation through the same response-conditioned attribution signal.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Retrieval-Augmented Generation, Context–Parametric Conflict, Neuron Attribution, Contextual Faithfulness

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_sxQLOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies contextual hallucination in RAG, where the model ignores the given context and answers from its own memory. The paper has three parts. First, it builds a detector. For each generated answer, it measures how much each FFN neuron contributes to the output, and trains a sparse linear classifier on these scores. Second, it runs a causal analysis on the 134 neurons that the detector selects most often. These neurons do affect whether the model follows the context, but they explain only a small part of this behavior. Third, it uses the detector for mitigation. It samples several candidate answers and picks the one the detector rates as most faithful. Experiments use three models and four datasets. The detector achieves the best AUROC in all 12 settings, and mitigation improves the faithful rate over single-pass baselines.

Critical strengths
  1. The causal analysis is careful and honest. It uses matched control neurons and proper statistical tests. The authors clearly report a negative result: these neurons are involved, but they are not enough to explain the behavior.

  2. The results are consistent. Both detection and mitigation improve across three models and four datasets, and mitigation does not change the model weights.

Critical weaknesses
  1. Simple detection baselines are missing. In ConFiQA, the faithful answer usually appears in the context, while the memory-based answer does not. So a simple check of whether the answer appears in the context may already work very well. A linear probe on hidden states is also missing. In addition, an existing method, SEP, already performs close to the proposed detector, and it even has higher Macro-F1 in 7 of the 12 settings. Overall, the advantage of the detector is not clear. This is my main concern.

  2. The mitigation comparison is not fair. The proposed method generates up to 16 candidates and reranks them, while all baselines decode only once. So the gain may come from sampling more candidates, not from the attribution signal. A fair comparison should use the same candidate pool with other simple scoring methods, and should report the extra compute cost.

  3. The setting is narrow. All tasks are short-answer QA with entity conflicts, and the models are relatively old (Llama-2 and Mistral-7B).

Questions for authors
  1. For detection, could you compare with two simple baselines? The first is a rule that checks whether the generated answer appears in the context. The second is a linear classifier trained on hidden states.

  2. Your mitigation method generates several candidate answers and picks one. Could you pick from the same candidates using other simple methods, and compare the results? Please also report how much extra compute your method needs, since the baselines only generate one answer.

Overall, the current experiments do not show that the attribution signal is better than simple alternatives.

Reviewer_AEMKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies contextual hallucination in RAG when parametric knowledge overrides conflicting retrieved evidence. It proposes NADM, which uses response-conditioned FFN neuron attribution to train a sparse linear detector and reuses that detector to select more context-faithful responses from sampled candidates. Activation-transfer analyses find that detector-selected neurons influence source preference but reproduce only a limited part of the donor–receiver difference, motivating their use as a predictive readout. Detection and mitigation are evaluated on ConFiQA and an adapted SQuAD2 setting across three model configurations from two families.

Critical strengths
  • NADM achieves the highest detection AUROC and Faithful rate across all tested settings, supporting the use of a shared attribution signal for detection and mitigation.
  • Causal experiments with matched controls show that selected neurons influence source preference but reproduce little of the donor–receiver difference. This supports their use as a predictive readout.
Critical weaknesses
  • The added value of attribution is not isolated. NADM considers up to 15 sampled continuations plus Vanilla, but no baseline reranks the same candidate pool with an existing detector. There is also no comparison with raw FFN activations under the same classifier and supervision.
  • Generalization beyond the dataset used to fit each detector remains untested. Detectors and acceptance margins are fitted separately for every model-dataset pair, and detection uses stratified cross-validation within each dataset. The current experiments also focus on short-form answers.
Questions for authors
  • How does NADM compare with SEP reranking the same candidates and raw FFN activations using the same classifier and supervision?
  • Does a trained detector transfer to an unseen dataset without refitting, and how does it perform on longer responses?
  • Are selected BOTH/OTHER responses evaluated for contextual support, given their exclusion from binary detection?
  • What are the total inference latency and peak memory use relative to baselines, including candidate generation and attribution scoring?
Reviewer_xEXbOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper combines neuron attribution, a sparse linear classifier, and candidate reranking to detect and mitigate contextual hallucination. The evaluation does not sufficiently establish the contribution of the attribution features or distinguish the benefits of the proposed scorer from those of additional sampling. The causal analysis shows limited effects under the tested interventions but does not establish the advantage of the subsequent reranking strategy. I find the methodological contribution and experimental evidence insufficient and recommend rejection.

Critical strengths

Contextual hallucination is a practically relevant problem. The paper evaluates multiple model configurations and includes matched controls in its activation-transfer experiments. The authors also appropriately distinguish predictive association from causal sufficiency.

Critical weaknesses

The paper does not establish a substantial methodological contribution. The detector is a supervised classifier over neuron-attribution features, and the mitigation method uses this classifier to rank sampled responses. Reusing a detector as a candidate scorer does not, by itself, constitute a substantial advance. The paper needs to establish what this combination enables beyond a conventional internal-state classifier followed by candidate selection. The current evaluation does not demonstrate that distinction.

The experiments do not establish the broader significance of the proposed internal signal. A separate detector is trained for each model–dataset pair, and detection is evaluated within the same dataset. These results show that the features predict the benchmark labels, but do not establish that they capture contextual faithfulness beyond dataset-specific answer patterns. The causal experiments show limited effects under the tested interventions and do not provide a sufficient mechanistic account. Consequently, neither a broadly useful detection method nor a substantial explanation of contextual hallucination is established.

Questions for authors

What is the main methodological advance beyond supervised classification of internal features followed by candidate reranking?

What evidence establishes that the learned signal captures contextual faithfulness beyond the particular model–dataset setting used to train the detector?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.