acceptodds
Under review as a conference paper at ICLR 2027

ReNA: Detecting Contextual Hallucinations in RAG via Reference-Contrast Neuron Atlas

Abstract

Retrieval-Augmented Generation (RAG) reduces reliance on parametric knowledge by grounding language models in retrieved evidence. However, models may still override accurate context and generate responses from conflicting parametric memory. Existing detectors either evaluate outputs externally or analyze internal signals through representations or predefined component functions, leaving open the question of which specific components consistently support different knowledge sources. We instead investigate which specific neurons consistently participate in supporting contextual versus parametric generation. We introduce ReNA, a Reference-Contrast Neuron Atlas for detecting contextual hallucinations in RAG. Given the same context and question, ReNA contrasts gradient-based neuron attribution between a context-supported reference and a conflicting parametric reference. It then constructs a sparse atlas of attention and FFN neurons exhibiting strong and consistent source preferences. An actual response is classified from its attribution pattern over the discovered atlas. Experiments across multiple models and datasets show that ReNA consistently outperforms strong output-based and internal-state baselines. Ablation studies further demonstrate the effectiveness of reference contrast and identity-specific neuron attribution. Causal interventions identify the Atlas as a distributed, branch-conditioned readout and amplifier, suggesting that faithful/parametric information propagates as a broad network state expressed across many components rather than through a small, independently sufficient circuit.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Retrieval-Augmented Generation, Contextual Hallucination Detection, Neuron Attribution

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_hBUAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ReNA, a detector for context–memory conflicts in RAG. Given a context, a question, and a short model-generated answer, it predicts whether the answer follows the context or reverts to the model’s parametric knowledge. For each training prompt, the authors teacher-force two reference answers under the same input: one supported by the context and one supported by parametric memory. They then compute positive gradient-times-activation attribution for every FFN neuron and every attention “neuron” (a value-coordinate within a head), normalize the scores into within-family shares, and take the difference between the two references. Neurons whose preference is directionally consistent across training examples and whose average contrast is large form a fixed per-layer “atlas.” This step uses only the references, not the actual responses or labels. At detection time, ReNA computes the same attribution for the actual response, reads out its shares on the atlas coordinates, and feeds them to a logistic-regression classifier. On the QA, MR, and MC subsets of ConFiQA and on a constructed SQuAD2 conflict set, with LLaMA-2-7B-Chat and LLaMA-2-13B-Chat, ReNA reaches average AUROCs of 0.936 and 0.927, above SEP and FrequencyAware. Causal experiments show that scaling the selected neurons shifts the contextual-versus-parametric logit margin more than matched random neurons, but that patching their activations restores the original preference less well than random neurons do.

Critical strengths
  1. The paper defines its units of analysis and attribution procedure precisely, and it separates atlas discovery (references only) from classifier fitting (actual responses and labels) cleanly.

  2. The same-prompt reference contrast is a sensible design. It controls for input-side variation when comparing two answer trajectories.

  3. The causal analysis is careful and honest. The authors use matched random controls, paired bootstrap intervals, and multiple-comparison correction, and they report the negative sufficiency result instead of hiding it.

  4. The matched-random-atlas ablation is a good control, and the authors discuss its implications openly.

Critical weaknesses
  1. The task may be close to trivial, and the paper skips the obvious simple baselines. The prompt limits answers to five words, so the actual response almost always equals one of the two references. The SQuAD2 construction makes this explicit: a parametric example must satisfy r_act = A_p exactly, and a faithful example must match the gold answer. In ConFiQA, the context-supported answer appears verbatim in the context, while the parametric answer usually does not. A baseline that just checks whether the response appears in (or overlaps with) the context could plausibly reach near-perfect AUROC. I’d also want to see a linear probe on hidden states at the same positions, and the difference in response log-likelihood with versus without context. Among the current baselines, only SEP is supervised, so the comparison mostly pits a supervised classifier against unsupervised methods.

  2. The hyperparameters change across settings without a stated selection protocol, and the results are sensitive to them. On 7B, the authors use K = 64/32 with γ = 0.7 on ConFiQA, K = 256/128 with γ = 0.55 on SQuAD2, and change the regularization C to 0.03 for that one setting only. On 13B, they use K = 512/256 with γ = 0. The paper never explains how it picked these values. If the authors chose them by looking at cross-validation scores without nested selection, the reported numbers are optimistic. Table A.1 also has inconsistencies: two 7B rows carry the † marker; 128/64 beats the chosen 64/32 on ConFiQA AUROC (0.9714 vs. 0.9638); and the text says 13B ConFiQA Macro-F1 peaks at 256/128, while the table shows 512/256 is higher (0.9144 vs. 0.8982). On 7B SQuAD2, AUROC swings between 0.73 and 0.85 across atlas sizes.

  3. The evidence that specific neuron identities matter is weak, and the causal analysis studies a different neuron set from the detector. A random atlas with matched layer and family composition already reaches 0.888 AUROC on ConFiQA, so most of the signal seems to come from attribution shares in general rather than from the selected identities. The intervention experiments don’t use the detector’s atlas. They use direction-balanced sets ranked by a Wilson lower bound, with 64 attention and 128 FFN neurons per layer (a 1:2 ratio), whereas the detector uses 64/32 (2:1). The scaling effect grows steadily with budget up to 6,144 neurons per layer, and patching sufficiency falls below random on all four datasets. I read these results as saying the selected neurons aren’t especially privileged, which sits awkwardly with the paper’s “atlas” framing.

  4. The scope and practicality are limited. The paper only tests LLaMA-2 models, single-entity short answers, and one type of conflict, far from realistic RAG settings with long responses and partial hallucinations. Each detection needs a backward pass over the response, and the paper reports no compute cost. Building the atlas requires a known parametric answer for every training example. The cross-domain transfer test only freezes the atlas and retrains the classifier on the target domain, and the three ConFiQA subsets share the same construction, so the test says little about generalization.

Questions for authors
  1. How do a simple “response appears in context” or lexical-overlap baseline, and a linear probe on hidden states, perform on these datasets?
  2. Does the 13B model use γ = 0? If so, why do ReNA and Magnitude Only differ in Table 2 for 13B?
  3. How did you choose K, γ, and C for each model–dataset setting? Was the selection nested inside the cross-validation folds?
Reviewer_3H7POpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies post-hoc detection of contextual hallucination in RAG: given a context–question prompt and a model-generated answer, decide whether the answer follows the retrieved context or conflicting parametric memory. ReNA contrasts gradient×activation neuron attribution between a context-supported and a conflicting parametric reference teacher-forced under the same prompt, retains the attention and FFN neurons showing strong and directionally consistent source preference as a sparse atlas, and classifies responses with a linear model over their attribution shares on that fixed atlas, with atlas discovery never using responses or labels. Experiments on three ConFiQA subsets and a self-constructed SQuAD2 conflict set with LLaMA-2-7B/13B-Chat show consistent gains over seven baselines. Ablation studies also demonstrate the effectiveness of each component. Complemented by cross-domain analysis and causal intervention analysis.

Critical strengths

The proposed method is well-motivated at each decision. The same-prompt reference contrast holds prompt, question, and format fixed and varies only the knowledge source — a stronger control than contrasting attribution across differently formatted inputs; attributing along actual response tokens avoids the functional-coherence assumption the paper criticises; AN+FN coverage matches the premise that source preference is not confined to one family; and the linear readout keeps the claim about the atlas rather than classifier capacity.

The design of evaluation is solid. The baseline methods spanning output-based, representation-based, attention-based, and mechanism-based families. It also conduct 5-fold cross-validation. The ablation study is also well-controlled. The layer/family/cardinality-matched random atlas (averaged over five draws, std reported) is the right control for "identity vs. dimensionality", and magnitude-only vs. consistency-only cleanly isolates the two selection criteria.

The causal analysis is rigorous. It uses direction-balanced sets with sign-stability ranking via Wilson lower bounds, paired per-example random-adjusted effects, bootstrap CIs, permutation tests with BH correction, plus a first-order diagnostic to explain the caling-vs-patching dissociation.

Critical weaknesses

This paper only conduct experiment on one model family (Llama 2 7 B and Llama 2 13 B). Since the atlas is model-specific by construction, cross-family generality is needed. It is not clear whether the proposed method can be generalize well to other model families, it is suggested to conduct experiments on one more model family, e.g., Qwen 3.

Minor comment:

No qualitative example or error analysis anywhere (a single end-to-end worked example plus a false-faithful vs. false-parametric breakdown would help a lot).

The latex command for citation \citet was throughout the entire paper, but many of them should use the \citep command (the one with the parenthesis).

Table 1 does not report the standard deviation in the five-fold cross-validation.

Can add the following missing references about knowledge conflicts between context and parametric knowledge:

  • Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts. ICLR 2024
  • Entity-Based Knowledge Conflicts in Question Answering. EMNLP 2021
  • Can LMs Generalize to Future Data? An Empirical Analysis on Text Summarization. EMNLP 2023
Questions for authors
  1. Does ReNA still detect correctly when responses are paraphrased rather than copied (e.g. temperature sampling or an explicit paraphrase of r_ctx)? Since attribution is computed on the response tokens, I want to know how much of the signal is surface form.
  2. Does the method extend to free-form multi-sentence answers, where the label is not a clean binary and the attribution mass spreads over many positions?
Reviewer_YyaLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ReNA, a reference-contrast neuron atlas for detecting contextual hallucinations under context–parametric knowledge conflict. Using a frozen language model, it contrasts gradient-times-activation attribution for context-supported and conflicting parametric reference answers under the same prompt. Neurons with strong and consistent reference preferences are selected into a fixed atlas. A logistic-regression classifier then predicts the state of an actual response from its attribution features over these coordinates. Experiments on ConFiQA and constructed SQuAD2 conflict cases report improvements over several detectors. Additional interventions show that the selected coordinates can modulate answer preference, while activation patching does not establish that the atlas independently carries the source-selection decision.

Critical strengths

1、The paper addresses a well-defined problem with a coherent methodological pipeline. Contrasting two reference continuations under the same prompt is a reasonable way to discover coordinates that differentially support competing answers. The subsequent use of a fixed atlas and a lightweight classifier makes the detection mechanism relatively explicit and inspectable.

2、The analysis includes meaningful controls and appropriately qualified causal conclusions. The matched random atlases help distinguish signal enrichment from the predictive capacity of arbitrary neuron subsets. The intervention experiments further examine whether the selected coordinates influence answer preference. Importantly, the authors report the negative sufficiency results and avoid claiming that the atlas forms a self-contained source-selection circuit.

Critical weaknesses

1、Knowledge-source effects are not sufficiently separated from construction- and answer-dependent attribution differences. The reference contrast fixes the context–question prompt, but the two continuations still differ in their tokens, teacher-forced prefixes, familiarity, likelihood, and potentially length and style. These factors can affect gradient–activation attribution independently of whether the model is relying on contextual or parametric information. This concern is particularly relevant to the constructed SQuAD2 evaluation. Cases are selected according to whether the context-conditioned answer switches to the correct answer or retains an incorrect closed-book answer, while other response types are excluded. Such filtering creates a relatively clean binary distinction, but may also induce regularities associated with answer identity or response confidence. Consequently, the current evidence does not sufficiently establish whether the atlas captures stable knowledge-source participation or features correlated with the construction of these categories. The cross-domain experiment provides some support for reusing neuron identities, but it retrains the classifier on the target domain and does not resolve this distinction. Stronger paired controls over answer wording and source role would materially strengthen the central interpretation.

2、The validation scope is too limited to establish robustness and practical value beyond the studied setting. All detection experiments use two sizes of the LLaMA-2-Chat family, and the intervention analysis uses only the 7B model. The evaluations focus on short-answer conflict cases rather than more natural RAG responses containing mixed, partial, or novel unsupported claims. Moreover, the model-specific construction of SQuAD2 cases means that results across model sizes do not establish behavior on an identical set of conflict examples. This leaves open whether the method yields stable and useful findings across different model families and construction choices. Its practical value is also difficult to assess without latency and memory measurements: detecting each completed answer requires an additional attribution pass with backpropagation, despite the sparsity of the final feature vector. Finally, Appendix C.4 states that the 13B experiments disable the directional-consistency filter. Under the stated selection rule, the full method would then appear equivalent to magnitude-only selection with the same budgets, yet Table 2 reports different results. This discrepancy needs to be resolved before the corresponding ablation evidence can be interpreted confidently.

Questions for authors

1、What evidence distinguishes source-selective attribution from answer-content and construction effects? In particular, can the identified patterns remain predictive under controlled changes to answer wording, prompt formulation, or the source role of matched answer alternatives? Please clarify what the existing experiments establish about this distinction.

2、How robust and practical is the method beyond the current model-specific short-answer setting? Evaluation on a common cohort across different model families, together with attribution latency and peak-memory measurements, would materially affect my assessment. Please also reconcile the disabled consistency filter in the 13B configuration with the different magnitude-only ablation results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.