acceptodds
Under review as a conference paper at ICLR 2027

Spot the Difference: Detecting Hallucinations through Context-Head Intervention and Sparse Autoencoder Feature Contrasts

Abstract

RAG aims to enhance factuality in LLMs, yet these models frequently generate unfaithful outputs that contradict or exceed the provided context. While the internal representations of the LLM often contain features sufficient to flag hallucinations, the additive nature of the residual stream allows ingrained parametric knowledge to progressively outweigh contextual information during inference. In this work, we investigate this internal conflict, termed the Knowledge-Context Gap (KCG), in the sparse autoencoder (SAE) feature space. We show that jointly modeling paired SAE features provides complementary signals for hallucination detection, though constructing the paired representations via prompt-level manipulation incurs computational overhead and representational drift. To address these challenges, we propose RAGap, a mechanistic framework that characterizes KCG by isolating the LLM's parametric knowledge directly within the residual stream. By identifying and suppressing specialized context heads through differentiable masking, RAGap enables an efficient characterization of the latent divergence between parametric knowledge and contextual information. Leveraging SAEs for interpretable feature extraction, our method consistently outperforms state-of-the-art hallucination detectors across multiple benchmarks. Our findings provide a novel mechanistic perspective on RAG faithfulness and offer a practical solution for reliable, inference-time hallucination detection.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Hallucination Detection, Retrieval-Augmented Generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_2ufMOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper studies hallucination detection in RAG. The authors call the conflict between the given context and the model's own memory the Knowledge-Context Gap. The idea is to compare two internal states of the model: one with the context, and one without it. The paper first shows that removing the context from the prompt and comparing the two sets of SAE features helps detection. But this needs an extra forward pass, and it may change the model's internal computation. So the authors propose RAGap. Instead of removing the context from the prompt, it learns which attention heads read the context, and masks them inside the model. It then extracts SAE features from both states and trains a classifier on them. Experiments on RAGTruth, Dolly, and TofuEval show better results than existing detectors.

Critical strengths
  1. The idea of comparing the states with and without context is intuitive. Masking context heads is also cheaper than running the model again without the context.

  2. The evaluation covers several benchmarks and models, and RAGap outperforms existing detectors in most settings.

Critical weaknesses
  1. It is unclear whether the gap signal actually matters. The ablations show that removing the gap features changes the results by only about 0.01. Also, the heads are masked at only one layer, after earlier layers have already read the context. So the two states may be very similar. This is my main concern.

  2. It is unclear where the gains come from. RAGap uses a stronger classifier than baselines such as RAGLens. Without a control that uses the same classifier on features from the normal state only, the gains may come from the classifier rather than from the gap.

Minor: The motivation in the introduction could be stronger. It is not fully clear why modeling the gap is needed beyond existing SAE-based detectors. In addition, some baseline numbers in Table 2 are identical for the 7B and 13B models. Please check them.

Questions for authors
  1. If you train the same classifier only on features from the normal state (with context), how does it compare with RAGap?

  2. How different are the features before and after masking the context heads? How does masking random heads compare with masking the selected context heads?

I think this paper has room for improvement. Fully addressing the concerns above and presenting a clearer story would make the paper much stronger.

Reviewer_uLSQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes RAGap, a supervised detector of unfaithful RAG responses. It uses differentiable masking to identify attention heads, suppresses their outputs to construct an alternative internal representation, and jointly classifies the original and intervened SAE features together with their interactions. Experiments across multiple models and benchmarks report improved detection performance and lower latency than prompt-level context removal.

Critical strengths
  1. The efficiency performs well. The local intervention reuses upstream computation and reduces detection latency by 27.3–32.0% relative to prompt-level context removal (Table 4).
  2. The evaluation covers multiple model families and RAG tasks. Improvements over the closely related RAGLens baseline show some useful evidence of practical detection value.
Critical weaknesses
  1. The claim of knowledge–context isolation is not established. As described in Section 4.2, the context and query concern different subjects, yet head discovery rewards the context object over the query’s answer. This could identify contextual distraction rather than faithful context use. Moreover, suppressing selected heads at one layer leaves earlier context-dependent representations intact. These gaps limit the main contribution.
  2. The specificity of the proposed intervention can not be found in the ablations. Masking all heads is a substantially stronger perturbation than masking a selected subset. The AUROC gains over this control in Table 3 are only 0.003–0.008. Furthermore, the “w/o KCG” variant removes the second representation, so it does not isolate the contribution of the interaction terms. The results therefore leave open whether the gains arise from the proposed mechanism or simply from adding a useful perturbed representation.
Questions for authors
  1. That will be helpful if the authors can provide a complete head-discovery prompt. With different subjects in the context and query, why is the context object the discovery target? Is any subject substitution or prompt transformation omitted from the description?
  2. Does RAGap outperform random-head interventions matched for head count and perturbation magnitude? What direct evidence shows that the intervention suppresses contextual influence while preserving parametric knowledge retrieval?
  3. How does head discovery across all layers translate into inference requiring only one additional FFN computation? Please specify the intervention layer, selected heads, and computations actually repeated.
Reviewer_JHKwOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents RAGap, a mechanistic framework that characterizes the conflict between LLMs' parametric knowledge and the input context, termed Kownledge-Context Gap (KCG). The framework identifies context heads through differentiable masking and supresses them to obtain KCG estimation. The proposed method outperforms prior art in hallucination detection on two LLMs and two benchmarks, while reducing detection latency by 27-32% compared to prompt-level KCG charaterization.

Critical strengths

S1. The detection performance is impressive.
S2. The design is intuitive.

Critical weaknesses

W1. The mechanistic claim is yet to be supported by the experiments, without discussion on head-level intervention for hallucination mitigation.
W2. The $ \lambda $ is not defined and discussed as a hyperparameter. The experimental details missed its value.
W3. The LRE dataset used for differentiable masking contains many-to-one relations, which is not suitable for identifying context heads if used without filtering.

Questions for authors

Please address Weaknesses.

1 comment

Thank you for your responses, W2 and W3 has been resolved.

Two questions remain:

  1. I noticed that the Llama2-13B numbers reported in the paper (0.8624) are different from the original numbers from RAGLens (0.8964), could you explain why?
  2. Does increasing context head outputs or supress its complement increase the model's ability to focus on the in-context information?

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response and Summary of Revisions

We thank Reviewers 2ufM, uLSQ, and JHKw for their helpful feedback during the discussion. We have updated the paper with all additional experiments and clarifications. All are in the revised PDF (changes in blue): classifier and interaction ablations in Section 5.3 (Table 3) and Appendix F (Tables 11–14), paired-feature analyses in Appendix A.5 (Tables 5–6), head-discovery prompts and selected heads in Appendices B.2, C.2, and G (Tables 7–9, 15), and clarified text in Sections 1, 3, 4, 6, and Appendix I.1.

1. Matched-classifier comparisons and interaction ablation (2ufM W1/W2/Q1, uLSQ W2). In Section 5.3 (Table 3) and Appendix F (Tables 11–13), we compare RAGap against classifiers using only normal-state features (h^+\hat{h}^+) across three backbones and three datasets:

  • Scaling from Single-branch (h^+\hat{h}^+) to Dual-branch (h^+\hat{h}^+) changes average Macro-F1 very little (from 0.7061 to 0.7070).
  • Under the same classifier architecture and parameter count, replacing h^+\hat{h}^+ with h^−\hat{h}^- in the second branch (Full) raises average Macro-F1 to 0.7211 and improves AUROC, Accuracy, and Macro-F1 across all evaluated models and datasets.
  • Removing the difference and product terms (w/o interaction) lowers all three metrics across every setting, supporting the contribution of explicit interaction terms when both representations remain available.

2. Head-selection specificity and feature differences before and after masking (2ufM W1/Q2, uLSQ W2/Q2).

  • Equal-cardinality masking (Appendix F, Table 14): Fixing the number of masked heads to ∣Sctx∣=14|\mathcal{S}_{ctx}| = 14 on Dolly (Llama2-7B), Accuracy and F1 increase from 0.8681 / 0.8676 to 0.8720 / 0.8714 as overlap with discovered context heads increases from 20% to 80%.
  • Feature differences on RAGTruth (Appendix A.5, Table 5): Head-level intervention preserves greater representational similarity than prompt-level removal (0.9252 vs. 0.8744 cosine, 0.2998 vs. 0.4941 L2), while the L2 distance and incomplete activation overlap (0.8443 Jaccard) show that h^+\hat{h}^+ and h^−\hat{h}^- remain distinguishable rather than collapsing into identical representations.
  • Conflict sensitivity (Appendix A.5, Table 6): Across 575 LRE facts answered correctly without context by Llama-3.2-1B, counterfactual contexts yield a larger mean absolute SAE difference than aligned contexts (1.2434 vs. 1.2243).

3. Context transmission, earlier-layer representations, and inference details (JHKw W2/W3, uLSQ W1/Q1/Q3).

  • Section 4.2 and Appendix B.2 (Table 7) clarify same-object filtering (o1≠o2o_1 \neq o_2), direct concatenation without subject substitution, head-discovery prompts, and why probing contextual information transmission via contextual distraction locates the shared routing heads.
  • Sections 4.1–4.3 and Appendix A.4 clarify the trade-off of leaving earlier context-dependent representations intact to preserve FFN key activations, correct "across all layers" to discovery only at the designated intervention layer, and detail why inference requires only a single-layer FFN recomputation (Eq. 10).
  • Appendices C.2 and G (Tables 8–9, 15) report λ=0.1\lambda = 0.1, the intervention layers and suppressed head indices for Llama2-7B and Llama2-13B, and the effect of λ\lambda (λ=0.1\lambda = 0.1 vs. 0.40.4) on head selection.

4. Detection vs. mitigation, motivation, and baseline check (JHKw W1, uLSQ W1, 2ufM Minor). We clarify our focus on hallucination detection rather than mitigation, make the two-step progression from Section 3 (prompt-level manipulation) to Section 4 (head-level intervention) explicit in both section openings, strengthen the motivation in Section 1, soften wording around "isolating parametric knowledge" (Abstract, Section 6, Appendix I.1), and update LN-Entropy with clarified baseline sources in Table 2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.