- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies contextual hallucination in RAG, where the model ignores the given context and answers from its own memory. The paper has three parts. First, it builds a detector. For each generated answer, it measures how much each FFN neuron contributes to the output, and trains a sparse linear classifier on these scores. Second, it runs a causal analysis on the 134 neurons that the detector selects most often. These neurons do affect whether the model follows the context, but they explain only a small part of this behavior. Third, it uses the detector for mitigation. It samples several candidate answers and picks the one the detector rates as most faithful. Experiments use three models and four datasets. The detector achieves the best AUROC in all 12 settings, and mitigation improves the faithful rate over single-pass baselines.
Critical strengths
-
The causal analysis is careful and honest. It uses matched control neurons and proper statistical tests. The authors clearly report a negative result: these neurons are involved, but they are not enough to explain the behavior.
-
The results are consistent. Both detection and mitigation improve across three models and four datasets, and mitigation does not change the model weights.
Critical weaknesses
-
Simple detection baselines are missing. In ConFiQA, the faithful answer usually appears in the context, while the memory-based answer does not. So a simple check of whether the answer appears in the context may already work very well. A linear probe on hidden states is also missing. In addition, an existing method, SEP, already performs close to the proposed detector, and it even has higher Macro-F1 in 7 of the 12 settings. Overall, the advantage of the detector is not clear. This is my main concern.
-
The mitigation comparison is not fair. The proposed method generates up to 16 candidates and reranks them, while all baselines decode only once. So the gain may come from sampling more candidates, not from the attribution signal. A fair comparison should use the same candidate pool with other simple scoring methods, and should report the extra compute cost.
-
The setting is narrow. All tasks are short-answer QA with entity conflicts, and the models are relatively old (Llama-2 and Mistral-7B).
Questions for authors
-
For detection, could you compare with two simple baselines? The first is a rule that checks whether the generated answer appears in the context. The second is a linear classifier trained on hidden states.
-
Your mitigation method generates several candidate answers and picks one. Could you pick from the same candidates using other simple methods, and compare the results? Please also report how much extra compute your method needs, since the baselines only generate one answer.
Overall, the current experiments do not show that the attribution signal is better than simple alternatives.