- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies why RAG models can still produce incorrect answers despite being given sufficient supporting evidence. It compares support-only and full-context conditions and uses probing and hidden-state patching to analyze internal states associated with answer changes. Experiments across multiple models and datasets show that interventions on pre-generation hidden states can repair or disrupt model outputs.
Critical strengths
- Broad and systematic experimental design. The paper goes beyond probing and includes layerwise scoring, residual patching, component-level intervention, and free-generation evaluation.
- Strong controls and cross-dataset validation. It includes early-layer, question-end, identity write-back, and random-source controls, and evaluates frozen probes and intervention locations on additional datasets.
- Careful evaluation and statistical analysis. The paper reports detailed sample filtering, bootstrap confidence intervals, and multiple-testing correction.
Critical weaknesses
- The paper does not fully answer its central question of why models fail despite sufficient evidence. The results mainly show that late hidden states differ between correct and incorrect conditions and that replacing these states can change the answer. This identifies where answer-related differences become consequential, but not why evidence utilization fails in the first place.
- The main patching result may be partly expected from the intervention design. The intervention is applied to the residual state immediately before answer generation, often at a late layer where substantial information about the upcoming answer is already encoded. Transferring a state from a correct-answer run may therefore amount to transferring an already-formed answer decision rather than revealing a mechanism specific to evidence utilization.
- The connection between probing and causal intervention is weak. The final intervention location is selected based on patching effects rather than probe performance alone, and attention/MLP outputs are intervened on without corresponding probing analyses. Moreover, question-end probes achieve relatively high AUROC, while patching the same position yields very low flip rates, showing a clear gap between decodability and causal efficacy.
- The Regression-vs.-Rescue probe may largely reflect signed correctness differences. Since the two groups are defined by opposite correctness transitions across the two contexts, a probe on may simply learn the sign of the difference between two correctness-related readouts rather than a distinct transition mechanism.
- The novelty over prior latent-intervention work is not sufficiently clear. Giannetti et al. already study RAG failures where correct evidence is present but distractors lead to incorrect answers, and show that latent interventions can recover correct generations. The paper should clarify what new mechanistic insight it provides beyond this prior work.
- UNKNOWN responses do not establish the absence of parametric knowledge. Because the closed-book prompt explicitly instructs the model not to guess and to output UNKNOWN when confidence is low, repeated UNKNOWN responses only show that no explicit confident answer is observed. They do not rule out latent or low-confidence parametric knowledge, and therefore do not fully exclude knowledge conflict.
Questions for authors
Please see the weaknesses.