- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper asks whether a hallucination-detection score that predicts errors also identifies the mechanism that causes them. It proposes an audit that checks whether the target is observable, whether it can be read out from internal representations, whether the predictive support is reproducible, and whether editing that support has a specific intervention effect. The main finding is that a signal can be predictive and internally readable without supporting a causal or intervention claim, and the evidence chain stops at different points across models and answer mappings.
Critical strengths
S1. The paper addresses an important issue in hallucination detection: high prediction accuracy does not necessarily mean that the causal mechanism underlying model hallucinations has been identified.
S2. The authors propose a relatively systematic auditing framework for evaluating the predictive power, feature stability, and intervention effects of internal representations.
S3. The paper combines simulation experiments with experiments on real LLMs to show that predictive features do not necessarily have reliable causal interpretations. The experimental results are informative and thought-provoking.
Critical weaknesses
W1. The paper mainly shows that existing evidence is insufficient to support causal interpretations, but it lacks a positive experiment on real LLMs that successfully validates a known causal mechanism.
W2. The experiments mainly use smaller-scale models and tasks that provide candidate answers. Validation on large-scale LLMs (such as Qwen-8B) and in free-form text generation settings is lacking.
W3. Several analyses are retrospective and reuse previously inspected sources (e.g., the four-codebook extension), and the multi-codebook comparisons in Table 7 are nominal without multiplicity correction. The paper should clarify which conclusions depend on these exploratory comparisons and report corrected or sensitivity analyses.
W4. In the simulation experiments, the proposed method uses additional intervention information while the baselines do not, making the comparison between methods insufficiently fair.
Questions for authors
see weakness
2 comments
Your comments on retrospective analysis and information access led us to revise the statistics and add a fairer comparison. The latter also changed the claim we make about the full rule.
W3. Reused sources and multiple comparisons
The original four-codebook extension is retrospective. Appendix F.1 reconstructs the specified 16-cell family using shared source-bootstrap draws and Bonferroni-adjusted percentile intervals. Means reproduce the cached estimates; the original semantic A/B intervals remain compatible with zero and relevant small effects. Those small effects remain unresolved.
We now retain the analysis chronology and distinguish original confirmation, retrospective expansion, development refits and numerical corrections. The multiplicity correction covers the stated family, while source-bootstrap intervals are conditional on the recorded supports and outcomes. Choices made through codebook selection, support refitting and the wider adaptive research history remain outside their coverage.
The 110-source confirmation fixes its amended endpoint, support, controls and analysis before new outcomes. The separate 192-source assay freezes a ten-comparison high-precision primary family. Native-logit sensitivity is post-hoc, with ten native contrasts and ten native-minus-projection differences in one secondary family; nine of ten native contrast intervals remain positive. The dose/behavior follow-up uses the opened 192 sources. Local source screening checks overlap in our records (Table 1; Appendices B, C, F and G).
W4. Equal-information comparison
We agree with your fairness concern. The original comparison left information access and rule design entangled. In the added replay, eight fixed rules receive the same development intervention ledger, anonymous candidate IDs, probe metrics and interchange information. Latent roles remain hidden until evaluation. Each rule receives the same acquisition budget, 10,800 block swaps and 300 probe fits per instance; selector CPU time is unmatched.
The key comparison across 64 stipulated-role instances is shown below. The model specifies the target path, favoring direct target-effect selection by construction. Table 16 and Appendix E.1 retain all eight rules and intervals.
| Selection rule | Precision | Recall | Exact-set recovery |
|---|---|---|---|
| Intervention-qualified rule | 1.000 | 0.969 | 0.938 |
| Direct target-effect top-two rule | 1.000 | 1.000 | 1.000 |
The simpler rule matches or exceeds the full rule under equal information. We have therefore withdrawn the claim of added selection benefit from the full rule.
The calibration's purpose is to test which evidence distinguishes known roles. Target-aligned intervention information separates stipulated semantic paths from predictive proxies and response bias; the simpler rule is sufficient for this result. Explicit causal correspondences are also central to Geiger et al., 2021. Our calibration uses them to evaluate the interpretation supported by each measurement.
What the revision contributes
The audit's value lies in the scientific distinctions it makes possible. Sections 3-5 and Table 4 show these distinctions on a positive local effect. The direction-source check separates support-space influence from predictive-coefficient function. Actual-answer evaluation measures agreement with the initial score, following Wang et al., 2024. RMS reveals changes concealed by signed means, and random references place those changes in context. Repair and degradation are measured at the behavioral endpoint.
The attached runner implements these checks, retains controls, zero directions and invalid outputs, and records the resulting interpretation. Its tests verify the implementation. The empirical studies supply the evidence about labels and causal effects. The main Gemma target is candidate support under supplied facts; Table 1 keeps auxiliary protocols and cross-model setting extensions distinct.
Your fairness request helped us distinguish the value of the evidence from the choice of selection rule. The revised contribution rests on the concrete changes in scientific interpretation above.
Thank you for recognizing the value of a systematic audit. We followed your suggestions with a real-model positive reference, larger-model free generation and sentence-level evaluation.
W1. Positive evidence in real language models
We use the published GPT-2 IOI heads [1] as an independently motivated external positive reference. Heads (9,6), (9,9), (10,0) and random controls are fixed before new outcomes. The supplement separates 16 development groups from 32 test groups using disjoint names/template indices. Excess swap effect over norm-matched random is 2.1266 [1.7365, 2.5334]. Actual generated answers also reveal non-target losses.
The reference demonstrates sensitivity to a literature-motivated local effect and reveals its non-target cost. This addresses the detection aspect of your request. Known-effect calibration under Gemma's different SAE/additive operator remains open (Appendix E.2).
The Gemma studies now provide positive local evidence as well. On 110 new sources, donor-minus-span-random responses are 0.0116 [0.0058, 0.0172] and 0.0077 [0.0034, 0.0120], using two-comparison-adjusted intervals. A separate frozen 192-source assay yields positive adjusted intervals for all ten primary query/evidence-versus-control contrasts. Evidence-only edits keep question and candidate fixed, establishing that the response persists without changing question wording. These tests establish local influence within the original predictive support (Sections 5.1-5.2; Appendices B-C).
Behavioral and preservation measurements then refine this finding. The 110-source signed means pass, while RMS exceeds the declared engineering band under both selected and random edits. In the 192-source follow-up, 3-11/1,536 verdicts change per target-direction/mapping/sign condition at 2%, including repairs and degradations. No degradation occurs in the specified year/title contexts, although the adjusted source-loss upper bound of 2.96% leaves the declared 2% requirement unresolved. The results separate local score effects, actual verdict changes and the stronger population preservation claim (Appendix G).
W2. Qwen3-8B and free-form answers
We agree that free generation is an important extension. We added Qwen3-8B with NF4 weights and BF16 compute, generating answers without supplied options. The SQuAD packet has 128 development, 64 calibration and 64 title-isolated test questions. Model-assisted review/adjudication gives these test labels (Appendix F.3).
With no confirmed error class, AUROC is undefined and detector fitting stops. We report this outcome as a setting extension.
The evidence-ablation development packet has 52 binary-evaluable outputs after final-label adjudication.
Entropy-minus-condition is 0.0447 [-0.0315, 0.1154], leaving its added value uncertain. A harder HotpotQA development packet identifies errors; a frozen independent detector test is still outstanding.
We also evaluate 80 factual sentences from 12 development responses. Two model-assisted reviews are reported separately. Corrected PCA-minus-position AUROC differences are -0.0116 [-0.0278, 0.0000] and 0.0051 [-0.0222, 0.0323]. This endpoint includes insufficient reference support alongside factual error, and the comparison leaves added detection value over position unestablished. The labels throughout these Qwen studies are model-assisted, not independently verified human gold.
Your suggestion led us to evaluate both free answers and factual sentences. We treat semantic labeling as part of that validation. The tables above report the resulting test-label composition and development detection comparisons directly.
[1] Wang et al. (2023), Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small. ICLR record.