acceptodds
Under review as a conference paper at ICLR 2027

What Does a Hallucination Score Tell Us? Auditing the Evidence from Prediction to Mechanistic Claims

Abstract

Internal representations offer a promising basis for detecting hallucinations in large language models and investigating how errors arise. Yet a score that predicts errors does not by itself establish which computation produced them or where an intervention should act. Connecting these claims requires tracing whether the observation, predictive support and intervention endpoint refer to the same target. We introduce a measurement-to-claim audit organized around three properties: risk observability, conditional quantitative readoutability and support identifiability. A separate intervention audit checks whether the output endpoint discriminates the target relation before testing selected-support effects against matched controls. Exact-prefix checks, candidate-conditioned readouts and support resampling establish the scope of predictive evidence; a controlled nonlinear calibration distinguishes known semantic-path supports from predictive proxies and response biases. Our main natural-model audit evaluates frozen supports on the same sources across four answer codebooks and both semantic mappings. Gemma retains internal predictive information across mappings, but output-endpoint discrimination and intervention specificity require separate evidence: even where both mapped endpoints discriminate the relation, a selected-support advantage over controls remains unestablished. A source-matched Qwen stress test stops earlier because none of its evaluated output endpoints reliably discriminates the relation, despite weaker internal readability. These results make the gap between prediction and mechanistic claims empirically traceable. The audit identifies which claim the evidence supports and which additional check is needed before a stronger interpretation is warranted.

Primary area:
interpretability and explainable AI
Keywords:
hallucination detection, mechanistic interpretability, internal representations, probing, causal intervention, evaluation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ZjNVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks whether a hallucination-detection score that predicts errors also identifies the mechanism that causes them. It proposes an audit that checks whether the target is observable, whether it can be read out from internal representations, whether the predictive support is reproducible, and whether editing that support has a specific intervention effect. The main finding is that a signal can be predictive and internally readable without supporting a causal or intervention claim, and the evidence chain stops at different points across models and answer mappings.

Critical strengths

S1. The paper addresses an important issue in hallucination detection: high prediction accuracy does not necessarily mean that the causal mechanism underlying model hallucinations has been identified.

S2. The authors propose a relatively systematic auditing framework for evaluating the predictive power, feature stability, and intervention effects of internal representations.

S3. The paper combines simulation experiments with experiments on real LLMs to show that predictive features do not necessarily have reliable causal interpretations. The experimental results are informative and thought-provoking.

Critical weaknesses

W1. The paper mainly shows that existing evidence is insufficient to support causal interpretations, but it lacks a positive experiment on real LLMs that successfully validates a known causal mechanism.

W2. The experiments mainly use smaller-scale models and tasks that provide candidate answers. Validation on large-scale LLMs (such as Qwen-8B) and in free-form text generation settings is lacking.

W3. Several analyses are retrospective and reuse previously inspected sources (e.g., the four-codebook extension), and the multi-codebook comparisons in Table 7 are nominal without multiplicity correction. The paper should clarify which conclusions depend on these exploratory comparisons and report corrected or sensitivity analyses.

W4. In the simulation experiments, the proposed method uses additional intervention information while the baselines do not, making the comparison between methods insufficiently fair.

Questions for authors

see weakness

2 comments

Thank you for recognizing the value of a systematic audit. We followed your suggestions with a real-model positive reference, larger-model free generation and sentence-level evaluation.

W1. Positive evidence in real language models

We use the published GPT-2 IOI heads [1] as an independently motivated external positive reference. Heads (9,6), (9,9), (10,0) and random controls are fixed before new outcomes. The supplement separates 16 development groups from 32 test groups using disjoint names/template indices. Excess swap effect over norm-matched random is 2.1266 [1.7365, 2.5334]. Actual generated answers also reveal non-target losses.

Correct completionsUneditedWith IOI deltas
IOI recipient name60/6447/64
Separate object-completion task42/6428/64

The reference demonstrates sensitivity to a literature-motivated local effect and reveals its non-target cost. This addresses the detection aspect of your request. Known-effect calibration under Gemma's different SAE/additive operator remains open (Appendix E.2).

The Gemma studies now provide positive local evidence as well. On 110 new sources, donor-minus-span-random responses are 0.0116 [0.0058, 0.0172] and 0.0077 [0.0034, 0.0120], using two-comparison-adjusted intervals. A separate frozen 192-source assay yields positive adjusted intervals for all ten primary query/evidence-versus-control contrasts. Evidence-only edits keep question and candidate fixed, establishing that the response persists without changing question wording. These tests establish local influence within the original predictive support (Sections 5.1-5.2; Appendices B-C).

Behavioral and preservation measurements then refine this finding. The 110-source signed means pass, while RMS exceeds the declared engineering band under both selected and random edits. In the 192-source follow-up, 3-11/1,536 verdicts change per target-direction/mapping/sign condition at 2%, including repairs and degradations. No degradation occurs in the specified year/title contexts, although the adjusted source-loss upper bound of 2.96% leaves the declared 2% requirement unresolved. The results separate local score effects, actual verdict changes and the stronger population preservation claim (Appendix G).

W2. Qwen3-8B and free-form answers

We agree that free generation is an important extension. We added Qwen3-8B with NF4 weights and BF16 compute, generating answers without supplied options. The SQuAD packet has 128 development, 64 calibration and 64 title-isolated test questions. Model-assisted review/adjudication gives these test labels (Appendix F.3).

LabelTest answers
Supported-correct62
Confirmed semantic error0
Unresolved2

With no confirmed error class, AUROC is undefined and detector fitting stops. We report this outcome as a setting extension.

The evidence-ablation development packet has 52 binary-evaluable outputs after final-label adjudication.

ScoreAUROC
Evidence condition only0.8676
NLL0.8947
Token entropy0.9123

Entropy-minus-condition is 0.0447 [-0.0315, 0.1154], leaving its added value uncertain. A harder HotpotQA development packet identifies errors; a frozen independent detector test is still outstanding.

We also evaluate 80 factual sentences from 12 development responses. Two model-assisted reviews are reported separately. Corrected PCA-minus-position AUROC differences are -0.0116 [-0.0278, 0.0000] and 0.0051 [-0.0222, 0.0323]. This endpoint includes insufficient reference support alongside factual error, and the comparison leaves added detection value over position unestablished. The labels throughout these Qwen studies are model-assisted, not independently verified human gold.

Your suggestion led us to evaluate both free answers and factual sentences. We treat semantic labeling as part of that validation. The tables above report the resulting test-label composition and development detection comparisons directly.

[1] Wang et al. (2023), Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small. ICLR record.

Your comments on retrospective analysis and information access led us to revise the statistics and add a fairer comparison. The latter also changed the claim we make about the full rule.

W3. Reused sources and multiple comparisons

The original four-codebook extension is retrospective. Appendix F.1 reconstructs the specified 16-cell family using shared source-bootstrap draws and Bonferroni-adjusted percentile intervals. Means reproduce the cached estimates; the original semantic A/B intervals remain compatible with zero and relevant small effects. Those small effects remain unresolved.

We now retain the analysis chronology and distinguish original confirmation, retrospective expansion, development refits and numerical corrections. The multiplicity correction covers the stated family, while source-bootstrap intervals are conditional on the recorded supports and outcomes. Choices made through codebook selection, support refitting and the wider adaptive research history remain outside their coverage.

The 110-source confirmation fixes its amended endpoint, support, controls and analysis before new outcomes. The separate 192-source assay freezes a ten-comparison high-precision primary family. Native-logit sensitivity is post-hoc, with ten native contrasts and ten native-minus-projection differences in one secondary family; nine of ten native contrast intervals remain positive. The dose/behavior follow-up uses the opened 192 sources. Local source screening checks overlap in our records (Table 1; Appendices B, C, F and G).

W4. Equal-information comparison

We agree with your fairness concern. The original comparison left information access and rule design entangled. In the added replay, eight fixed rules receive the same development intervention ledger, anonymous candidate IDs, probe metrics and interchange information. Latent roles remain hidden until evaluation. Each rule receives the same acquisition budget, 10,800 block swaps and 300 probe fits per instance; selector CPU time is unmatched.

The key comparison across 64 stipulated-role instances is shown below. The model specifies the target path, favoring direct target-effect selection by construction. Table 16 and Appendix E.1 retain all eight rules and intervals.

Selection rulePrecisionRecallExact-set recovery
Intervention-qualified rule1.0000.9690.938
Direct target-effect top-two rule1.0001.0001.000

The simpler rule matches or exceeds the full rule under equal information. We have therefore withdrawn the claim of added selection benefit from the full rule.

The calibration's purpose is to test which evidence distinguishes known roles. Target-aligned intervention information separates stipulated semantic paths from predictive proxies and response bias; the simpler rule is sufficient for this result. Explicit causal correspondences are also central to Geiger et al., 2021. Our calibration uses them to evaluate the interpretation supported by each measurement.

What the revision contributes

The audit's value lies in the scientific distinctions it makes possible. Sections 3-5 and Table 4 show these distinctions on a positive local effect. The direction-source check separates support-space influence from predictive-coefficient function. Actual-answer evaluation measures agreement with the initial score, following Wang et al., 2024. RMS reveals changes concealed by signed means, and random references place those changes in context. Repair and degradation are measured at the behavioral endpoint.

The attached runner implements these checks, retains controls, zero directions and invalid outputs, and records the resulting interpretation. Its tests verify the implementation. The empirical studies supply the evidence about labels and causal effects. The main Gemma target is candidate support under supplied facts; Table 1 keeps auxiliary protocols and cross-model setting extensions distinct.

Your fairness request helped us distinguish the value of the evidence from the choice of selection rule. The revised contribution rests on the concrete changes in scientific interpretation above.

Reviewer_ykQhOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks what an internal hallucination score can establish beyond prediction. It follows the evidence from available information and predictive support to the interpretation of intervention effects. A controlled nonlinear model supplies known roles for the calibration study. The main Gemma experiment holds sources and selected supports fixed while changing answer codebooks and semantic mappings. Internal readouts remain informative, but output discrimination depends on the mapping, and selected-support edits show no established advantage under the discriminative A/B mappings. Qwen reaches a different limit at output discrimination. The contribution is an experimental way to locate gaps between predictive evidence and mechanistic claims.

Critical strengths
  1. Holding sources and edited supports fixed makes the answer-remapping result convincing. It shows how a patch effect can track token preference across opposing semantic mappings, exposing a concrete problem in the interpretation of interventions.

  2. The controlled nonlinear study includes known semantic paths, predictive proxies and response biases. Readers can see how the proposed checks distinguish these roles, which gives the broader argument a useful experimental foundation.

  3. The intervention comparisons use sign-balanced edits and matched controls. Aggregating by source respects the dependence between repeated prompts, and the discussion interprets intervals spanning zero appropriately.

Critical weaknesses
  1. The natural-model experiment lacks a positive control under the same intervention protocol. Its sensitivity to a relation-specific effect remains unknown, limiting what can be learned from the inconclusive selected-support contrasts.

  2. The multi-codebook extension reuses sources examined earlier. Its bootstrap intervals do not account for choices informed by that inspection or for refitting the support, so the extension offers limited independent confirmation.

  3. Qwen changes both the support family and the control construction, and none of its evaluated output endpoints reliably discriminates the relation. This provides a useful additional case, but leaves the Gemma intervention finding without a matched cross-model replication.

Questions for authors

1.What effect sizes remain plausible under the reported natural-model intervals?

2.Which findings are specific to the frozen support and edit procedure used here?

4 comments

Thank you for your careful assessment. Your effect-size question helped us replace a broad inconclusive label with a quantitative answer. We now distinguish the original test's limited sensitivity from positive local effects in the new frozen studies.

W1/Q1. Positive control, assay sensitivity and plausible effect sizes

For the original 42-source semantic endpoint, these are the 16-cell-adjusted intervals and conditional sensitivity estimates (Appendix F.2). Effects and MDEs are in logit-margin units.

MappingAdjusted interval80% MDEPower at effect 0.010
A=supported[-0.00600, 0.01947]0.0176about 21%
B=supported[-0.00470, 0.01686]0.0148about 35%

Zero, small negative effects and small positive effects of interest remain compatible with the original data. The calculation is specific to this endpoint.

The original-interface 1%/2%/4% development check, constructed gradient references and revised-interface dose studies are reported separately. Calibration against an independently known natural effect under the original Gemma SAE protocol remains open.

New local-effect evidence

On 110 new Gemma source groups, the fixed eight-coordinate support and readout yield AUROCs of 0.7659 and 0.7511. A separate frozen 192-source assay constructs evidence-only directions while holding the question fixed. Effects below are donorward logit-margin responses.

Assay and target-minus-random contrastA=supportedB=supported
110 sources, input-defined query donor0.0116 [0.0058, 0.0172]0.0077 [0.0034, 0.0120]
192 sources, evidence donor0.01727 [0.01546, 0.01903]0.00428 [0.00320, 0.00535]

The 110-source intervals are two-comparison-adjusted 97.5% intervals; the 192-source intervals adjust its ten-comparison family. This confirms local influence within the imposed support, with the donor construction and predictive-weight readout playing different roles (Sections 5.1-5.2; Appendices B-C).

All ten primary high-precision 192-source target/control comparisons have positive adjusted lower bounds. In the post-hoc native-logit analysis, nine of ten remain positive. Scaling and projection differ between studies, so each retains its own estimate; the original 42-source interval is unchanged (Section 5.2; Appendix C).

The opened 192-source follow-up evaluates all directions at 1%/2%/3%/4%; all 40 high-precision contrast intervals are positive after correction. Actual behavior is tested at 2%, where 3-11/1,536 verdicts change per condition. The dose results establish persistence of the local score response, while the generated-answer results show its limited behavioral reach (Appendix G).

What the preservation checks add

All 18 signed-mean checks pass in the 110-source study, yet four selected-direction RMS lower bounds exceed 0.05. RMS is in logit-margin units. Selected intervals adjust four comparisons; random intervals are descriptive.

Non-target task and mappingSelected RMS [98.75% CI]Random RMS [95% CI]
Year reading, A=supported0.0725 [0.0671, 0.0778]0.0704 [0.0680, 0.0726]
Year reading, B=supported0.0756 [0.0705, 0.0807]0.0701 [0.0674, 0.0727]
Code copying, A=supported0.0608 [0.0548, 0.0668]0.0623 [0.0580, 0.0665]
Code copying, B=supported0.0621 [0.0555, 0.0684]0.0623 [0.0580, 0.0665]

Mean preservation and input-level invariance give different answers on the same sources. Both families pass 0.10 in post-hoc sensitivity. The random reference places the engineering failure in context, while functional damage requires behavioral evidence. This target-change/preservation design is related to RAVEL (Section 5.3; Appendix B.4).

In the specified 192-source year/title contexts, all 16 target-direction cells retain 192/192 correct completions. We observe no degradation, although the adjusted source-loss upper bound of 2.96% leaves the declared 2% population guarantee unresolved.

The published IOI heads provide an external local-effect reference. On 32 new test groups, excess swap effect over norm-matched random is 2.1266 [1.7365, 2.5334], with actual-answer and object-task measurements reported together. This addresses the detection part of your positive-control request in GPT-2 (Appendix E.2; Wang et al., 2023).

Your question about support-specific findings is important for interpreting the positive result. We have separated what is confirmed on new sources from what is tested by refitting the support.

W2/Q2. Source reuse and findings conditional on frozen supports

The original four-codebook extension reuses inspected sources. Appendix F.1 now supplies shared source-bootstrap draws and a specified 16-cell multiplicity correction. These intervals describe uncertainty conditional on the recorded supports and outcomes. Codebook selection and other choices made after inspection remain outside that correction, so we identify the extension as retrospective.

The earlier 128-source collection has a separate batch-layout correction. An additional 28,700 forwards replay archived edits with zero replay error. Central interactions remain unchanged, while unilateral confidence estimates and two preservation decisions change. The corrected results retain their status as analyses of opened material.

The later 110- and 192-source protocols each freeze interfaces, supports, controls, source budgets and analysis before new model outcomes. Local ID/entity/title screening checks overlap in our records. The later dose and actual-output follow-up uses the opened 192 sources. Table 1 and Appendices B, C and G now make these distinctions visible.

The natural-model conclusion concerns local influence within the fixed eight-coordinate space under input-defined edits. Six development-only refits across two splits and K=4/8/16 retain native interactions of 0.0126-0.0340 on the opened 87 sources. Their amplitudes vary, and they repeat only part of the control set.

A separate 32/32-source development/test branch selects coordinates and predictive coefficients anew. Directly editing that fitted direction gives the following target-minus-random effects in logit-margin units (Appendix D).

MappingPredictive-coefficient direction minus random [interval]
A=supported0.0090 [-0.0016, 0.0193]
B=supported0.0089 [-0.0036, 0.0215]

Both intervals include zero. Its inputs and fitting rules differ from the donor study, so the two direction rules require separate interpretations.

New-source testing and support refitting answer different reproducibility questions. The former tests a fixed intervention; the latter examines dependence on support selection. Table 4 separates support-space influence from predictive-direction function. The refit results describe the selectors actually tested.

W3. Cross-model evidence

We agree that a matched replication would strengthen the evidence. It requires comparable support families, controls, operators and endpoints. Historical Qwen changes these components; the Qwen3-8B free-answer and RAVEL-style studies test broader settings. We have removed the replication implication and report each extension separately (Appendices E.2 and F.3).

The title-isolated 64-question 8B free-answer test has 62 supported-correct answers, no confirmed semantic errors and two unresolved outputs under model-assisted review. Detection AUROC is undefined because the test lacks confirmed errors. These labels are model-assisted rather than independently verified human gold. This free-answer study addresses a different endpoint from Gemma's intervention task.

The interface bridge reproduces all 336 legacy hashes and initial scores on the opened 42 sources. Score/verdict agreement changes as follows.

InterfaceA=supportedB=supported
Legacy51.19%61.90%
Revised100%100%

Endpoint alignment is itself an empirical question. Instructions, tokenization and prefill change together, so the improvement belongs to the revised interface as a whole. This addresses the score/text issue discussed by Wang et al., 2024 (Appendix C.1).

Thank you for the detailed response, especially the MDE sensitivity analysis and the new frozen 110- and 192-source experiments, which directly resolve my questions. I really like this work, keep my score at 4 (Clear acceptance), and will strongly support acceptance in the discussion.

Thank you for taking the time to read our response and for your support. Your questions helped us see where we needed to be more precise about what the original experiment could detect and what the new frozen experiments add. We're glad the additional analyses answered those questions and made the paper clearer.

Reviewer_7GSEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a “measurement-to-claim audit” for internal hallucination detectors, asking when predictive hidden-state signals justify stronger mechanistic or interventional claims. The audit tests whether the signal is observable at the right comparison unit, recoverable by a fixed held-out readout, and localized reproducibly. A separate intervention audit checks whether the output endpoint itself discriminates the target before comparing selected-support edits with matched random controls.

Empirically, the paper shows that pooled AUROC can hide weak within-question discrimination, while candidate-conditioned readouts on Gemma-2-2B remain strong. Predictive supports are reproducible beyond random baselines, though much of the overlap comes from shared coordinates. A toy example shows that AUROC-based localization can select proxies rather than causal features. The output-mapping audit further shows that stable internal readouts do not guarantee stable behavioral endpoints, and the answer-revision experiment finds that false “verified” feedback can make the model abandon correct answers.

Critical strengths
  • This paper studies an important and well-posed problem. Internal-probe work on hallucination often slides from “predicts errors” to “identifies the mechanism” or “tells us where to intervene.” Insisting that the target, comparison unit, interface and endpoint be declared and held fixed is a useful discipline for the field.
  • The output-mapping manipulation is a particularly useful methodological tool. Swapping which token means “supported” while keeping the underlying support fixed cleanly shows that a YES/NO endpoint can conflate the intended relation with token preference. Gemma’s AUROC of 0.369 for the NO=supported mapping is a strong illustration of this issue. The use of sign-balanced edits and the separation of the relation-specific interaction from the common shift are also good practices that many patching studies overlook.
  • This paper separates coordinate Jaccard, decoder-span overlap, and signed direction cosine, then shows that shared coordinates account for 0.583 of the 0.584 span overlap. This is a useful decomposition that helps avoid overinterpreting apparent “stable subspace” results.
Critical weaknesses
  • The three audit properties largely repackage established distinctions: information versus use in probing, answerability versus answer evaluation, and within- versus between-group discrimination. Proposition 1 follows directly from tie-aware AUROC, while Eq. 1 is a standard decomposition. The authors partly acknowledge this, noting that the goal is not to introduce “three independent theories,” so the paper’s contribution depends heavily on the empirical audit. That makes the experimental limitations below especially important.
  • No natural-model positive control, making the main null result hard to interpret. Every natural-model audit chain stops, so the paper never demonstrates that the full procedure can succeed on a real LLM. As a result, the null selected-versus-random contrast under A/B could reflect many simpler explanations, including a weak intervention dose (2% of state norm), unstable support selection (development OOF AUROC 0.645, bootstrap Jaccard 0.204), or limited power with only 42 source groups. The limitations section acknowledges that a same-protocol positive control remains unfinished, but this is arguably the key experiment needed to validate the audit itself. Dose-response analysis and a power analysis would also make the null result much easier to interpret.
  • The paper’s core principle of target continuity is not maintained across its own experiments. The abstract emphasizes tracing whether the observation, predictive support, and intervention endpoint refer to the same target, but the empirical sections use different targets, datasets, and protocols. Collision ceilings study future-branch correctness on TriviaQA; readout and geometry use exact-match correctness for supplied HotpotQA candidates; the intervention audit studies candidate–evidence relation judgments on 2Wiki; and answer revision uses a separate 2Wiki birth-date task. Models and layers also vary across experiments, and the appendix notes that some records should not be pooled. While the mapping audit does hold its target fixed within that experiment, the paper never traces one target end-to-end from observation through localization to intervention. As a result, the paper validates the pieces of the audit separately rather than demonstrating the integrated audit it proposes.
Questions for authors
  • Please report the raw YES-minus-NO margin interaction, with intervals, under both YES/NO mappings. If it is consistent across mappings, why is this not evidence of a relation-specific effect on the model’s native verdict channel? What is the principled argument for requiring the effect to survive an instructed mapping that the model demonstrably does not follow?
  • Why does Gemma PCA-32 select below random (1.19%) on natural candidates? How do you reconcile 0.939 AUROC on supplied candidates with internal readouts losing to length controls on natural ones?
  • How were the four supplied HotpotQA candidates constructed? How do surface baselines (candidate length, lexical overlap with the question or evidence) perform on that set? The external verifier’s 0.996 AUROC suggests the task may be easy in ways unrelated to hallucination.
2 comments

Thank you for the close reading. Your questions about target continuity and the native YES/NO channel led us to strengthen the natural-model case and recover positive evidence that our earlier interpretation had discounted.

W1/W3. Contribution and target continuity

We agree that information versus use, answerability versus answer evaluation, and within- versus between-question discrimination are established distinctions. Proposition 1 gives a tie-aware constraint; Equation 1 gives a standard decomposition. The contribution rests on what the executable audit reveals when these checks are applied to the same result. We have made that empirical contribution more explicit.

Your continuity concern was well founded. The Abstract, Introduction and Section 4 now center the Gemma case on candidate support under supplied evidence. The binary year task tests support versus contradiction under those facts, with historical truth and insufficient evidence outside its scope. Table 1 separates this case from the auxiliary TriviaQA, HotpotQA, geometry, simulation, IOI and Qwen audits.

On 110 new Gemma source groups, the original eight SAE coordinates and readout weights are fixed before outcomes. The edit direction comes from the input pair. The results separate predictive readout, local influence and actual correctness (Section 5.1; Appendix B).

MeasureA=supportedB=supported
Readout AUROC0.76590.7511
Query donor minus same-span random, logits0.0116 [0.0058, 0.0172]0.0077 [0.0034, 0.0120]
Score/verdict agreement, all prompts435/440429/440
Valid and correct verdict, all prompts285/440274/440

The effect intervals are two-comparison-adjusted 97.5% intervals. Scores track the model's answers much better than the answers track correctness.

A separate frozen assay on 192 further sources changes evidence years while holding question and candidate fixed. All ten primary contrasts are shown below, in donorward logit-margin units with family-adjusted 99.5% intervals (Section 5.2; Appendix C).

Direction contrastA=supportedB=supported
Query flip minus span random0.01624 [0.01443, 0.01804]0.00467 [0.00378, 0.00557]
Evidence flip minus span random0.01727 [0.01546, 0.01903]0.00428 [0.00320, 0.00535]
Query flip minus query paraphrase0.01383 [0.01174, 0.01592]0.00450 [0.00320, 0.00581]
Evidence flip minus evidence paraphrase0.01947 [0.01715, 0.02177]0.00695 [0.00551, 0.00841]
Evidence flip minus evidence order0.01566 [0.01367, 0.01761]0.00418 [0.00300, 0.00538]

The response persists without changing question wording. High precision applies to the final projection; transformer states remain BF16.

Both studies test local influence within the original fixed support. Their direction, scaling and projection rules differ, and the later dose/behavior analysis reuses the opened 192 sources.

Table 4 shows what the checks add. All 18 signed-mean preservation checks pass, yet four RMS bounds exceed 0.05, as do random references. The engineering criterion reveals input-level variation hidden by the mean. Behavioral measurements separately assess answer changes and non-target performance.

Q1. Raw YES-minus-NO margin under both instructions

You were right to ask for native margins. The raw YES-minus-NO selected-minus-random interactions are positive under both original instructions (Section 5.3; Appendix F.1).

Source groups and instructionInteraction [interval]
Original 42, first instruction0.055573 [0.030679, 0.080603]
Original 42, second instruction0.029591 [0.011016, 0.048284]
Separate frozen 87, aggregate0.020061 [0.011907, 0.028460]

We agree with your inference. The native-channel relation response is supported by these data. We have corrected our earlier interpretation. Mapping compliance matters for the stronger instructed-semantic claim. A constant shift shared by both relations cancels in the difference-in-differences interaction; target-dependent or nonlinear biases remain possible. We therefore report native and semantic A/B endpoints separately.

The opened 42-source interface bridge helps explain why endpoint validation matters. Legacy score/verdict agreement is 51.19%/61.90%, compared with 100%/100% under the revised interface. Because instructions, tokenization and prefill change together, this comparison evaluates the revised interface as a whole (Appendix C.1). It addresses the first-token/text concern raised by Wang et al., 2024.

We agree with the purpose of your positive-control request. A useful audit must detect effects as well as assess their interpretation. Your suggestions led to sensitivity tests and an important PCA correction.

W2. Positive control, intervention dose, support stability and power

For the original 42-source semantic endpoint, the 16-cell-adjusted intervals and conditional sensitivity estimates are as follows (Appendix F.2). Effects and MDEs use logit-margin units.

MappingAdjusted interval80% MDEPower at effect 0.010
A=supported[-0.00600, 0.01947]0.0176about 21%
B=supported[-0.00470, 0.01686]0.0148about 35%

Small positive effects remain compatible with the data. These are endpoint-specific estimates.

Appendix F.2 retains the original-interface 1%/2%/4% development check on 12 historical sources. The opened 192-source follow-up evaluates frozen directions at 1%/2%/3%/4%, with all 40 high-precision target-minus-control intervals positive after correction. The score response persists across this dose range. At 2%, actual verdict changes are much less frequent, affecting 3-11/1,536 prompts per target-direction/mapping/sign condition, with both repairs and degradations. In the specified year/title contexts we observe no degradation. The adjusted source-loss bound is 2.96%, leaving the declared 2% population preservation requirement unresolved (Appendix G).

The original OOF AUROC is 0.645 and bootstrap Jaccard is 0.204. Six development-only refits across two splits and K=4/8/16 yield native interactions of 0.0126-0.0340 on the opened 87 sources, with varying amplitudes. A separate 32/32-source development/test branch directly edits newly fitted predictive coefficients. Readout remains informative, while both direction-minus-random intervals include zero. These analyses test sensitivity to fitting and inputs; the 110/192 studies test the original fixed coordinates (Appendix D).

The external IOI reference fixes published heads (9,6), (9,9), (10,0) before outcomes, using disjoint development/test names and template indices. On 32 test groups, the excess swap effect is 2.1266 [1.7365, 2.5334] over norm-matched random heads. Actual correct-name completion changes from 60/64 to 47/64, and applying the deltas to object completion changes 42/64 to 28/64. The reference detects a literature-motivated local contribution and exposes its non-target cost. Known-effect calibration under the original Gemma SAE operator remains open (Appendix E.2; Wang et al., 2023).

Q2. Gemma PCA-32 on natural candidates

Your question revealed a mismatch between instruction-model verifier states used for fitting and base-model continuation states used for evaluation. We withdraw the old result as a same-condition counterexample. Corrected selection rates are shown below (Appendix F.1).

EvaluationQuestionsTop-1 selection
Old PCA-32, mismatchedhistorical pool1.19%
PCA-32, aligned model/prompt50614.03%
PCA-32, six overlapping sources removed50014.00%
Dense, same aligned 500 questions50010.20%

PCA-minus-length is 0.49 percentage points [-1.78, 2.77], leaving their relative performance uncertain.

The supplied task uses PCA-64 and a constructed four-option pool with oracle 1.000 and uniform accuracy 0.250. The natural task uses eight-candidate pools with exact-match oracle 0.2075. These differences in candidate availability, state acquisition and fitting explain why the two results must be evaluated under their own protocols. Natural-candidate exact match also measures answer matching rather than semantic error.

Q3. Supplied-option provenance and surface controls

Each supplied pool contains the official answer and three distinct answers from other questions in the same coarse answer-shape class. Evidence contains target support and sentences mentioning distractors; candidate and sentence orders are shuffled. All interfaces and controls use the same 500 questions and 2,000 candidates, fitting only on the original 160 source questions.

Character count, token count and position are near chance, but the stronger lexical control exceeds Dense on the same supplied pool.

InterfaceAUROCTop-1 selection
Mention-sentence/question Jaccard0.964 [0.952, 0.975]0.936 [0.916, 0.955]
Dense0.939 [0.923, 0.953]0.858 [0.826, 0.888]

These are 95% question-bootstrap intervals. The surface control changes how the high supplied-candidate AUROC should be interpreted. It identifies a lexical solution; Dense's reliance on it remains undetermined. Figure 4 and Appendix A.5 give the features. The external verifier's 0.996 uses a separate model/resource condition.

Comments on the paper

1 comment

We thank the reviewers and chairs for their careful reading and thoughtful feedback. Our paper asks when an error-predictive internal signal supports a claim about the model's computation. The contribution is an executable audit that shows which additional evidence changes the interpretation of the same result.

Across the three reviews, reviewers recognized the importance of this question and identified value in the measurement discipline, controlled comparisons and empirical analyses.

1. A useful measurement discipline and a systematic approach

"Insisting that the target, comparison unit, interface and endpoint be declared and held fixed is a useful discipline for the field." (Reviewer 7GSE)

"The authors propose a relatively systematic auditing framework for evaluating the predictive power, feature stability, and intervention effects of internal representations." (Reviewer ZjNV)

Keeping these choices explicit lets readers assess prediction, localization and intervention against the evidence each requires.

2. Controlled comparisons that clarify intervention effects

"The output-mapping manipulation is a particularly useful methodological tool." (Reviewer 7GSE)

"Holding sources and edited supports fixed makes the answer-remapping result convincing." (Reviewer ykQh)

Holding the support fixed makes the endpoint's role in an intervention claim testable. The mapping comparison examines the intended relation alongside answer-token preference.

3. Empirical analyses that change how predictive structure is interpreted

"This is a useful decomposition that helps avoid overinterpreting apparent “stable subspace” results." (Reviewer 7GSE)

"Readers can see how the proposed checks distinguish these roles, which gives the broader argument a useful experimental foundation." (Reviewer ykQh)

Shared coordinates account for 0.583 of 0.584 span overlap. Known-role calibration separates semantic paths, predictive proxies and response biases. In each case, an added check resolves an ambiguity left by predictive performance or aggregate overlap.

Responses and manuscript changes

A focused natural-model case. We study candidate support under supplied evidence in Gemma-2-2B-IT. Both frozen assays find local influence within the fixed predictive SAE support. Entries below are donorward logit-margin effects, with family-adjusted intervals, rather than accuracy gains.

Assay and contrastA=supportedB=supported
110 new sources, query donor minus random0.0116 [0.0058, 0.0172]0.0077 [0.0034, 0.0120]
192 new sources, evidence donor minus random0.01727 [0.01546, 0.01903]0.00428 [0.00320, 0.00535]

The 110-source intervals adjust two comparisons. The 192-source assay changes supplied years while holding question and candidate fixed; all ten primary high-precision contrasts exceed random, paraphrase or order controls after family adjustment. Each assay retains its own direction, scaling and projection rules. Table 1 and Sections 5.1-5.2 distinguish support-space influence from predictive-weight function.

  • A clearer account of what each check adds. Table 4 connects measurements to their interpretation, separating support-space influence, predictive-direction function, generated verdicts and non-target changes.

  • Positive reference and sensitivity. Fixed heads from the published IOI study yield excess swap effect 2.1266 [1.7365, 2.5334] over norm-matched randoms on 32 new test groups. Actual answers and non-target losses are reported together. This provides an external local-effect reference. MDE, support refits and opened-source dose analyses specify the sensitivity of the individual measurements (Appendices D-G).

  • Corrections and fairer comparisons. We report positive native YES-minus-NO responses, withdraw the mismatched PCA 1.19% as a clean counterexample, and include the stronger lexical baseline. Equal-information results favor the simpler rule, so we have removed the full-rule superiority claim. Qwen3-8B adds a free-generation setting extension.

  • An executable specification. The attached CPU runner, worked examples and source-level exports make the audit reproducible. Its 29 passing tests cover software behavior. The experiments report label provenance and intervention results separately (Section 3; Appendix B).

The requests for positive references and broader settings helped us distinguish sensitivity from scope. The audit preserves positive local effects while separating claims about predictive support, computation and behavior. We appreciate the reviewers' help in making this value concrete.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.