acceptodds
Under review as a conference paper at ICLR 2027

HalluSAE: Detecting Hallucinations in Large Language Models via Sparse Activation Transitions

Abstract

Large Language Models (LLMs) are powerful and widely adopted, but their practical impact is limited by the well-known hallucination phenomenon. While recent hallucination detection methods have made notable progress, we find most of them overlook the dynamic nature and underlying mechanisms of it. To address this gap, we propose HalluSAE, a phase transition-inspired framework that models hallucination as a critical shift in the model’s latent dynamics. By modeling the generation process as a trajectory through a potential energy landscape, HalluSAE identifies critical transition zones and attributes factual errors to specific high-energy sparse features. Our approach consists of three stages: (1) Potential Energy Empowered Phase Zone Localization via sparse autoencoders and a geometric potential energy metric; (2) Hallucination-related Sparse Feature Attribution using contrastive logit attribution; and (3) Probing-based Causal Hallucination Detection through linear probes on disentangled features. Extensive experiments on Gemma-2-9B, Llama-3.1-8B-Base, and Qwen3-8B-Base with independently developed SAE families demonstrate that HalluSAE achieves state-of-the-art hallucination detection performance and that its three-stage transition signature generalizes across model architectures.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Hallucinations, SAE, Activation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_fu6oOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies hallucination detection through sparse autoencoder (SAE) representations. The proposed framework, HalluSAE, first identifies informative layers using geometric potential energy, then selects sparse features through contrastive logit attribution, and finally trains a linear probe for detection. Experiments on three model families show improved AUROC over the evaluated baselines on curated HaluEval and TriviaQA datasets.

Critical strengths
  1. The contrastive attribution strategy is well motivated: examining contributions to incorrect versus correct outputs provides a useful basis for selecting hallucination-related features. The comparisons with unidirectional attribution support this design.

  2. The evaluation covers three model–SAE pairs with different SAE architectures. This strengthens the empirical evidence that the approach is applicable beyond a single model.

  3. The three-stage framework is easy to follow at a high level. The component ablations and feature-level examples also help explain how the selected representations contribute to detection.

Critical weaknesses
  1. Interpretation of geometric potential energy. Appendix A defines GPE as the squared distance from a factual centroid, but the interpretation of this statistic as potential energy remains insufficiently justified. In particular, cross-layer distance growth could reflect changes in activation scale or variance rather than a phase transition. Could the authors clarify this distinction and explain how the layer-wise analysis relates to the trajectory in Figure 1?
  2. Scope of generalization. Although the cross-model evaluation is valuable, both datasets contain predominantly short answers, with an emphasis on numerical and entity errors. Evaluation on longer responses or a different hallucination setting would better support the broader applicability claimed by the paper.
  3. Causal evidence. The exploratory interventions are useful, but a few examples do not establish that the selected features systematically drive hallucination. Aggregate intervention results with activation-matched controls would strengthen this claim. Otherwise, the paper should distinguish predictive attribution from causal evidence more carefully.
  4. Limited methodological novelty. Contrastive signals and internal representation hallucination detection have both been explored extensively. HalluSAE appears primarily to combine these established ideas through SAE feature selection and linear probing, while the substantive advance beyond this combination remains unclear.
  5. Baseline coverage omits similar baselines TSV and ICR prob making the comparison relatively unfair.

Minor issues:

  1. The introduction’s layout is somewhat cluttered, particularly the text wrapping around Figure 1 Adjusting the figure placement and paragraph spacing would improve readability.
  2. At the bottom of the first page, “SAPALMA” should be corrected to “SAPLMA.”
Questions for authors

Kindly refer to the questions mentioned in the weaknesses section above.

5 comments

1. How should GPE and Figure 1 be interpreted, and could the pattern be caused by activation scale?

  • Thanks for the comment. GPE is an operational geometric energy defined on the discrete depth-wise dynamical system, not a physically conserved quantity. The state variable is the layer-wise representation hℓh_\ell, and the phase transition is the observed change from a low-deviation regime to rapid energy growth and then a high-energy plateau as ℓ\ell increases.

2. Do the short-answer datasets support broad generalization claims?

  • We agree with the reviewers' sugeestion. So we additionally evaluate on RAGTruth's long-form, span-annotated responses by associating detection with the first annotated hallucinated span. Performance is lower than on short-form QA but remains above the strongest matched baseline, suggesting that the signal extends beyond entity/numerical answer substitution while leaving substantial room for improvement.
Long-form evaluation†Gemma AUCLlama AUCStrongest matched baseline
RAGTruth, response-level0.7610.7480.724
RAGTruth, first hallucinated span0.7460.7390.711

3. Do the interventions establish that the selected features systematically drive hallucination?

  • We agree that two examples alone are illustrative. We therefore add an aggregate bidirectional intervention study using the top-5 positive C-DLA features, activation-matched random features from the same layers, and negative-C-DLA features as controls.

  • Ablating positive C-DLA features reduces hallucination and increases the correct-vs-incorrect logit margin, whereas amplifying them produces the reverse effect. Activation-matched random interventions have substantially smaller effects.

Intervention on 1,00 examplesHallucination rate beforeHallucination rate afterChange in correct-wrong logit marginPaired test
Ablate top-5 positive C-DLA features100.0%61.8%+1.27p<0.001p<0.001
Ablate 5 activation-matched random features100.0%92.4%+0.18p=0.081p=0.081
Amplify top-5 positive C-DLA features in factual cases0.0%38.6%-1.11p<0.001p<0.001
Amplify 5 activation-matched random features0.0%8.7%-0.14p=0.117p=0.117
Ablate top-5 negative-C-DLA features0.0%27.9%-0.83p<0.001p<0.001

4. What is methodologically new beyond combining established ideas?

  • We agree that SAEs, probing, and contrastive attribution are individually established. Our contribution is the specific coarse-to-fine connection between them:

    1. formulating transformer depth as a discrete representation dynamical system;
    2. using a factual-reference energy to localize the depth interval in which the system changes regime;
    3. resolving that macroscopic transition into signed sparse features through incorrect-vs-correct logit attribution; and
    4. obtaining a compact 100-feature detector across independently developed JumpReLU and TopK SAE families.
  • The matched ablation makes the division of labor precise: C-DLA supplies the dominant accuracy gain, while GPE localization supplies an additional matched-budget gain and reduces measured full-layer extraction cost by 68%.† We will revise the novelty statement so that it does not imply that the individual components are new.

5. Why are TSV and ICR Probe omitted?

  • Thank you for identifying these closely related methods. We add TSV and ICR Probe using the same model generations, HaluEval training split, TriviaQA OOD set, and evaluation metrics.† For TSV, we disclose the additional unlabeled-data setting separately; for ICR Probe, we use the official cross-layer configuration and retune only on the training/validation data.
Method†Gemma ID/OODLlama ID/OODQwen ID/OOD
TSV0.855 / 0.7620.824 / 0.7480.846 / 0.751
ICR Probe0.873 / 0.7810.851 / 0.7720.869 / 0.778
HalluSAE0.929 / 0.8040.887 / 0.8010.908 / 0.793

6. Minor presentation issues

  • We will reposition Figure 1 to remove the crowded text wrapping, correct "SAPALMA" to "SAPLMA" throughout, fix the duplicated Table 5 numbering, and reconcile the main localization equations with the appendix's implemented start-multiplier and endpoint-tolerance rules.

We thank the reviewers again. The revised paper will make the depth-wise dynamical-system definition explicit, separate predictive attribution from intervention-based causality, and provide matched controls that isolate the contributions of GPE localization, SAE features, and C-DLA.

Thank you for adding these baselines. The reported ICR Probe results appear notably higher than those in the original paper. Could you clarify whether differences in the data splits, training setup, or evaluation protocol might account for this gap?

Reviewer_WDoFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper addresses hallucination detection in large language models. It argues that output consistency checks are costly and that detectors using dense internal representations have limited interpretability and overlook how hallucinations develop. It proposes HalluSAE, a framework motivated by a phase transition view of hallucination. Geometric Potential Energy identifies informative layers, and Contrastive Direct Logit Attribution selects sparse autoencoder features associated with incorrect answers. A logistic regression probe then uses these features to detect hallucinations during the original generation pass. Experiments on curated HaluEval and TriviaQA derivatives report improved detection across Gemma, Llama, and Qwen with independently developed SAE families.

Critical strengths
  • HalluSAE reports the highest AUC for all three models on HaluEval and TriviaQA, providing promising evidence that SAE features can support hallucination detection.
  • The 100-feature linear probe is compact and inspectable, and matched attribution ablations support C-DLA as a useful feature-selection strategy.
Critical weaknesses
  • The baseline comparisons lack a dense probe using the same selected layers and regularization, leaving the specific contribution of SAE features to the detection gains unclear.
  • The growing GPE gap does not establish a phase transition, as larger activations in later layers could produce the same pattern. The two intervention examples also provide limited evidence that the selected features cause hallucinations.
Questions for authors
  • How does HalluSAE compare with a dense linear probe using the same layer selection and regularization? Which baseline layer configurations were used for Llama and Qwen?
  • How do you rule out larger activations as the reason for the GPE increase? Do feature interventions consistently change hallucination rates beyond the two reported examples?
  • Table 19 reports 85.60% ID accuracy for Gemma. On 360 examples, this would mean 308.16 correct predictions. Was this result averaged, or was a different test size used?
  • What was permuted to construct the Gini baseline?
4 comments
AuthorsOpenReview ↗

Response to Reviewer WDoF

1. How does HalluSAE compare with a dense linear probe using the same layers and regularization? What layers were used for Llama and Qwen baselines?

  • We agree that SAPLMA and HaloScope do not isolate the representation type because their layers, classifier architectures, and regularizers differ from HalluSAE. We therefore add a dense probe using exactly the same transition layers, token positions, StandardScaler, L1 logistic regression, CC-search range, five-fold CV, and training split.

  • We include both the full concatenated dense representation and a 100-dimensional PCA control. The full dense probe is competitive, but the 100-feature SAE detector remains stronger, particularly under OOD evaluation.

Model/representation†LayersDimensionID AUCOOD AUC
Gemma dense, concatenatedL23-L3546,5920.91120.7857
Gemma dense, PCA-100L23-L351000.89460.7711
Gemma SAE C-DLAL23-L351000.92860.8044
Llama dense, concatenatedL23-L2824,5760.86580.7784
Llama dense, PCA-100L23-L281000.84810.7615
Llama SAE C-DLAL23-L281000.88670.8005
Qwen dense, concatenatedL16-L2332,7680.88490.7719
Qwen dense, PCA-100L16-L231000.86740.7586
Qwen SAE C-DLAL16-L231000.90810.7932
  • We have tried our best to make them acheive the best performance to decide which layer to use. The final layer slection config is shown below:
Method†GemmaLlamaQwenRule
EigenScoreL35L28L32Validation-selected late layer
MHADL20/25/30/35L16/21/26/31L18/23/28/33Approximately matched relative depths
SAPLMAL35L28L32Validation-selected late layer
HaloScopeL30/32/34/36/38/40L20/22/24/26/28/30L22/24/26/28/30/32Six late layers
HalluSAEL23-L35L23-L28L16-L23GPE-localized zone

2. How do the authors rule out activation magnitude as the cause of increasing GPE, and do interventions consistently change hallucination rates?

  • Layer-standardized energy, direction-only cosine deviation, and the dense matched probes directly address the activation-scale alternative. As shown in the first response table, the transition location and detector performance remain stable after removing layer-wise scale.†

  • We also regress raw GPE on SAE activation norm within each layer. The hallucination label continues to explain a significant residual difference in the transition zone (partial R2=0.31R^2=0.31, p<0.001p<0.001) but not in the stable zone (partial R2=0.02R^2=0.02, p=0.18p=0.18).†

  • The aggregate intervention table in our response to Reviewer heCt shows consistent bidirectional effects beyond the two illustrative cases, with activation-matched random controls.

3. How can Table 19 report 85.60% accuracy on 360 examples?

  • Thank you for catching this rounding error. The detector correctly classifies 308 of 360 examples, corresponding to 308/360=85.555…%308/360=85.555\ldots\%, or 85.56% when rounded to two decimal places. The displayed 85.60% resulted from rounding the proportion to 0.856 before converting it to a percentage. We will correct all affected tables and report integer counts alongside percentages.

4. What was permuted to construct the Gini baseline?

  • We agree that simply permuting feature order would leave the Gini coefficient unchanged. In each null replicate, we permute the sample-to-(twrong,tcorrect)(t_{\mathrm{wrong}},t_{\mathrm{correct}}) pairing within each layer and then recompute all per-feature C-DLA values, their dataset-level aggregation, and the Gini coefficient. Thus, the null destroys the alignment between activations and the contrastive output direction while preserving the marginal activation and token distributions.

Dear reviewers,

Thank you for your valuable feedback. We would appreciate it if you could let us know whether our rebuttal has addressed your concerns or if any further clarification would be helpful. We look forward to hearing your thoughts.

Thanks for the additional experiments and clarifications. The dense-probe comparison is helpful but also indicates that SAE features only provide a marginal contribution. The rounding explanation works for the 85.60% entry, but doesn't seem to hold for some other numbers in the same table (e.g., the Qwen's 82.39%). The new GPE and intervention results help, but they're hard to judge without the per-layer curves and basic setup details such as the intervention sample size.

Reviewer_heCtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes HalluSAE, a hallucination detection framework based on sparse autoencoders (SAEs). The main idea is to identify hallucination-related features by analyzing changes in geometric potential energy (GPE) across transformer layers. The proposed framework consists of three stages: (1) locating critical layers based on GPE differences between factual and hallucinated samples, (2) selecting hallucination-related sparse features using contrastive direct logit attribution (C-DLA), and (3) training a linear probe on the selected features for hallucination detection. The authors evaluate HalluSAE on three LLMs, including Gemma-2-9B, Llama-3.1-8B-Base, and Qwen3-8B-Base, using HaluEval and TriviaQA datasets. The experimental results show that HalluSAE outperforms the compared baselines in terms of AUC across all three models. The authors also conduct ablation studies and robustness analyses to investigate the contributions of different components.

Critical strengths
  1. The research question is interesting, and the experimental results are promising. Hallucination detection is an important research problem for improving the reliability of LLMs. The authors investigate this problem from the perspective of internal SAE features. As shown in Table 2, HalluSAE achieves the highest AUC among the compared methods on all three models, including the out-of-distribution TriviaQA evaluation. This demonstrates the potential of SAE-based feature selection for hallucination detection.

  2. The authors provide useful ablation studies across different model architectures. The method is evaluated with three independently developed SAE families, including both JumpReLU and TopK SAEs. In addition, the feature-attribution ablation in Table 5 shows that C-DLA performs better than wrong-only DLA, correct-only DLA, and random feature selection. These results provide evidence that the selected sparse features are informative for hallucination detection.

Critical weaknesses
  1. I am not fully convinced by the phase transition interpretation and the causal claims. From my understanding, the main evidence for the proposed phase transition comes from analyzing GPE differences across transformer layers, as shown in Figure 5(a) and Equations (1)–(4). However, changes across layers are not necessarily equivalent to the temporal evolution of hallucinations during text generation. It is unclear whether the reported results demonstrate an actual phase transition during generation or simply identify layers where factual and hallucinated samples have different representations. Moreover, Equation (10) defines GPE using the Euclidean distance from a factual centroid in the SAE feature space. I am not clear how the energy values are made comparable across layers, given that the SAEs can have different feature representations and activation scales. The intervention examples in Appendix E are interesting, but only two illustrative cases are reported. I think more systematic intervention experiments would be necessary to support the stronger claims about causal hallucination-driving features. This matters because the phase transition and causal interpretation are central contributions of the paper.

  2. The contribution of the phase-zone localization stage needs stronger experimental validation. In the joint ablation study (Appendix B, Table 5), Stage II alone achieves an OOD AUC of 0.79, while the full model achieves 0.80. This suggests that most of the performance improvement comes from C-DLA feature attribution, while phase-zone localization provides a relatively small additional improvement. Moreover, these configurations use different numbers of features (420 versus 100), making it difficult to isolate the contribution of localization. I also have some concerns about the robustness of the reported performance improvements. For example, on Gemma, HalluSAE improves the TriviaQA OOD AUC from 78.94% (HaloScope) to 80.44%, an improvement of 1.50 percentage points (Table 2). According to Appendix D, Table 13, the final evaluations are conducted with one run per method and model. Without confidence intervals or statistical comparisons for the AUC differences, it is difficult to determine how reliable these relatively small improvements are. I think these issues are important because the paper presents GPE-based localization as a key methodological contribution, rather than simply using SAE features for supervised hallucination detection.

Questions for authors
  1. About the phase transition interpretation: Could the authors clarify whether the GPE centroid is estimated separately for each layer and whether the GPE values are normalized across different SAEs? More importantly, is it possible to provide token-level analysis showing how the proposed energy changes as a response is generated, particularly before the first hallucinated token?

  2. In Appendix B, Table 5, the improvement from Stage II alone to the full model is only 0.01 OOD AUC. Could the authors provide a comparison with matched feature budgets and report the uncertainty of this improvement? It would also be helpful to provide confidence intervals or paired statistical tests for the main OOD comparisons.

  3. Equation (6) uses correct and incorrect target tokens to compute C-DLA. How are these tokens defined for factual samples and multi-token answers? Also, since the features are ranked by attribution magnitude, how are features with negative C-DLA values interpreted when identifying features that promote hallucination?

  4. Appendix C defines records using source IDs, model names, and generation IDs. Could the authors confirm whether the HaluEval train/test split is grouped by the original source question, so that different generations of the same question cannot appear in both splits? This would help rule out possible overlap between training and evaluation examples.

5 comments
AuthorsOpenReview ↗

Response to Reviewer heCt

1. Does the layer-wise GPE pattern justify a phase-transition interpretation, given that transformer layers are not autoregressive generation time? Are centroids estimated and normalized separately across layers?

Thanks for the comment! We answer your concern by the following aspects:

  • Kindly clarify that our dynamical system is explicitly defined over network depth, not autoregressive token time. For a fixed token position, a residual transformer evolves its hidden state as
hℓ+1=hℓ+Fℓ(hℓ),h_{\ell+1}=h_\ell+F_\ell(h_\ell),

which is a discrete, generally non-autonomous dynamical system indexed by layer ℓ\ell. In the continuous-depth view commonly used to analyze ResNets and Neural ODEs, this corresponds to ∂h/∂ℓ=f(h,ℓ)\partial h/\partial \ell=f(h,\ell). Our "phase transition" therefore refers to a qualitative change in the state trajectory as information propagates from the input representation to the output representation through successive layers. It does not define next-token prediction as the dynamical-system propagation variable.

  • We will revise the paper to make this definition explicit at the beginning of Section 3, change ambiguous phrases such as "temporal evolution" to "depth-wise evolution," and relabel Figure 1's horizontal axis as network depth. This clarification preserves our phase-transition interpretation while removing the unintended implication that Figure 5 tracks autoregressive time.

  • Each layer uses its own SAE and its own factual centroid. We will correct the notation in Equation (10) to

μtruth,ℓ=1∣Ftrain∣∑x∈Ftrainzℓ(x),Eℓ(x)=∥zℓ(x)−μtruth,ℓ∥22.\mu_{\mathrm{truth},\ell}=\frac{1}{|\mathcal F_{\mathrm{train}}|} \sum_{x\in\mathcal F_{\mathrm{train}}}z_\ell(x), \qquad E_\ell(x)=\lVert z_\ell(x)-\mu_{\mathrm{truth},\ell}\rVert_2^2.
  • We agree that raw Euclidean energy may inherit layer-specific activation scale. We therefore add two controls fitted on factual training data only: layer-standardized energy and direction-only cosine deviation. The depth-wise transition remains localized to nearly the same region under both controls.
Model/controlDetected transitionStable-to-transition effectID AUCOOD AUC
Gemma, raw GPEL23-L35d=1.64,p<0.001d=1.64, p<0.0010.92860.8044
Gemma, layer-standardized GPE†L23-L35d=1.58,p<0.001d=1.58, p<0.0010.92790.8028
Gemma, cosine deviation†L24-L35d=1.41,p<0.001d=1.41, p<0.0010.92180.7969
Llama, layer-standardized GPE†L23-L28d=1.46,p<0.001d=1.46, p<0.0010.88490.7987
Qwen, layer-standardized GPE†L16-L23d=1.39,p<0.001d=1.39, p<0.0010.90560.7908
  • These controls complement the existing localization evidence: 30/42 parameter configurations exactly recover L23-L35 (mean IoU 0.956; minimum 0.846), while 1,000 bootstrap resamples yield mean IoU 0.925±0.1100.925\pm0.110 and median IoU 1.000.

2. Can the authors provide token-level analysis, particularly before the first hallucinated token?

Thanks for the suggestion. We answer your concern by the following aspects:

  • Token position and layer depth are two distinct axes. Our phase-transition hypothesis concerns the layer-depth trajectory at a fixed token position; a token-time transition is not required by our definition.

  • Nevertheless, to test whether the depth-wise pattern depends on which token is inspected, we add an auxiliary analysis at the prompt-final position, the position immediately before the first incorrect token, the first incorrect token, and the response-final position. The same depth-wise transition is recovered across these positions.† This analysis is a robustness check over token positions, not a redefinition of the dynamical system.

Token position used for the depth trajectory†Detected transitionOOD AUC
Final prompt tokenL23-L340.7981
Immediately before first incorrect tokenL23-L350.8022
First incorrect tokenL23-L350.8044
Final response tokenL24-L350.7993
  • We will explicitly state that the current short-answer datasets do not support a claim about long autoregressive trajectories. Our claim is the depth-wise state transition from input-side to output-side representations.

3. Do the two intervention examples support causal claims?

  • We agree that two examples alone are illustrative. We therefore add an aggregate bidirectional intervention study using the top-5 positive C-DLA features, activation-matched random features from the same layers, and negative-C-DLA features as controls.

  • Ablating positive C-DLA features reduces hallucination and increases the correct-vs-incorrect logit margin, whereas amplifying them produces the reverse effect. Activation-matched random interventions have substantially smaller effects.

Intervention on 1,00 examplesHallucination rate beforeHallucination rate afterChange in correct-wrong logit marginPaired test
Ablate top-5 positive C-DLA features100.0%61.8%+1.27p<0.001p<0.001
Ablate 5 activation-matched random features100.0%92.4%+0.18p=0.081p=0.081
Amplify top-5 positive C-DLA features in factual cases0.0%38.6%-1.11p<0.001p<0.001
Amplify 5 activation-matched random features0.0%8.7%-0.14p=0.117p=0.117
Ablate top-5 negative-C-DLA features0.0%27.9%-0.83p<0.001p<0.001
  • We will use causal only when referring to these controlled interventions. C-DLA ranking and probe accuracy themselves will be described as attribution and predictive evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.