acceptodds
Under review as a conference paper at ICLR 2027

HDIR: Training-free Hallucination Detection via Discriminative Internal Representations

Abstract

Large language models have achieved remarkable success in natural language processing, but they remain prone to hallucinations, generating plausible yet factually incorrect content. Effective hallucination detection is therefore crucial for assessing the factuality of generated content. Existing methods mainly rely on output signals or internal representations: the former are efficient and training-free but overlook richer semantic information encoded in model states, while the latter can better exploit such information but often require additional training. To address these limitations, we propose Hallucination Detection based on Discriminative Internal Representations (HDIR), a training-free framework that exploits internal representations for hallucination detection. Our analysis shows that the norm of attention-weighted internal representations provides a strong discriminative signal for hallucination detection. We define this measure as Semantic Intensity and further propose a Cohen's -based selection strategy to identify the most discriminative attention heads and dimensions, thereby strengthening the discriminative signal of Semantic Intensity. Extensive experiments across multiple benchmarks and LLMs demonstrate that HDIR achieves strong overall hallucination detection performance, generalizes well across datasets, and complements output-level uncertainty signals, with their combination yielding further performance gains.

Primary area:
interpretability and explainable AI
Keywords:
Large Language Models, Hallucination Detection, Internal Representations, Representation Analysis, Trustworthy AI

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_mwwMOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes HDIR for hallucination detection. It averages middle-layer hidden states with attention weights, takes the ℓ2 norm, and uses Cohen's d with greedy AUROC search on a labeled analysis set to pick a few heads and dimensions. It is evaluated on five QA datasets with five 3B to 8B models. However, the reported baseline results are highly questionable and suggest a problem in the experimental pipeline.

Critical strengths

The question of detecting hallucinations without sampling is practically relevant, and the evaluation covers several model families and datasets.

Critical weaknesses
  1. The experimental results are not reliable. I am not suggesting any misconduct, but the setup clearly has serious problems. Semantic Entropy and SAPLMA are both strong baselines in my experience, yet here Semantic Entropy averages about 0.51 AUROC, falls below 0.5 on three models and to 0.42 on PopQA. The supervised SAPLMA probe stays at 0.53 to 0.62. PTrue is 0.41 on Llama3.1-8B, which means its score direction is inverted. A trained probe losing to one or two neurons selected with the same kind of labels is not plausible unless the training data or implementation differ. The paper does not say what SAPLMA and MIND were trained on. Since HDIR's advantage is measured against these numbers, the main comparison cannot be trusted. I strongly recommend the authors re-examine the whole experimental setup before resubmitting.

  2. The method is not training-free. It requires 1,000 labeled samples per model, with labels from a 72B judge, and selects heads and dimensions by maximizing AUROC on them. This is supervised feature selection and needs the same kind of data as probe-based methods, which removes the distinction the paper is built on. A linear probe trained on the same layer with the same labels is the obvious baseline and is missing.

  3. The design has no clear grounding. The final score is the absolute value of one to five neurons (Table 5, e.g. a single dimension for Llama3.1-8B), and nothing in the paper shows what these neurons represent, so "Semantic Intensity" is just a name. The selection is also unstable. Changing the judge changes the selected dimensions (Table 7), selecting on 200 target-dataset samples gives near-chance AUROC on both Llama models (Table 3), and the Normalized Gap variant falls to 0.43 on Mistral. Since the attention weights sum to one, the representation is an average over positions whose norm may simply track sequence length or attention-sink mass, which is not examined.

  4. Even taking the numbers at face value, the problem is not solved. AUROC is 0.69 to 0.80, and on Llama3.1-8B HDIR is below simple length-normalized entropy (0.7346 vs 0.7551).

Questions for authors
  1. Please check the implementation and sign convention of Semantic Entropy and PTrue, and state what data SAPLMA and MIND were trained on.

  2. How does a logistic-regression probe on the same middle-layer representation, trained on the same 1,000 labeled samples, compare with HDIR?

  3. How does Semantic Intensity correlate with response length and with attention mass on the first token? Does HDIR remain effective after controlling for length?

1 comment

The authors did not provide any response during the rebuttal phase, so I am maintaining my score.

Reviewer_hkdgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes Hallucination Detection based on Discriminative Internal Representations (HDIR), a framework that exploits internal representations for hallucination detection. It identifies the most discriminative attention heads and dimensions to produce discriminative signals for hallucination detection. HDIR is evaluated across five QA datasets and five open-weight LLMs.

Critical strengths
  • The task addressed in this paper is important, and the experiments cover multiple datasets and multiple open-source LLMs.
  • The authors conducted ablation studies, which validate the positive impact of their selection of attention heads and dimensions.
Critical weaknesses
  • The authors claim that HDIR is "training-free," but this description is not entirely accurate. Although HDIR does not involve training a classifier, it utilizes 1,000 labeled samples to select attention heads and dimensions—a process that essentially leverages supervised information. Furthermore, since the selection of these heads and dimensions is critical to the effectiveness of HDIR, certain implementation details remain unclear. For instance, "A judge model assesses response correctness as ground truth for evaluation", how accurate are the labels the judge model provides? Regarding the statement, "For each dataset, the target LLM generates responses to the questions following the test set," what specific questions are being referred to, and is there any overlap with the test set? (This is a crucial point; otherwise, there is a risk of data leakage.) Additionally, the paper should provide experimental results for a simple supervised probe baseline trained on the same labeled data.

  • The paper fails to sufficiently describe its novelty relative to recent internal-state detectors. Existing methods—such as MIND and various prompt-guided internal-state methods—already leverage hidden states or signals derived from attention mechanisms to achieve efficient hallucination detection. As it stands, the contribution of this paper appears limited to a specific aggregated statistic combined with a supervised selection process. Consequently, the authors need to provide a clearer explanation of the paper's novel contributions.

Questions for authors

None

Reviewer_DtuKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes HDIR, a hallucination detector that works from a single forward pass. At the middle layer, it aggregates hidden states using per-head token weights, obtained by averaging each head’s attention matrix over the query dimension. The l_2 norm of the result (“Semantic Intensity”) is the core signal, and its reciprocal serves as the hallucination score. The authors observe that hallucinated generations have lower Semantic Intensity than factual ones, most strongly in intermediate layers. To sharpen the signal, they select a small subset of attention heads and hidden dimensions by ranking components by Cohen’s d on a labeled analysis set (1000 samples per model, labeled by a judge LLM), then greedily keeping the subset that maximizes analysis-set AUROC. The evaluation covers five open-domain QA datasets and five 3B–8B LLMs. HDIR reports average AUROCs of roughly 0.69–0.80, generally above output-based baselines (LNPE, SelfCheckGPT, Semantic Entropy, SEU, PTrue) and representation-based baselines (SAPLMA, MIND, INSIDE). Fusing HDIR with LNPE gives further gains. Appendices cover alternative pooling schemes, code hallucination, an alternative judge, layer configurations, and cross-model transfer.

Critical strengths
  1. The method is simple and cheap: one generation pass, no sampling, no gradient training, and the final detector reads only a handful of scalars.
  2. The paper is clearly written and easy to reproduce. The selected heads and dimensions are reported (Tables 5 and 7).
  3. The appendix analyses go beyond the main results: alternative representations, layer choice, judge robustness, cross-model transfer, and a code-generation sanity check.
  4. HDIR appears complementary to output-level uncertainty, and the fusion results are a useful practical finding.
Critical weaknesses
  1. The “training-free” framing is misleading, and the most important baseline is missing. Heads and dimensions are chosen by greedily maximizing AUROC on 1000 labeled examples. This is supervised feature selection, which amounts to fitting a very low-capacity classifier. The natural comparison is a linear probe (e.g., logistic regression) trained on the same 1000 examples at the same layer, on either the attention-pooled or mean-pooled representation. Without it, the reader cannot tell whether the norm-based score adds anything over standard probing given the same supervision. The paper also does not say what data and which layers SAPLMA and MIND were trained on. If these supervised baselines saw less or different data than HDIR’s analysis set, the comparison is not controlled.

  2. Several baseline numbers are implausibly low. Semantic Entropy falls below 0.5 AUROC in many settings (e.g., 0.479 on Llama3.1-8B, 0.416 on PopQA), and PTrue scores 0.41 on Llama3.1-8B. Both are worse than chance and well below values typically reported for these methods on similar QA benchmarks. The trained probe SAPLMA scoring about 0.55 is also surprising. Possible causes include the low sample count (5 samples), the short generation budget, the clustering or NLI setup, or score orientation. Whatever the cause, these numbers weaken the main comparison. The paper cites, but does not compare against, stronger recent methods such as HaloScope, EigenScore-style metrics, and semantic-entropy probes.

  3. Plausible confounds are not ruled out, and “Semantic Intensity” may not be semantic.

  • Eq. (1) averages causal attention over queries. Early positions, and especially BOS/attention-sink tokens, receive systematically large weights, and these tokens are known to carry massive activations in a few specific dimensions. The score may therefore mostly reflect sink-token attention mass and massive-activation dimensions. That the optimum is reached with 1–5 dimensions is consistent with this explanation.
  • The sequence appears to include the prompt. The signal may then reflect the model’s familiarity with the question or entity rather than properties of the generated answer, which would be consistent with the especially strong PopQA result.
  • Response length directly changes T and the weight distribution, but it is not controlled for.

Ablations that exclude sink tokens, restrict pooling to response tokens, and control for length are needed before the semantic interpretation is justified.

  1. The selection procedure is unstable, and the paper overstates its robustness. Appendix D claims substantial dimension overlap across judges. Comparing Tables 5 and 7, however, the selected dimensions for Llama3.1-8B (3947 vs. 3524, 4081) and Phi-4-mini are completely disjoint, and Phi-4-mini’s head set shrinks from four heads to one. Table 3 shows Same-Dataset selection collapsing to about 0.53 on both Llama models. Table 2 shows the Normalized Gap variant dropping to 0.43 on Mistral-7B. Nested greedy selection (heads, then dimensions, then subset size) on 1,000 samples carries a real overfitting risk. No variance across seeds or analysis-set resamples is reported.

  2. The statistical evidence is limited. Each test set has only 500 questions, generations are capped at 64 tokens (essentially short-form QA), and no confidence intervals or significance tests are given. Many reported differences, including HDIR vs. LNPE and the fusion variants, are 0.01–0.03 AUROC and may fall within noise. Figure 1 plots only mean differences with unnormalized scales, which says little about separability. In Table 8 the chosen middle-layer configuration is not the best variant on several models, and HDIR is below LNPE on Llama3.1-8B.

Questions for authors
  1. How does HDIR compare with a logistic-regression probe trained on the same 1,000-sample analysis set at the same middle layer?
  2. Does the signal persist when BOS/sink tokens are excluded and pooling is restricted to response tokens only?
  3. Can you report variance (e.g., bootstrap CIs or multiple analysis-set resamples) for the selected components and the resulting test AUROC?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.