- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes HDIR for hallucination detection. It averages middle-layer hidden states with attention weights, takes the ℓ2 norm, and uses Cohen's d with greedy AUROC search on a labeled analysis set to pick a few heads and dimensions. It is evaluated on five QA datasets with five 3B to 8B models. However, the reported baseline results are highly questionable and suggest a problem in the experimental pipeline.
Critical strengths
The question of detecting hallucinations without sampling is practically relevant, and the evaluation covers several model families and datasets.
Critical weaknesses
-
The experimental results are not reliable. I am not suggesting any misconduct, but the setup clearly has serious problems. Semantic Entropy and SAPLMA are both strong baselines in my experience, yet here Semantic Entropy averages about 0.51 AUROC, falls below 0.5 on three models and to 0.42 on PopQA. The supervised SAPLMA probe stays at 0.53 to 0.62. PTrue is 0.41 on Llama3.1-8B, which means its score direction is inverted. A trained probe losing to one or two neurons selected with the same kind of labels is not plausible unless the training data or implementation differ. The paper does not say what SAPLMA and MIND were trained on. Since HDIR's advantage is measured against these numbers, the main comparison cannot be trusted. I strongly recommend the authors re-examine the whole experimental setup before resubmitting.
-
The method is not training-free. It requires 1,000 labeled samples per model, with labels from a 72B judge, and selects heads and dimensions by maximizing AUROC on them. This is supervised feature selection and needs the same kind of data as probe-based methods, which removes the distinction the paper is built on. A linear probe trained on the same layer with the same labels is the obvious baseline and is missing.
-
The design has no clear grounding. The final score is the absolute value of one to five neurons (Table 5, e.g. a single dimension for Llama3.1-8B), and nothing in the paper shows what these neurons represent, so "Semantic Intensity" is just a name. The selection is also unstable. Changing the judge changes the selected dimensions (Table 7), selecting on 200 target-dataset samples gives near-chance AUROC on both Llama models (Table 3), and the Normalized Gap variant falls to 0.43 on Mistral. Since the attention weights sum to one, the representation is an average over positions whose norm may simply track sequence length or attention-sink mass, which is not examined.
-
Even taking the numbers at face value, the problem is not solved. AUROC is 0.69 to 0.80, and on Llama3.1-8B HDIR is below simple length-normalized entropy (0.7346 vs 0.7551).
Questions for authors
-
Please check the implementation and sign convention of Semantic Entropy and PTrue, and state what data SAPLMA and MIND were trained on.
-
How does a logistic-regression probe on the same middle-layer representation, trained on the same 1,000 labeled samples, compare with HDIR?
-
How does Semantic Intensity correlate with response length and with attention mass on the first token? Does HDIR remain effective after controlling for length?
The authors did not provide any response during the rebuttal phase, so I am maintaining my score.