- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Framed by Goodhart's Law, this paper treats three uncertainty-based hallucination detection metrics (First Output Token Entropy, Semantic Entropy, and EigenScore) as operational proxies for the underlying construct. Through three targeted perturbation modes and a set of ordinary-variation studies, conducted across four model configurations and four QA datasets, the paper shows that metric decisions can systematically diverge from factuality and semantic consistency, demonstrating the consistency of this phenomenon.
Critical strengths
- Well-posed research questions with an attack design that accurately reflects them. RQ1 (whether targeted perturbations expose gaps between metrics and human judgements) and RQ2 (whether such gaps also arise under ordinary variation without optimization intent) target the most fragile point in hallucination detection's path from benchmark correlation to deployment reliability, and both problems genuinely exist. Modes 1/2/3 respectively target the structural assumptions of each metric: first-token locality, NLI-based clustering, and representational dispersion, covering both false-negative and false-positive directions; the diagnostic framework and the problem formulation are closely connected.
- Restrained claims and clear boundaries. The authors clearly distinguish confabulations from stable errors, score shifts from threshold crossings, and construct-level from implementation-level failures, and repeatedly emphasize that ASR measures proxy–construct divergence under a diagnostic protocol rather than deployment risk, giving the paper a well-delimited line of argument.
- Experimental coverage and documentation completeness. The combination of three metrics, four models, and four datasets, the per-mode success criteria, and the human audit with reported IAA statistics give the paper good reproducibility and value.
Critical weaknesses
- The models, datasets, and baseline metrics are all dated. The model lineup remains at LLaMA-2-Chat, Mistral-v0.2, and Qwen2-72B; the three attacked metrics, FTE, SE, and EigenScore, are likewise works from several years ago; and the datasets (SQuAD, TriviaQA, NQ, PopQA) are similarly old. I recommend that the authors extend their evaluation to more recent models, methods, and datasets.
- The failure analysis proceeds metric by metric, lacking evidence of generalization to other hallucination detection methods. Each of the three modes exploits the specific assumptions of a particular implementation, e.g., FTE's first-token locality, SE's DeBERTa clustering, and EigenScore's middle-layer embedding dispersion, making them essentially three independent attacks. The Goodhart framing implies a general pathology of proxy metrics, yet the evidence covers only three points within the uncertainty-based family. The paper offers no experiments or arguments on whether retrieval-based, model-based, or supervised detection methods are equally fragile.
- A circular NLI dependency runs through the entire evidence chain: the construct's ground truth and the audited metric share the same failure modes. DeBERTa-large-MNLI plays four roles in the paper: it is the clustering component inside SE (the target of Mode 2), the Stage-2 factual-correctness oracle for all three modes, the arbiter of the semantic-equivalence/single-cluster constraint in Mode 3's success criterion, and the candidate filter for the Mode 2 attack. This means that in Mode 2, the determination that "the facts have been corrupted" and the observation that "SE failed to detect the corruption" come from the same model. Attack success may merely reflect correlated NLI errors across the two stages rather than a genuine divergence between the metric and the construct. The human evaluation was the right idea as an independent check, but inter-annotator agreement on ES consistency is near zero — the point where independent evidence is most needed is precisely where it is least trustworthy.
Questions for authors
See Above.