acceptodds
Under review as a conference paper at ICLR 2027

What Hallucinations Have We Measured? An Empirical Study

Abstract

Hallucination remains one of the most critical challenges in deploying large language models (LLMs) reliably and safely. Yet the term is used inconsistently across the research community. Analyzing hallucination-titled papers from major NLP and ML venues, we find that explicit definitions are increasingly rare and that the term conflates fundamentally different error types while failing to distinguish hallucinations from expressed uncertainty. This ambiguity obscures what is being measured and why models fail, limiting progress in detection and mitigation. In this work, we show that hallucinated responses should be distinguished from expressed uncertainty and that hallucination is not a unitary phenomenon. We identify two types of model incorrect responses: confabulation, where a model consistently produces incorrect responses, and semantic illusion, where the response correctness varies with the prompt. These two types exhibit distinct empirical signatures and are systematically conflated in existing evaluations. We formalize this distinction with operational definitions and diagnostic criteria, and show that separating them improves interpretability and reveals limitations of current detection methods. We advocate for future work to explicitly distinguish and report these error types.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM, hallucinations

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_dtc1OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper distinguishes hallucinations into confabulation, semantic illusion, and expressed uncertainty, demonstrating differences in detection and mitigation effectiveness across categories.

Critical strengths

The paper provides a systematic empirical evaluation of existing hallucination detection and mitigation methods, revealing meaningful performance differences across distinct error categories.

Critical weaknesses
  1. Limited Novelty. The paper primarily repackages well-established phenomena, including prompt sensitivity, knowledge elicitation, and abstention, into a new taxonomy without identifying fundamentally new LLM behaviors.

  2. Arbitrary and Budget-Dependent Categorization. Confabulation and semantic illusion are defined using a finite set of prompt variations. Since the prompt space cannot be exhaustively explored, the same question may receive different labels as the evaluation budget increases.

  3. Insufficient Evidence for Distinct Mechanisms. The paper does not demonstrate that confabulation and semantic illusion arise from fundamentally different mechanisms. Their differences may simply reflect variations in question difficulty or knowledge accessibility.

  4. Predictable Experimental Findings. The observation that prompt-sensitive errors are easier to correct is largely expected from their operational definitions. Similarly, encouraging abstention naturally increases expressed uncertainty without necessarily improving factual accuracy.

  5. Unclear Added Value of the Taxonomy. The paper does not establish whether its discrete categories provide additional predictive or diagnostic value beyond continuous measures of prompt sensitivity, answer consistency, and recovery probability.

Questions for authors

NA

3 comments

Thank you for the review. We address below. We tried to be very concise and clear for reviewer's time. We would appreciate it if the reviewer could consider our contribution in their final evaluation.

W1/W4 — Novelty/ unexpected findings

  • Frequent misuse of prompt variance and external knowledge (literature audit, Fig. 2, Table 4 for each paper)
  • SI/ CF labeling is plausible ($\S$6.2, $\S$6.3, Appendices K, L, M)
  • Interesting Findings
    • Finding1) CF is blindspot ($\S$8, H2)
      • Even external-knowledge methods, which should supply the absent knowledge, mitigate CF less than SI.
      • Useful way of using them is the key.
      • Whether inference-time methods can reach knowledge that prompting cannot (i.e., for CF) is an open question.
    • Finding 2) Uncertainty estimation is not always for hallucinations ($\S$9, H3)
      • Existing hallucination methods, including those that are not using uncertainty estimation, are detecting UNK the most, rather than SI or CF (Fig. 5).
      • Uncertainty estimation based hallucination methods also detect UNK the most but do not mitigate them well.
    • Finding 3) Uncertainty representation that works for hallucinations ($\S$10, H4)
      • SI is working with semantic space, not verbal signal
      • UNK is good with verbal signal
      • CF is working with latent signal

W2 — Budget dependence

  • All our main figures (Figs. 7, 8, 13, 14) are reported from V(Q)=1 to 30.
    • They show the similar patterns across labels.
  • Label Stability
    • Labels stabilize as k grows: cohen's κ\kappa = 0.8 at k=10 and > 0.92 at k=20, with <1.8pp spread across random paraphrase subsets (Appendix K, Figs. 3,4, and Table 13).
    • Random seeds and decoding settings (Appendix L, Table 14).
    • Paraphrases add no answer information (Fig. 4c).
    • Using frontier models (Appendix I).
    • Standard deviation (SD), 95% range, agreement, cohen's κ\kappa, and label flipping rate (Appendix K)
    • Stopping-rule labeling for custom data (Appendix M)

W3 — Mechanisms, Q difficulty, and knowledge accessibility

  • We do not claim distinct internal mechanisms; the taxonomy is behavioral, and labels are diagnostics, not intrinsic model properties ($\S$6.1, Remark 1).
  • Knowledge accessibility
    • a criteria to label CF/SI: knowledge is accessible if any of semantically equivalent variants elicits a correct response ($\S$6.2).
  • Similar question difficulty for models
    • Fig. 4c -- Paraphrases add almost no answer information for either type, yet only SI questions are ever answered correctly.
    • Table 9 -- on the same questions, methods favor either SI or CF (across V(Q) from 1 to 30)
    • The CF/SI gap holds under different setup of multi-hop questions (Appendix J, Table 12).

W4 — Our findings differ from what would be expected

  • Expected: external-knowledge methods, which supply the absent knowledge, should fix CF. Found: they still mitigate CF less than SI (§\S 8, Fig. 6) -- so CF is a blind spot.
  • Expected: hallucination detectors should detect hallucinations (H3). Found: they detect UNK, which is not a hallucination, more than SI and CF (S1; Fig. 5). H3 is not supported (§\S 9).
  • Expected: allowing the model to say "I don't know" should make the remaining CF easier to detect. Found: CF detection drops (52% →\rightarrow 36%; Fig. 5), and the remaining CF are missed in about half the cases (50.4%; Table 3).
  • Expected (if CF were just harder questions): every method should find CF harder. Found: on the same questions, some methods favor CF (LapEigvals −24.1, HEB −42.6; Table 9).

W5 — Added predictive and diagnostic value

  • Diagnostic:
    • We cannot separate CF from UNK (both have ρ\rho near zero) if we only take into account the continuous measures.
    • Our labels are discrete so can have CF and UNK separately.
      • This shows that methods detect UNK, which is not a hallucination, the most and miss CF ($\S$9).
  • Predictive:
    • Please see findings answered for W1/W4
    • Similar patterns hold under multi-hop (Table 12)
      • CF stays near zero with small standard deviation (0.02–0.03), while SI is far higher with large standard deviation (0.16–0.23).

I have carefully read the authors' response, and I appreciate their efforts to address my concerns. However, I still have the same concerns as before. I think my main concern about the research problem has not been fully addressed. Therefore, I have decided to keep my original score unchanged.

Thank you for your response. Would you let us know which of your concerns remain?

Many thanks, Authors

Reviewer_hLoMOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the conceptual ambiguity in existing LLM hallucination research and proposes a behavioral taxonomy distinguishing three categories. CF refers to consistently incorrect responses across prompt variants, while SI describes errors that can be corrected through semantically equivalent prompts. The authors analyze existing hallucination research and evaluate detection and mitigation methods on factual QA benchmarks. Their findings suggest that SI is substantially easier to mitigate than CF, while uncertainty-based detectors tend to favor explicitly uncertain responses.

Critical strengths
  1. Meaningful problem formulation. The paper addresses an important limitation in hallucination research: existing evaluations often aggregate fundamentally different error behaviors into a single metric.
  2. Interesting empirical observations. The paper systematically examines prompt sensitivity and evaluates multiple existing methods. The substantial mitigation gap between SI and CF provides useful evidence that different error behaviors may require different intervention strategies.
Critical weaknesses
  1. Limited novelty and dependence on operational definitions.
  2. Insufficient evidence for the generality of the conclusions.
Questions for authors
  1. How does the proposed CF/SI taxonomy differ fundamentally from existing classifications based on response consistency and prompt sensitivity? What additional insights does it provide beyond measuring the probability of answering correctly across prompt variants?
  2. Since CF and SI labels depend on the number and selection of prompt variants, how robust are the conclusions to different paraphrase generation strategies and evaluation budgets?
  3. Could the authors provide per-method results and statistical significance tests to clarify whether the observed mitigation differences arise from error types rather than differences between the evaluated methods?
  4. How well do the proposed taxonomy and empirical conclusions generalize beyond short-form factual QA to reasoning, summarization, and retrieval-augmented generation tasks?
1 comment

Thank you for the review. We address each point below, kept concise for the reviewer's time. We'd appreciate it if the reviewer could consider them to finalize their evaluation on this meaningful work.

W1 / Q1 — Difference from existing classifications

  • Prior taxonomies classify by source or contradiction (intrinsic/extrinsic, factuality/faithfulness; §\S 3). Ours classifies by behavioral knowledge accessibility, and separates expressed uncertainty (UNK).
  • The correctness rate ρ\rho alone cannot separate CF from UNK (both ρ\rho is almost 0; Appendix F). This separation reveals that hallucination methods detect UNK, which is not a hallucination, more than actual hallucinations (SI and CF) (§\S 9, Table 3).
  • On the same questions, methods favor opposite types: SelfCheckGPT favors SI (+21.6pp), while LapEigvals (−24.1) and HEB (−42.6) favor CF (Appendix H, Table 9). A single consistency measure would not predict this.

Q2 — Robustness of paraphrases for SI labeling

  • Labels stabilize as k grows: cohen's κ\kappa = 0.8 at k=10 and > 0.92 at k=20, with <1.8 pp spread across random paraphrase subsets (Appendix K, Table 13)
    • Since no subset of paraphrases changes the results, labels do not depend on the choice or quality of individual paraphrases.
  • Random seeds and decoding settings (Appendix L, Table 14).
  • Paraphrases add no answer information (Fig. 4c).
  • Using frontier models (Appendix I).
  • Standard deviation (SD), 95% range, agreement, cohen's κ\kappa, and label flipping rate (Appendix K)
  • Stopping-rule labeling for custom data (Appendix M)

Q3 — Per-method results and significance

  • Per-method results (Table 9, Appendix H)
  • The SI–CF mitigation gap (29.5$ \pm$7.5 vs. 3.9$ \pm$7.3%) is large relative to the spread across methods (Table 3)
  • SI > CF holds within every intervention stage across V(Q) from 1-30 (Fig. 7)

W2 / Q4 — Generality

  • Multi-hop reasoning (Appendix J, Table 12) holds the same patter
    • CF stays near zero (0.05–0.11) while SI is far higher (0.41–0.56)
  • Frontier models (GPT, Claude, Gemini) (Appendix I, Table 11)
    • Label proportions stabilize with κ\kappa
    • Uncertainty elicitation migrates CF into UNK
  • We chose closed-book QA because it provides verifiable ground truth and makes model knowledge the sole explanatory variable (Appendices E and F)
    • But we observe the same pattern for multi-hop questions, which add a retrieval-like step of recalling an entity before its property. Standard RAG where V(Q) would also need to perturb the source could be a natural next step (Appendix J)
Reviewer_jUaFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that hallucination evaluations conflate expressed uncertainty with confidently incorrect responses. It proposes two categories of incorrect behavior: confabulation errors where responses remain incorrect regardless of the inference strategy (prompt/ instruction wording or decoding configurations), and semantic illusion, where some inference conditions elicit incorrect answers and others elicit correct answers without additional information.

The authors motivate these distinctions through a literature audit and researcher survey. They empirically evaluate methods from 14 papers on factual QA drawn from TriviaQA and SimpleQA, primarily using two models. They examine label stability across paraphrases and decoding settings, use two labeling schemes, and evaluate methods on held-out prompts.

They find that uncertainty-based detectors generally assign higher hallucination scores to expressed uncertainty than to confident errors, and that confabulation is harder to mitigate than semantic illusion. External-knowledge methods provide the strongest confabulation mitigation among the evaluated approaches. The authors recommend reporting these behaviors separately and testing prompt-based recovery before introducing external knowledge.

Critical strengths
  1. The paper presents a valid and consequential critique of evaluation validity. Separating expressed uncertainty from confident errors exposes how detection scores can obscure performance on persistent, confidently wrong answers. This matters because apparent benchmark improvements may otherwise reflect sensitivity to abstention rather than better identification of the errors researchers intend to address.
  2. Clear writing and presentation of the framework. The paper communicates its central distinctions through explicit definitions, illustrative examples, and a well-organized empirical analysis. These choices help readers understand the proposed categories, their operational limitations, and their relevance to evaluation.
Critical weaknesses
  1. Limited insight beyond the proposed taxonomy. The paper provides useful terminology and empirical documentation, but several central findings closely follow the distinctions used to construct the categories. For example, uncertainty-based detectors responding strongly to expressed uncertainty, and prompt-invariant errors being hard to mitigate with prompt-based correction, are largely expected conclusions. The paper does not make it clear what additional understanding the taxonomy provides beyond familiar distinctions between uncertainty, prompt sensitivity, and question difficulty.
  2. The taxonomy’s added explanatory and practical value is unclear. The two identified subtypes of hallucinations are distinguished by whether tested questions ever result in a correct response. Therefore, their different correction rates may therefore reflect differences in underlying difficulty, rather than additional structure captured by the taxonomy. Held-out evaluation shows that this pattern generalizes, although it leaves that interpretation open. While separating expressed uncertainty from confident errors provides a useful evaluation perspective, the paper would benefit from clarifying what the confabulation/ semantic illusion distinction adds beyond observed correctness rates. For example, showing how the categorization changes a conclusion about existing methods or informs intervention selection would help convey its practical contribution.
Questions for authors

How broadly should the claim that confabulation requires external knowledge be interpreted? Does it describe the relative effectiveness of the evaluated methods under the tested conditions, or suggest that external knowledge is generally necessary?

2 comments

Thank you for the review. We address below. We tried to be very concise and clear for reviewer's time. We'd appreciate it if the reviewer can consider them to finalize their evaluation on this meaningful work.

W1 -- novel findings

  • Frequent misuse of prompt variance and external knowledge (literature audit, Fig. 2, Table 4 for each paper)
  • SI/ CF labeling is plausible ($\S$6.2, $\S$6.3, Appendices K, L, M)
  • Findings
      1. CF is blindspot (\S8, H2)
      • Even external-knowledge methods, which should supply the absent knowledge, mitigate CF less than SI.
      • Useful way of using them is the key.
      • Whether inference-time methods can reach knowledge that prompting cannot (i.e., for CF) is an open question.
      1. Uncertainty estimation is not always for hallucinations ($\S$9, H3)
      • almost all currently existing hallucination methods are detecting UNK the most, rather than SI or CF.
      • Uncertainty estimation based hallucination methods also detect UNK the most but do not mitigate them well.
      1. Latent representations for CF ($\S$10, H4)
      • SI is working with semantic space, not verbal signal
      • UNK is good with verbal signal
      • CF is working with latent signal

W2 -- question difficulties between SI and CF for models are similar and the performance asymmetry comes from the nature of methods.

  • Fig. 4c -- Similar answer/label information quantity
    • The information quantity is almost zero for either SI and CF type.
    • And they are pretty equally distributed despite their different accuracy.
  • Appendix H, Table 9 -- Methods with different dimensions show clear asymmetry pattern
    • SelfCheckGPT favors SI (+21.6pp), while LapEigvals (−24.1) and HEB (−42.6) favor CF.
  • Appendix J, Table 12-- multi-hop questions show the similar knowledge accessibility
    • SI does not get huge advantage of the multi-hop question.
    • This means that the SI question levels are pretty similar, and the error comes rather stochastically.

W2 — How the conclusion changes with SI and CF

  • Fig. 5 -If a hallucination method with good accuracy does not detect confident incorrectness (SI and CF), a deployed system would give users little reduction in incorrect responses, a critical AI safety issue.
    • SI detection does not change under uncertainty elicitation (58% →\rightarrow 58%), unlike CF (52% →\rightarrow 36%).
    • Prompt variance and external knowledge mitigate CF far less than SI (Fig. 6).

Q — “How broadly tested about CF requiring external knowledge”

We do not claim external knowledge is generally necessary (hence "likely" in the abstract). Findings are as 1's:

  • Even external-knowledge methods, which should supply the absent knowledge, mitigate CF less than SI.
  • Useful way of using them is the key.
  • Whether inference-time methods can reach knowledge that prompting cannot (i.e., for CF) is an open question.

We tested across:

  • V(Q) from 1 to 30 (Fig. 7)
  • per-method analysis (Tables 9)
  • model-knowledge assumptions at V(Q)=10 (Table 10)

We additionally report the original question difficulty below, in addition to the difficulty of each paraphrase (earlier response + our draft).

Question difficulty does not explain SI vs. CF: Per-token log⁡P(A∣Q)\log P(A \mid Q) (mean ±\pm SD).

ModelSICF
Phi-3−4.73±2.00-4.73 \pm 2.00−5.41±2.52-5.41 \pm 2.52
Phi-4−8.28±4.55-8.28 \pm 4.55−7.53±4.06-7.53 \pm 4.06
Llama 3.1 8B−8.25±4.39-8.25 \pm 4.39−7.89±4.07-7.89 \pm 4.07
Gemma 3 12B−16.64±9.00-16.64 \pm 9.00−16.99±9.94-16.99 \pm 9.94
  • SI and CF have nearly the same likelihood, so the difference in question difficulty between them is negligible. Difficulty does not explain why a question is SI or CF.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Comments for AC and Reviewers

We thank the reviewers for their time.

All the review contents fall under two main concerns, and both are addressed in our paper: (1) whether the findings are novel and not due to question difficulty or phrasing artifacts, and (2) whether they are robust and general. Our draft contains substantial evidence for both across the experimental results, and we revised it so that this is now visible at first glance. We summarize it below and would appreciate the reviewers and AC taking it into account.

We believe our definitions are a meaningful step forward for valid hallucination research. Current hallucination research often conflates three different model outputs, making it difficult to understand model behaviors and to develop the right methods.

Our novelty: new, concrete definitions

To our knowledge, this is the first work to define hallucination as a confidently incorrect response, separate from expressed uncertainty (UNK), and to introduce two error types based on a re-elicitation test: semantic illusions (SI), which some semantically equivalent re-elicitation corrects, and confabulations (CF), which none does ($\S$4, $\S$6).

  • Aggregation is the current norm. Only 35% of recent hallucination papers define hallucination as confident incorrectness. Yet 95.8% of surveyed first authors say it should be distinguished from uncertainty, and only 16.7% report that they did so ($\S$2).
  • Practical labeling. We show how to assign the labels on custom data, how to verify that paraphrases add no hints (ΔI≈0\Delta I \approx 0), and that the analysis is robust across 1–30 paraphrases of each question, even with a modest subset of them ($\S$6, Appendices K, M).

(1) Unexpected findings (they are not due to question difficulty)

Concern (reviewers)EvidenceWhere
Findings are predictable (jUaF, dtc1)Hypothesis H3 failed: hallucination detectors detect expressed uncertainty (UNK), which is not a hallucination, more than confident incorrectness (SI and CF)$\S$9, Fig. 5, Table 3
External knowledge, which supplies the missing knowledge, still mitigates CF less than SI$ \S$8, Fig. 6
Allowing "I don't know" does not help: the remaining CF are missed in about half the cases (50.4%)Table 3
Methods never see V(Q)\mathcal{V}(Q) or the labels, only the original questions$ \S$6.2
Could be question difficulty (jUaF, dtc1)Original question difficulty (per-token log⁡P(A∣Q)\log P(A \mid Q)) is nearly the same for SI and CF across different modelsearlier response
On the same questions, methods favor opposite types: SelfCheckGPT +21.6+21.6pp (SI) vs. LapEigvals −24.1-24.1, HEB −42.6-42.6 (CF)Table 9
Paraphrases add no answer information (ΔI≈0\Delta I \approx 0), yet only SI is ever answered correctlyFig. 4c
Value beyond continuous measures (hLoM, dtc1)Continuous ρ\rho cannot separate CF from UNK (both ρ≈0\rho \approx 0); this separation reveals that detectors favor UNK$ \S$9 , Appendix F
Practical value (jUaF)Knowledge-absence methods improve CF mitigation by 8.18.1–15.915.9 points, but SI by only 0.70.7–1.11.1Table 10

(2) Robustness and generality

Concern (reviewers)EvidenceWhere
Depends on variant budget (hLoM, dtc1)Labels change only CF →\to SI; Cohen's κ≈0.8\kappa \approx 0.8 at k=10k=10 and ≥0.92\geq 0.92 at k=20k=20, with ≤1.8\leq 1.8pp spread across random paraphrase subsets$ \S$6.2, Figs. 3 and 4, Appendix K
Stable across random seeds and decoding settings$ \S$6.2, Figs. 4a,b, Appendix L
Per-method results and significance (hLoM)Per-method gaps reported; the SI–CF mitigation gap (29.5±7.529.5 \pm 7.5 vs. 3.9±7.33.9 \pm 7.3%) is large relative to the spread across methodsTable 9, Table 3
Generality beyond QA (hLoM)Multi-hop: CF stays near zero (ρ=0.05\rho = 0.05–0.110.11), while SI is far higher (0.410.41–0.560.56)Appendix J
Frontier models (GPT, Claude, Gemini) show the same label behaviorAppendix I
Scope of "CF requires external knowledge" (jUaF)Describes the evaluated methods under tested conditions: external knowledge helps CF most, but still less than SI$\S$8
ReproducibilityStopping-rule labeling protocol; released sample question set with paraphrase pools and labelsAppendix M

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.