- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes a span-level hallucination taxonomy, categorizing hallucinations into 5 semantic components, including actor, state, temporal, object, and action. The authors collect 2000 hallucinated model generations and ask 5 annotators to code each sample and annotate hallucinations at the claim level. Through an iterative process, the authors form a hallucination taxonomy and decision procedure to annotate hallucinations with high inter-annotator agreement. The experimental results suggest that the proposed taxonomy achieves higher agreement than taxonomies used in prior work.
Critical strengths
- Annotating hallucination at the span level is usually costly and noisy. This paper proposes a new taxonomy, grounded in Conceptual Dependency Theory, making the annotation more reliable.
- The paper provides detailed definitions for each hallucination type and a clear taxonomy decision procedure, making it easy for the community to follow and adopt the pipeline for other hallucination datasets.
- The paper is generally well written and easy to read.
Critical weaknesses
- Lack of details in the methodology section: Although the paper provides details of the hallucination taxonomy and decision procedure, the discussion of open coding and quality assessment is insufficient, making it hard to assess the soundness of the annotation results. Specifically,
- How many iterations of open coding are performed? The paper mentions only that the open coding process was conducted iteratively (Line 158), without specifying the number of iterations or the termination criteria.
- How are the identified themes connected to CDT and hallucination taxonomy? The 5 themes identified in Appendix A.2 do not directly map to the hallucination taxonomy. The paper should provide more details on their connection and how annotators are instructed to annotate the taxonomy after open coding.
- How is Cohen's kappa computed on the development set? The paper states that the development set was annotated by one human annotator and two LLMs (Line 149). Is the agreement measured between the human and LLM? How do the authors ensure that the LLM's annotation is reliable?
- How is pairwise Cohen’s kappa performed on the testing set? The annotation is done by 5 annotators, while pairwise Cohen’s kappa is designed for 2 annotators. It is unclear how the authors compute inter-annotator agreement.
- How is the quality of localization measured? The paper measures the quality of hallucination classification at the claim level, whereas hallucination localization is done at the span level. The paper should also discuss how the authors measure the quality of hallucination localization.
- The experiment is biased toward the taxonomy proposed in this paper: The paper compares its own taxonomy against taxonomies derived from seven hallucination categorization schemes used in prior work. However, apart from taxonomy design, there are many other reasons to make annotators have higher agreement on its own taxonomy. Specifically,
- Unlike its own taxonomy, where the annotators spend many iterations on forming and refining the themes, it is natural to expect that annotators may be less familiar with taxonomies used in prior work.
- The agreement of its own taxonomy is obtained after multiple iterations on the same development set. Thus, the agreement is natural to be higher than annotating taxonomies without iterative refinement.
- The evaluation and analysis are too thin:
- The refinement analysis should naturally be put in Section 3, as it is part of the rationale for the design of the decision tree. Other than that, there is no analysis in Section 5.
- The only evaluations are Cohen's kappa on the testing set and comparison against prior work on the development set. The limitation of the comparison is discussed in the previous point.
- I expect more analyses and experiments based on the annotation results of the proposed taxonomy. For example, how do LLMs hallucinate each type of error; which type is the most hallucinated; and what is the inter-annotator agreement across error types? Some analyses discussed in the appendix are worth putting in the main paper with more details (e.g., there is no table or figure for confusion analysis in the current appendix).
- The paper lists automatic hallucination detection as future work, which is reasonable. But I think it is fair to run some simple hallucination detection algorithms (e.g., LLM-as-a-Judge and uncertainty-based methods [1, 2, 3]) on the testing set to see how existing algorithms perform on detecting span-level hallucinations.
- While the paper proposes a hallucination annotation pipeline to obtain high agreement across annotators, the process seems expensive in terms of time and cost. It is unclear how easy others can adopt the same pipeline to a larger hallucination dataset. It would be more informative if the authors could report the time and money spent on annotating these data.
[1]: Zhang et al. 2023. Enhancing uncertainty-based hallucination detection with stronger focus. [2]: Guerreiro et al. 2023. Looking for a needle in a haystack: A comprehensive study of hallucinations in neural machine translation. [3]: Malinin and Gales. 2021. Uncertainty estimation in autoregressive structured prediction.
Questions for authors
- In Section 3.1, the author mentions that they collect 2000 hallucinated model outputs. However, in Section 3.2, the 2000 samples are then divided into 1550 model outputs for training, and 150 and 300 hallucinated atomic claims for development and testing, respectively. Does each model output contain only one atomic claim? Or what is the unit of the collected samples?
- Do the authors know why Cohen's kappa on the testing set is higher than on the development set? Since the themes are refined based on the evaluation results on the development set, it is expected to see higher agreement on the same set.
2 comments
I thank the authors for their response. However, I don't think my concerns are properly addressed and I will keep my score
1. Open Coding and Termination
Open coding involved four training-set rounds until no new categories emerged and boundaries stabilized. We will clarify this in Sec. 3.1.2. This interpretive process depends on annotators, example order and judgments. Table 3 shows development-set taxonomy variants; Sec. 5.1 reports disagreements prompting refinement. Reproducibility concerns final definitions, guidelines and procedure, not identical exploratory iterations.
2. CDT and the Final Taxonomy
CDT does not map one-to-one to the taxonomy. Its inventory (PP, PA subtypes, ACT, STATE and actor/object/recipient roles) was explored as candidate nodes. Open coding identified output themes; CDT grounded them theoretically, and agreement guided selection and consolidation. Table 3, Sec. 5 and Appendix A.4 document this process. Primitives without empirical separability were not retained.
Five categories map as follows:
PP represents ACTOR and OBJECT entities. LOCATION and PROPERTY were subsumed under ACTION_ERROR and OBJECT_ERROR. Finer-grained hierarchy has more mappings but put as future work.
3. Development-Set Kappa
Agreement averages kappa across human–LLM 1, human–LLM 2 and LLM 1–LLM 2 pairs. LLMs are not ground truth. Final reliability uses independent humans on the held-out test set.
4. Test-Set Pairwise Kappa
For five annotators, average pairwise kappa is the arithmetic mean of ten unique coefficients ((\binom{5}{2}=10)), not a single multi-annotator calculation. We will clarify this in the methodology.
5. Localization Quality
The study operates on atomic claims. The 450 development/test claims were decomposed using GPT-4o with human validation (Sec. 3.1.2), then fixed before annotation. Each claim is segmented into ACTOR, ACTION and OBJECT using the deterministic left-to-right rule (Sec. 3.6.2); the ordered procedure identifies the first failing component, label and location.
Five annotators labelled the same 300 test claims; κ=0.865 measures agreement on the failing component and span within each claim. We did not measure agreement on decomposition or mapping spans to original outputs (Sec. 6). Decomposition is treated as an established factuality-evaluation step; FActScore and SAFE offer automated approaches. We focus on attribution within claims, where disagreement was observed (Sec. 5.1). This is an assumption, not an empirical finding: results establish consistency within atomic claims, not end-to-end localization in unrestricted text.
6. Prior-Work Taxonomies
Baseline taxonomies used published definitions and trained annotators on the same set under standardized guidelines. Our advantage is cognitive grounding and deterministic resolution of category overlap.
7. Independent Validation
Five final-validation annotators, independent of taxonomy development, evaluated the held-out test set.
8. Refinement Analysis
The appendix refinement analysis, referenced in the main text, documents stages supporting the final procedure.
9. Scope of Evaluation
The manuscript reports overall agreement and confusion patterns. Category-level distributions across LLMs and agreement by error category remain useful extensions.
10. Additional Analyses
Further analyses were omitted due to ICLR page limits.
11. Automatic Detection
We develop a taxonomy for localization; automatic detection is distinct. The taxonomy provides span-level annotations for training and evaluating detection systems. LLM-as-a-Judge and uncertainty-based approaches require alignment with span-level localization for meaningful comparison. These are future directions, not validation prerequisites.
12. Annotation Cost and Scalability
Manual annotation requires identifying, localizing and categorizing claims. Our procedure improves consistency and reproducibility, not effort. Time was not systematically recorded because annotators worked independently with different tools and environments. We will clarify this limitation; standardized measurement of time, throughput and costs remains future work.
13. Samples and Atomic Claims
The 2,000 samples are model outputs, each potentially containing multiple claims; 1,550 outputs were allocated to training. Development and test comprise 150 and 300 atomic claims. These 450 claims were decomposed using GPT-4o with human validation (Sec. 3.1.2) and fixed before annotation.
14. Development- and Test-Set Kappa
Development agreement involved one human and two LLMs; testing involved five independent humans. Values are not directly comparable given differences in annotators, samples and taxonomy maturity. Test agreement was higher on unseen claims using the frozen taxonomy and procedure; the test set was not used for refinement.