- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper focuses on hallucination detection in LLM outputs in a gray-box regime, where models are hosted behind an external API and expose only the log-probabilities of the top-K tokens. The authors propose a supervised method that converts raw probabilities and token indices into vectors and predicts correctness from this information. They provide experiments with 3 open-weight and 1 closed-weight models on several QA datasets. Additionally, the paper presents out-of-distribution experiments with model shift and dataset shift, and ablates several components of the method.
Critical strengths
- The paper proposes a gray-box approach to hallucination detection in LLMs that can be applied to API-based systems. The method outperforms the considered baselines and shows good transferability between models.
Critical weaknesses
- The paper claims its main contribution is token-identity signals, presented as a missing signal in probability-only gray-box detection. This claim is factually incorrect. Several existing methods exploit the same idea, e.g., CCP, which is cited in the work but not connected to the proposed approach, and STC [1]. Both explicitly integrate the semantics of alternative tokens.
- The retokenization procedure is also concerning. How did you ensure that every token in the source model’s vocabulary exists in the phi-4 tokenizer, either as a token or as a sequence of subtokens? Additionally, only one external tokenizer is tested, which raises questions about the transferability of the method.
- The fixed random code table is another concern: it is a major component of the method, yet its role is unclear. The fact that pre-trained codes do not outperform fixed random codes suggests that the method gains nothing from semantics. It simply uses additional information for learning, which is unrelated to the core idea of the paper. Why are the codes fixed rather than trained? How stable are they, e.g., did you test different initializations?
- The experimental evaluation is weak. First, the paper omits numerous baselines: seq.prob (the sum of log-probabilities), mean token entropy [2], self-certainty [3], CCP, STC, as well as sampling-based [4,5] and black-box methods [6]. Second, the benchmarks are limited to QA tasks only. Third, the paper uses lexical matching to obtain correctness labels, which is an unreliable criterion [7]. LLM-as-a-judge should be used instead.
- Transfer works only under model shift, while results under dataset shift are considerably weaker, even though the task (QA) remains the same. Transferability is therefore still limited.
- Why is ATP not the highest-scoring component in TDS? Was sampling used during generation? If so, this needs to be clarified. Moreover, according to the ablation study, the ATP component is useless and even degrades performance, so the authors should clarify why it is used in the method. Finally, even when sampling is used, ATP should in most cases be included in TDS, so this performance degradation is what one would expect from the definition.
- Minor issues:
- The notation is confusing. If is the candidate token at generation step , what does mean? The index appears twice. Also, , but what is , and why does it disappear on the right-hand side? Overall, why are probabilities denoted by both and ?
- Line 178: missing period.
[1] Semantic Token Clustering for Efficient Uncertainty Quantification in Large Language Models (Cao et al., EACL 2026)
[2] Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph (Vashurin et al., TACL 2025)
[3] Scalable Best-of-N Selection for Large Language Models via Self-Certainty. (Z. Kang et al., NeurIPS 2025)
[4] Semantic Density: Uncertainty Quantification for Large Language Models through Confidence Measurement in Semantic Space. (Qiu et al., NeurIPS 2024)
[5] Uncertainty Quantification for LLMs through Minimum Bayes Risk: Bridging Confidence and Consistency (Vashurin et al., NeurIPS 2025)
[6] Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback (Tian et al., EMNLP 2023)
[7] Revisiting Uncertainty Quantification Evaluation in Language Models: Spurious Interactions with Response Length Bias Results (Santilli et al., ACL 2025)
Questions for authors
See weaknesses 2,3, and 6.