acceptodds
Under review as a conference paper at ICLR 2027

Can Gray-Box Rival White-Box? Anticipatory Consistency Propagation with Top- Candidates for LLM Hallucination Detection

Abstract

Hallucinations in large language models are a major obstacle to their applications. While white-box detectors achieve strong performance by exploiting internal model states, such information is inaccessible for closed-source LLMs. Gray-box methods, on the other hand, offer a practical alternative by leveraging only API-accessible Top- candidates and their probabilities, but their performance remains limited. In this paper, we seek to bridge this gap by introducing rich external semantic representations into gray-box detectors. However, since external embedding models and the target LLM are trained independently, an inherent information gap could exist between the external semantic space and the target model’s generation dynamics. To address this, we propose **Prop-K**, a consistency-guided propagation method for gray-box hallucination detection. Prop-K captures the predictive consistency of the target model’s generation process at two complementary levels: **local agreement consistency** measures how Top- candidates support or conflict with the current Top-1 prediction at each step, while **global anticipatory consistency** captures whether early predictive expectations align with or deviate from what the model generates later. Together, these two consistency signals guide semantic propagation throughout the generated sequence, potentially aligning external semantic embeddings with the target model’s predictive behavior. Empirically, Prop-K consistently outperforms existing gray-box methods with an average gain in AUROC of 0.04–0.07 **across 7 target LLMs and 4 benchmarks**. Notably, despite having no access to internal model states, Prop-K matches and, in some cases, even surpasses the strongest white-box detector with such privileged access.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Hallucination Detection

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_bQu1OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Prop-K is a supervised gray-box hallucination detector using Top-K candidate tokens, probabilities, and frozen external embeddings. Local candidate aggregation and cross-step anticipatory propagation refine token representations, followed by semantic and temporal anchor readout. Experiments cover four datasets and seven target LLMs, showing gains over gray-box baselines and average performance close to CHARM.

Critical strengths
  1. The anticipatory graph provides a distinctive way to exploit cross-step Top-K records without accessing target-model hidden states.

  2. Evaluation spans seven LLMs and four datasets with representative black-, gray-, and white-box baselines. Module ablations and a parameter-matched attention control provide useful supporting evidence.

Critical weaknesses
  1. The motivation for the local “support” interpretation and the necessity of the architecture need clearer justification. Similarity between independently encoded tokens is not question-conditioned agreement: two related city names may be mutually exclusive answers. Learned compatibility and gating may mitigate this, but their ability to do so is not directly established. The paper explains the cues individually more clearly than why rank, probability gap, entropy, cosine similarity, and an extra compatibility network must be combined in this particular way. The most distinctive contribution appears to be the cross-step anticipatory graph; local aggregation and dual-anchor fusion need stronger justification of their added value. CCP should be discussed to distinguish prior semantic/probabilistic candidate modeling from this contribution.

  2. Existing comparisons and module ablations are useful but insufficiently isolate that contribution. Stronger, reasonably tuned classifiers using identical candidate embeddings and probabilities are needed to separate richer inputs from propagation. Matched edge-perturbation controls would test anticipatory connectivity, and cue-wise and semantic/temporal/fusion ablations would test component necessity; the main ablations cover only one model and two datasets. Please clarify baseline tuning budgets and white-box layer selection. Finally, a nonsignificant difference from CHARM supports competitiveness on the tested settings, rather than establishing general equivalence to white-box detection.

Questions for authors
  1. How should semantic relatedness between independently encoded tokens be interpreted as support under a specific question? What evidence shows the learned gates handle related but competing candidates, and why are all five local cues and the dual-anchor components needed?

  2. How does Prop-K compare with well-tuned classifiers using identical embeddings/probabilities? Do gains persist under degree- and distance-matched edge perturbations and component-wise ablations beyond the current model/dataset pair? Please clarify tuning budgets and white-box layer selection.

  3. Please distinguish the contribution from CCP and qualify the claim of white-box-level performance: the CHARM comparison does not establish statistical equivalence.

10 comments

Response to W1 and Q1

We thank the reviewer for these insightful comments.


(1) How is local support conditioned on the question?

We fully agree with the reviewer that due to the inherent limitations of the gray-box setting, we have to use independent external token embeddings, whose similarity in the local module may not fully capture question-conditioned agreement. This can be partially mitigated by candidate-token probabilities from the target LLM, which retain context (question) information.

Besides, the local gating function is jointly learned with the global propagation module, which captures longer-range token dependencies, so this could also help question-conditioned information to be indirectly injected to local estimation. We hope this could alleviate your concern.


(2) Whether LAC’s learned gate properly handle related but competing candidates

We further investigate whether the learned gate helps LAC distinguish candidates that are semantically related to Top-1 but express competing answers. Specifically, we test whether the Top-1 representation learned by LAC with gating can better encode the support/conflict relationship between the local candidate set and Top-1 prediction.

We obtain a ground-truth consistency label as follows. (1) For each generation step, given the question, the answer information, and the current generation context, we use an LLM to judge whether each Top-KK candidate expresses the same or an equivalent answer as the generated Top-1 (consistent) or a different/competing answer (inconsistent). (2) We then aggregate these candidate-level consistency labels into a step-level consistency label by majority voting.

After obtaining step-level consistency labels, we train a linear probe classifier to predict them using the Top-1 representations learned by LAC as features. We examine two settings: (1) with gating, and (2) without gating. A higher probe accuracy indicates that LAC better encodes whether the candidate sets support or conflicts with the current Top-1 prediction.

We use 1,000 model-generated responses and randomly sample five generation positions from each response. We further construct a hard subset of inconsistent positions where most alternatives belong to the same semantic category as the Top-1 token but express different answers. For example, the Top-1 token may name one city while most alternatives name other cities. Such cases are particularly challenging (candidates semantically related to the Top-1 token but competing with it).

Below, we report both the overall accuracy and the hard-subset accuracy:

Gating settingOverall accuracy (%) ↑Hard-subset accuracy (%) ↑
without gate54.054.09
with gate83.2069.01

As can be seen, the LAC-gating substantially improves probe accuracy from 54.05% to 83.20% overall and from 4.09% to 69.01% on the hard subset. This shows that gating helps LAC better encode whether semantically related candidates support or compete with the Top-1 prediction, even when they belong to the same semantic category but express different answers.


(3) Why combine all five local cues and both anchor types?

We use ablations to show that the five local cues and two anchor types all make important contributions (using Gemma3-12B and Llama3-8B, on HaluEval and GSM8K).

AUROC (%)

SettingLlama3-8B / HaluEvalLlama3-8B / GSM8KGemma3-12B / HaluEvalGemma3-12B / GSM8K
Full82.40 ± 0.5091.30 ± 1.1084.60 ± 0.7084.80 ± 1.40
w/o rank81.82 ± 0.0790.43 ± 0.6184.16 ± 0.1583.37 ± 1.07
w/o probability gap81.50 ± 0.5989.76 ± 0.8182.98 ± 1.3683.00 ± 0.28
w/o entropy81.78 ± 0.4890.53 ± 0.9883.96 ± 0.0583.41 ± 2.51
w/o cosine81.60 ± 0.7090.63 ± 1.2183.94 ± 0.0882.58 ± 1.68
w/o compatibility81.21 ± 0.1588.96 ± 2.5983.22 ± 0.4483.84 ± 1.05
Compatibility only81.25 ± 0.1188.45 ± 1.1382.45 ± 0.2883.21 ± 0.76
Semantic anchors only81.77 ± 0.0888.23 ± 0.8583.17 ± 0.5083.33 ± 1.78
Temporal anchors only81.01 ± 0.3787.73 ± 1.3383.20 ± 0.0180.14 ± 1.43

As noted, removing any individual cue lowers AUROC by 0.44–2.34%. Retaining only compatibility reduces AUROC by 1.15–2.85%. Using only semantic or temporal anchors leads to mean AUROC drops of 1.65-2.76%. These results demonstrate the importance of each component/module.

Response to W2 and Q2

We thank the reviewer for these valuable suggestions. We’ve extended our ablations to both Llama3-8B and Gemma3-12B on HaluEval and GSM8K, as in the following.


(1) Comparison with a tuned classifier using identical inputs.

We augmented HALT with the same external encoder and Top-20 candidate tokens/probabilities as Prop-K (candidate embeddings projected, fused with HALT’s probability features).

We tuned the augmented HALT’s learning rate, dropout rate, and hidden dimension over 11 configurations per model–dataset setting.

AUROC (%)

MethodLlama3-8B / HaluEvalLlama3-8B / GSM8KGemma3-12B / HaluEvalGemma3-12B / GSM8KMean
HALT + Top-20 embeddings73.9386.5180.2583.4881.04
Prop-K82.4091.3084.6084.8085.78

As can be seen, Prop-K shows a significant advantage over this tuned classifier (+ 4.73% AUROC averaged over all four settings). This means simply combining external candidate embeddings and probability features is not sufficient, and resolving the mis-match between target LLM and external embedding is more important.


(2) Do gains persist after matching node degree and positional distance?

Following your suggestion, we retrained a variant with graph edges rewired while matching node degree and positional distance. Compared with the full model, matched rewiring reduces AUROC by 1.59–4.47 % across the four settings, supporting the value of the specific cross-step connections beyond the matched graph statistics.

AUROC (%)

SettingLlama3-8B / HaluEvalLlama3-8B / GSM8KGemma3-12B / HaluEvalGemma3-12B / GSM8K
Full82.40 ± 0.5091.30 ± 1.1084.60 ± 0.7084.80 ± 1.40
Matched rewiring80.02 ± 0.4586.83 ± 0.6983.01 ± 1.0281.98 ± 0.59
Δ\DeltaAUROC (pp)+2.38+4.47+1.59+2.82

(3) Do propagation and readout gains persist across settings?

Below we report new ablation results on the contribution of the propagation/readout modules across settings (mean/std of AUROC for 3 random seeds).

AUROC (%)

SettingLlama3-8B / HaluEvalLlama3-8B / GSM8KGemma3-12B / HaluEvalGemma3-12B / GSM8K
Full82.40 ± 0.5091.30 ± 1.1084.60 ± 0.7084.80 ± 1.40
w/o local propagation79.65 ± 0.3483.45 ± 0.3779.10 ± 0.7079.80 ± 0.40
w/o global propagation79.25 ± 0.0688.67 ± 1.0382.70 ± 0.5082.10 ± 0.50
w/o both propagation stages72.63 ± 0.4381.78 ± 0.8471.40 ± 0.6072.60 ± 0.80
Semantic anchors only81.77 ± 0.0888.23 ± 0.8583.17 ± 0.5083.33 ± 1.78
Temporal anchors only81.01 ± 0.3787.73 ± 1.3383.20 ± 0.0180.14 ± 1.43

As noted, removing either propagation stage or either readout component leads to drop of average AUROC by 1.65%–5.28%. These show the important gains of these modules.


(4) Tuning budgets and white-box layer selection.

For clarity, we summarize the tuning settings and white-box layer-selection protocol below.

Prop-K. (Appendix C.3 and Table 8) learning rates {10−4,3×10−4,10−3}\lbrace10^{-4},3\times10^{-4},10^{-3}\rbrace, dropout rates {0.1,0.2,0.3}\lbrace0.1,0.2,0.3\rbrace, local and global propagation rounds each in {1,2}\lbrace1,2\rbrace, and window sizes {8,12}\lbrace8,12\rbrace.

White-box layers. (Appendix C.4) SAPLMA, HaMI, and CHARM follow their original feature extraction procedures and select among layers at 40%, 60%, 80%, and 100% of model depth, with indices rounded up. Layers were selected by validation AUROC.

Response to W1, W2, and Q3

We thank the reviewer for these valuable comments, our responses are as follows.


(1) Difference between Prop-K and CCP

(1) CCP mainly considers local, stepwise evidence, including candidate probabilities and semantic relations determined by a natural language inference (NLI) model at each step. In comparison, Prop-K models both local candidate consistency and global, multi-step anticipatory consistency through longer-range info. propagation.

(2) CCP exploits token semantic information through a scoring function, whereas Prop-K uses fine-grained representation learning via information propagation. Specifically, CCP computes a scalar confidence score at each position by aggregating and normalizing candidate probabilities based on NLI judgments. In contrast, Prop-K performs message passing over high-dimensional embedding vectors with adaptive gating based on consistency signals to control weights.


(2) The CHARM comparison does not establish statistical equivalence.

We highly agree with the reviewer that the statistically nonsignificant difference between Prop-K and CHARM on the tested datasets does not constitute general evidence for method equivalence and cannot be generalized to white-box or grey-box settings. We have revised the manuscript consistently from introduction to conclusion to use more conservative descriptions and avoid overclaiming. Thank you for this rigorous and valuable comment.”

Reviewer_FScyOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies hallucination detection for LLMs under a gray-box setting, where the detector can only observe the generated token together with its top-K candidate tokens and associated probabilities. The authors propose Prop-K, a detection framework that combines predictive information from the target model with semantic representations obtained from an external encoder. The core idea is to model two forms of consistency: local agreement consistency measures whether alternative top-K candidates semantically support the selected token at each generation step, and global anticipatory consistency connects earlier predictive distributions with tokens generated at later steps, capturing longer-range consistency across the generation trajectory. These signals are propagated through graph-based modules, followed by a dual-anchor readout that summarizes semantic and temporal information for answer-level hallucination detection. Experiments on TriviaQA, HaluEval, HotpotQA, and GSM8K across multiple open-source and closed-source LLMs show that Prop-K consistently outperforms the evaluated black-box and gray-box baselines and performs competitively with several white-box detectors. However, there are still several methodological and experimental issues remaining. Detailed comments are as follows.

Critical strengths
  1. The paper makes nontrivial use of gray-box predictive information by explicitly modeling relationships among the selected token. The decomposition into local candidate agreement and longer-range predictive consistency is interesting and well-motivated.
  2. The evaluation is relatively comprehensive, covering 4 benchmarks and multiple open-source and closed-source target models. It also includes comparisons against black-box, gray-box, and white-box detectors.
Critical weaknesses
  1. The interpretation of the proposed consistency measures requires stronger justification. In Eq. (7), pi(yj)p_i(y_j) is interpreted as anticipatory probabilistic support from an earlier generation step i for a token generated later at position j. However, pip_i is a next-token distribution. Therefore, it represents the probability of generating yjy_j at position i, rather than its actual future position j. For a perfectly coherent sequence, a future token can legitimately have very low probability at an earlier position simply because other syntactically or semantically necessary tokens must occur first. It is therefore unclear whether this quantity captures genuine anticipatory consistency.
  2. The current experimental evidence does not fully support the conclusion that gray-box detection can rival white-box detection. Specifically, this conclusion depends heavily on the selected white-box and black-box baselines. Several potentially strong recent methods [A][B][C][D] appear absent, while some reproduced baseline results seem lower than those reported in the corresponding original works. The manuscript does not sufficiently clarify whether these discrepancies arise from different target models, datasets, prompts, or reimplementation details. Since this comparison is central to both the title and the paper’s main conclusion, a more complete and carefully reconciled baseline evaluation is important.
Questions for authors
  1. The interpretation of global anticipatory consistency is required. As aforementioned, pip_i is the next-token distribution at position i, pi(yj)p_i(y_j) appears to measure whether yjy_j is suitable as the immediate next token at i, rather than whether it will be appropriate after several intervening tokens. A formal or intuitive justification for treating pi(yj)p_i(y_j) as anticipatory support for a token generated at a later position j would help determine whether GAC measures genuine longer-range predictive consistency.
  2. The effectiveness of the external text encoder needs further analysis. The supplementary material indicates that decoded target model tokens are independently encoded using a frozen external text encoder. Since LLM tokens often correspond to punctuation, whitespace-sensitive units, numbers and so on, it is unclear whether context-free sentence-embedding cosine similarity consistently represents meaningful token semantics. The existing encoder ablation is useful, but all tested encoders follow essentially the same token-independent protocol. Stronger controls are therefore needed to establish what information these consistency signals capture.
  3. The practical positioning relative to recent black-box detectors is insufficiently established. The paper motivates gray-box detection as a trade-off between deployable but poor performance black-box methods and stronger but inaccessible white-box methods. However, the black-box comparison mainly focuses on relatively early-stage baselines such as SelfCheckGPT and Semantic Entropy, while recent work [A][B] has substantially expanded the black-box design space. Therefore, it is unclear whether the proposed Prop-K still provides a sufficiently compelling practical advantage over SOTA black-box alternatives.
  4. The title and main conclusion rely heavily on the claim that Prop-K can match or surpass white-box hallucination detectors. However, several recent or strong white-box methods [C][D] appear absent, and some reproduced baseline results seem noticeably lower than those reported in the corresponding original works. Could the authors explain whether these discrepancies arise from differences in target models, datasets, prompts, layer-selection procedures, hyperparameters, or implementation?
  5. A more thorough evaluation of generalization is needed. The current experimental protocol appears to train and tune Prop-K separately for specific combination of model and dataset. If so, the reported performance across multiple LLMs and datasets demonstrates matched-setting effectiveness, but not cross-model or cross-domain generalization.
  6. Tables 3 and 13 provide evidence for the impact of the propagation modules and learned weighting mechanism, respectively. However, the incremental contribution of the explicit consistency features beyond the learned pairwise compatibility term ηt,k\eta_{t,k} in the local gate remains unclear. A controlled ablation that removes or masks these explicit consistency features and retrains the detector, while keeping others unchanged, would help isolate and quantify their additional contribution.

Reference

[A] FactSelfCheck: Fact-Level Black-Box Hallucination Detection for LLMs. EACL 2026.

[B] Zero-source LLM Hallucination Detection with Human-like Criteria Probing. ICML 2026.

[C] Unconditional Truthfulness: Learning Unconditional Uncertainty of Large Language Models. EMNLP 2025.

[D] Steer LLM Latents for Hallucination Detection. ICML 2025.

7 comments

Response to W1 and Q1

We thank the reviewer for this valuable comment. We agree that, mathematically, the softmax distribution pip_i represents only next-token probabilities at step ii. However, recent work [1] showed that this distribution also contains predictive information far beyond the immediate next token - in terms of substantial overlap between top-ranked next-token candidates and distant future tokens. Our case study on GSM8K with Gemma3-12B showed that the top-50 candidates have 22.9%–59.3% overlap with future windows of 2–10 tokens. This supports the anticipatory capacity of pi(yj)p_i(y_j). In fact, many earlier studies also reported implicit look-ahead and planning behavior in LLMs [2–4].

Importantly, this anticipatory capacity is statistical, not deterministic. So we quite agree with the reviewer that individual exceptions could occur. But such anticipatory information is empirically a useful signal in our study, whose removal leads to a 1.90-2.70% performance drop (Table 3).


References

[1] Yihe Liu et al. Next-ToBE: Probabilistic Next Token-Bag Exploitation for Activating Anticipatory Capacity in LLMs. ICLR, 2026.

[2] Dhananjay Ashok and Jonathan May. Language Models Can Predict Their Own Behavior. NeurIPS, 2025.

[3] Zhichen Dong et al. Emergent Response Planning in LLMs. ICML, 2025.

[4] Huanyi Xie et al. Predicting LLM Output Length via Entropy-Guided Representations. ICLR, 2026.


Response to Q2

We thank the reviewer for this question. We have added three analyses to examine the information captured by external token embeddings.


(1) Does raw cosine similarity capture candidate relations?

We constructed a diagnostic dataset by pairing generated tokens with their alternative Top-KK candidates and annotating each pair as “consistent” or “inconsistent” using DeepSeek-V4 as a reference LLM. On the 10,000 valid annotated pairs, raw embedding cosine similarity alone achieves an AUROC of 83.63% against the annotated labels, without training any additional classifier. This shows that the cosine similarity could capture useful semantic relations. We hope this could alleviate your concern.


(2) Do external embeddings add value beyond fixed random vectors?

After replacing external token embeddings with random vectors, the averaged AUROC decreases from 85.78% to 83.33%, with declines in all four settings, indicating that the external embeddings provide useful information.


(3) How much do punctuation and number embeddings contribute?

After replacing the external embeddings of punctuation and number tokens with random vectors, the mean AUROC decreased to 85.32%, only 0.46% below Full, indicating a limited contribution from the external embeddings of such token types.

AUROC (%)

SettingLlama3-8B / HaluEvalLlama3-8B / GSM8KGemma3-12B / HaluEvalGemma3-12B / GSM8K
Full82.40 ± 0.5091.30 ± 1.1084.60 ± 0.7084.80 ± 1.40
All embeddings randomized80.50 ± 0.8188.22 ± 1.2682.91 ± 0.1881.68 ± 0.05
Punctuation/numbers randomized81.70 ± 0.3790.18 ± 1.8383.98 ± 0.3785.42 ± 0.04

Response to W2 and Q3

We thank the reviewer for these constructive suggestions. We have added evaluations of the two recent black-box detectors, FactSelfCheck and HCPD (reference [A] and [B] in the review), as below.

For FactSelfCheck, we used Llama-3.1-70B-Instruct to extract facts from the response and verify whether they are supported by additional sampled responses. Due to the token cost consideration, we performed this evaluation to Mistral-7B across four datasets.

For HCPD that uses GRPO to train hallucination detection model, we evaluated it in two settings following the paper [1]: (1) directly using the authors’ released checkpoint, and (2) performing GRPO training on our data.

For setting (1), results on Mistral-7B are listed below. Here, the released checkpoint is based on Qwen2.5-7B-Instruct and was trained on Llama-3.1-8B responses to TriviaQA. As can be seen, our method has 8.24, and 13.72% gains in averaged AUROC against FactSelfCheck and HCPD, respectively.

AUROC (%)

MethodTriviaQAHaluEvalHotpotQAGSM8K
FactSelfCheck77.3670.0171.7879.58
HCPD (released checkpoint)76.3664.0861.9074.50
Prop-K84.6080.8080.8085.50

For setting (2), we fine-tuned HCPD using GRPO on the benchmark's training split. Due to the substantial computational cost, we have so far completed the experiment on Mistral–TriviaQA, requiring approximately 165 A100 GPU-hours. HCPD achieved 79.95% AUROC, compared with 84.60% for our method under the same setting. This demonstrates the competitive performance of our gray-box approach, even against a black-box method requiring expensive GRPO tuning.


References

[1] Jiahao Yang et al. Zero-source LLM Hallucination Detection with Human-like Criteria Probing. ICML, 2026.

Response to W2 and Q4

We thank the reviewer for these important comments. Based on your suggestion, we have added comparisons with two recent white-box methods.


(1) Comparison with recent white-box baselines.

We evaluated TAD and TSV (ref [C] and [D]) across four open-source target models and four datasets. All methods share the same generated responses, data splits, and correctness labels as our setting. Below, we report the average AUROC for each method across 4 target LLMs and 4 datasets (16 combinations).

AUROC (%)

MethodMistral-7BLlama3-8BGemma3-4BGemma3-12BOverall
TAD79.4979.7982.2680.8580.60
TSV77.7385.6085.2675.9481.13
Prop-K82.9085.0086.9084.7084.90

As can be seen, the average AUROC of our method is 4.30% and 3.77% higher than TAD and TSV, respectively. This shows the competing performance of our method even against s.o.t.a. white-box methods.


(2) Why do scores differ across papers?

We thank the reviewer for this valuable comment. Hallucination detection performance is sensitive to several factors, including target LLM and its version, evaluation dataset, and answer-generation prompts. These factors can yield hundreds of possible experimental combinations, while each paper typically reports results under only a particular setting. Consequently, substantial differences across reported results are not surprising. For example, [1] reported that HaMI's performance varied by up to 6.30% across target LLMs even on the same dataset. Similarly, [2] showed that different generation prompts could lead to an AUROC difference of 9.13%, even with all other choices fixed.

We therefore believe that a rigorous and fair comparison requires running all competing methods under exactly the same experimental settings, including target LLMs, datasets, generation prompts, and ground-truth labels, etc. This is precisely what we have done. We also strictly followed the parameter configuration procedures described in the competing methods’ original papers, as detailed in Appendix C.4. So we believe our reported comparisons are rigorous and fair, and we hope this could address your previous concern.


(3) Baseline implementation and layer selection.

TAD follows the official two-stage Ridge implementation, with regularization strength and answer-level aggregation selected by training-set cross-validation. TSV uses the official default configuration. SAPLMA and HaMI follow their original architectures and main training hyperparameters. CHARM uses the authors’ Movies configuration for the three QA datasets and Math for GSM8K, chosen by task type before evaluation.

As has been described in Appendix C.4, SAPLMA, HaMI, and CHARM select among layers at 40%, 60%, 80%, and 100% of model depth (excluding the input embedding layer), with indices rounded up. For each model–dataset combination, the layer is selected by validation AUROC and fixed for testing. For CHARM, this selection applies to its activation features; the remaining settings follow the reference configuration.

To ensure consistent comparisons, Prop-K and all baselines are evaluated using the same generated responses, data splits, and correctness labels, to guarantee a rigorous and fair comparison.


References

[1] Mengjia Niu, Hamed Haddadi, and Guansong Pang. Robust Hallucination Detection in LLMs via Adaptive Token Selection. NeurIPS, 2025.

[2] Jiahao Cheng et al. Chain-of-Thought Prompting Obscures Hallucination Cues in Large Language Models: An Empirical Evaluation. Findings of EMNLP, 2025.

Reviewer_8wvBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Prop-K for answer-level hallucination detection using Top-K candidates and probabilities, avoiding the internal access required by white-box methods. Prop-K enriches the probability-based features of existing gray-box detectors with external token embeddings, using local agreement consistency and global anticipatory consistency to guide propagation. A dual-anchor readout supports supervised classification. Experiments across seven target LLMs and four benchmarks show consistent improvements over gray-box baselines and average performance close to CHARM on open-source targets.

Critical strengths
  • Prop-K consistently outperforms gray-box baselines across seven target LLMs and four benchmarks, with average performance close to CHARM on open-source targets. Achieving these gains without internal model states makes the method useful for closed-source LLMs.
  • Ablations show that both propagation stages and learned weighting contribute to the gains. Comparisons with uniform weights and parameter-matched self-attention provide useful support for the design.
Critical weaknesses
  • As written, dual-anchor fusion is invariant to permuting the early, middle, and late temporal anchor vectors. This weakens the claim that the readout preserves stage-specific information.
  • The interpretation of global anticipatory consistency needs stronger controls. A later token's appearance among earlier next-token candidates can reflect frequent or repeated tokens, and absent support is represented only by missing edges. The self-attention and uniform-weight comparisons do not distinguish anticipation from these regularities.
Questions for authors
  1. Does the implemented readout include stage information omitted from Eqs. (15)–(18) that breaks its permutation invariance?
  2. Is there evidence that GAC's gains persist after controlling for token repetition and graph density?
  3. What evidence shows alignment with the target LLM's predictive behavior beyond the improved class separation demonstrated by the linear probes?
  4. How are GSM8K responses with a correct final answer but false intermediate statements labeled?
4 comments

Response to W1 and Q1

We thank the reviewer for carefully examining the permutation invariance of the dual-anchor readout. We agree that the interaction feature RR in Eq. (15) is permutation-invariant w.r.t. temporal anchors. But the final classifier input includes not only RR, but also the semantic anchors (um\mathbf{u}_m) in Eq. (13) and the original, ordered temporal anchors (vn\mathbf{v}_n) in Eq. (14). Namely, the ordered temporal anchors are directly preserved like a skip connection, allowing the classifier to fully exploit temporal ordering (early, middle, late). So the overall readout is not constrained to be permutation-invariant.

We will clarify this in more detail in the revision. The corresponding code can be verified in our uploaded file prop_k_model.py (lines 382–389).

We further performed an ablation study that removed the ordered temporal anchors from the final readout. This leads to an AUROC drop of 1.10% on HaluEval and 1.23% on GSM8K, supporting the value of preserving temporal orders using temporal anchors as features.

Response to W2 and Q2

We thank the reviewer for this question. We added two controls, retraining each variant with three random seeds:


(1) Does graph density explain GAC’s gains?

For each sample, we preserve the edge count but randomly rewire the edges. Node degrees and positional distances are not matched.


(2) Do non-adjacent repetition edges explain the gains?

We remove edges connecting identical tokens at non-adjacent positions, while retaining all edges between adjacent positions.

AUROC (%)

SettingLlama3-8B / HaluEvalLlama3-8B / GSM8KGemma3-12B / HaluEvalGemma3-12B / GSM8K
Full82.40 ± 0.5091.30 ± 1.1084.60 ± 0.7084.80 ± 1.40
Random rewiring79.29 ± 0.5189.85 ± 0.5383.50 ± 1.6581.65 ± 0.78
Repetition control82.28 ± 0.5591.68 ± 1.5683.98 ± 1.0984.40 ± 1.10
w/o GAC79.25 ± 0.0688.67 ± 1.0382.70 ± 0.5082.10 ± 0.50

With the edge count fixed, random rewiring reduces mean AUROC by 1.10–3.15%, whereas removing edges between non-adjacent repeated tokens has little effect (-0.62% to +0.38%), with mean AUROC remaining above no GAC in all four settings. These results support the value of the specific cross-step connections beyond graph density alone and suggest that non-adjacent repetition edges are not the main source of GAC’s gains.

We will include these controls and their three-seed results in the revised manuscript.

Response to Q3

We thank the reviewer for this inspiring suggestion. To verify whether our two key modules, Local Agreement Consistency (LAC) and Global Anticipatory Consistency (GAC), can effectively refine external representations to better align with and capture the target LLM's inner predictive behavior, we added the following analyses.


(1) Local candidate preference.

At each step, the target LLM's Top-K candidates strongly reflect its local predictive behavior. We therefore examine whether, after LAC updates, the refined external Top-1 token representation becomes more consistent with the target LLM's Top-KK candidate distribution. To quantify this consistency, we define

stL=∑k=220pt(ct,k)∑r=220pt(ct,r)cos⁡!(ct,k,yt),s_t^{L} = \sum_{k=2}^{20} \frac{p_t(c_{t,k})}{\sum_{r=2}^{20} p_t(c_{t,r})} \cos\\!\left(\mathbf{c}_{t,k},\mathbf{y}_t\right),

where yt=ct,1y_t=c_{t,1} is the generated Top-1 token and bold symbols are external embeddings.

This signal measures the probability-weighted semantic agreement between external Top-1 representation and the target LLM's top-candidates. A larger stLs_t^L means stronger semantic support. If this signal can be more accurately recovered by LAC-refined representations, it indicates that LAC can better capture target LLM’s local predictive behavior.


(2) Global anticipatory consistency.

Across generation steps, the target LLM’s earlier predictions may provide anticipatory support for content generated later. We therefore examine whether, after GAC propagation, the refined representation at each step can better recover the target LLM’s earlier predictive expectations. To quantify this consistency, we define

stG=∑i∈HtI![yt∈Ki],s_t^{G} = \sum_{i\in\mathcal{H}_t} \mathbb{I}\\!\left[y_t\in\mathcal{K}_i\right],

where Ht\mathcal{H}_t contains valid positions 2–8 steps before tt, and Ki\mathcal{K}_i is the Top-20 candidate set at position ii.

This signal measures how often the current token received predictive support from earlier steps. A larger stGs_t^G indicates stronger support from previous generation. If this signal can be more accurately recovered from GAC-refined representations, it indicates that GAC can better capture target LLM’s long-range anticipatory behavior.

Regression targetStageBefore R2R^2After R2R^2ΔR2\Delta R^2
Local semantic support stLs_t^LLAC0.28420.6555+0.3713
Historical candidate support stGs_t^GGAC0.28390.4573+0.1734

Having defined the two signals, we train two MLP probes to predict them from the external Top-1 token representations before and after the corresponding LAC and GAC modules, using Gemma3-12B on GSM8K. As can be seen above, both signals are substantially more recoverable after the propagation modules, showing that LAC and GAC allow better capturing target LLM’s local and global behavior. We hope this could alleviate your concern.


Response to Q4

We thank the reviewer for raising this interesting angle. Our original DeepSeek labeling assessed answer correctness only by the final-answer (following the same protocol in GSM8K dataset paper [1]), so such responses with “incorrect middle-step but correct final answer” could be assigned the non-hallucination label.

We find that such cases are relatively rare. Specifically, we audited 200 GSM8K test responses from each of the seven target models, and found 11 such cases among 1,400 responses (0.79%).

To test whether this issue affects our results, we specifically evaluated all 1,319 GSM8K responses from Llama3-8B, with 22 responses and highest ratio (1.67%) of “incorrect middle-step but correct final answer”. After changing these labels from non-hallucination to hallucination while keeping our model unchanged, our AUROC further increased from 91.3% to 92.70%, indicating that this labeling issue does not affect our conclusion.


References

[1] Training Verifiers to Solve Math Word Problems. 2021.

I thank the authors for their detailed responses, and would like to keep my positive rating.

Dear Reviewer,

We highly appreciate your thoughtful review, valuable comments, and the time and effort you have dedicated. Your feedback has helped us greatly improve the paper, and we have learned a great deal from this review. We will make sure all revisions are thoroughly incorporated into the revised manuscript. Thank you so much.

Best regards,

All the authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.