acceptodds
Withdrawn from ICLR 2027

SpanProbe: An Adaptive Hallucination Detection Framework Based on Token Evidence Profiles

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal large language models (MLLMs) can generate fluent visual descriptions but remain prone to hallucinations, limiting their reliable deployment. Existing hallucination detection methods based on model-internal hallucination signals typically rely on predefined readout strategies. Because tokens within a span carry different information, fixed readout strategies struggle to adequately capture the relative importance of individual tokens, making it difficult to reliably assess the hallucination risk of a complete semantic span. We find that hallucination signals tend to concentrate on a small number of tokens within each span and that the token carrying the strongest signal varies across samples. Therefore, accurately characterizing the relative importance of individual tokens for hallucination discrimination under such a non-uniform distribution is essential for reliable span-level hallucination detection. To address this challenge, we propose SpanProbe, an adaptive hallucination detection framework based on Token Evidence Profiles (TEPs). SpanProbe extracts visual-attention mass, full-attention entropy, and attention-head output norms from every attention head in every decoder layer to characterize each token's visual dependence, attention dispersion, and activation strength. It then dynamically generates sample-dependent aggregation weights from the token TEPs and uses these weights to construct a span-level representation for hallucination detection. Furthermore, SpanProbe employs a risk-gated local rollback-and-regeneration strategy. Using the risk scores produced by the pooling module, it identifies high-risk semantic spans and selectively revises high-risk content while preserving previously accepted low-risk content, thereby mitigating hallucinations during generation. Extensive experiments across multiple benchmarks demonstrate that SpanProbe generally outperforms existing baselines. On M-HalDetect, which contains multi-token semantic spans, SpanProbe achieves an average AUPRC of 77.43%, outperforming the strongest baseline by 8.05 percentage points.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Multimodal Large Language Models, Hallucination Detection, Hallucination Mitigation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_o4AfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SpanProbe, a framework for detecting hallucinations in MLLMs through adaptive aggregation of token-level evidence within semantic spans. The proposed framework constructs TEP from attentions and learns sample-dependent token weights to estimate span-level hallucination risk. The framework also use local rollback and regeneration to mitigate hallucinations. Several experiment show improved performance across multiple benchmarks and backbone models.

Critical strengths

(S1) The proposed method outperformed existing method on most datasets.

(S2) The empirical evaluation across multiple benchmarks and backbones is comprehensive and structured.

(S3) The cross-dataset transfer experiments shows that the proposed method can generalize.

Critical weaknesses

(W1) This paper offers limited methodological contributions for ICLR.

(W2) The novelty of SpanProbe is limited. Previous works like [1] have explored span-level hallucination detection based on attention statistics.

(W3) In Table 3, the authors should use LLM-as-a-Judge like FLEUR[2] to check the original captioning performance is preserved. SPICE evaluates captions based on scene graphs, so I think it's not enough to assess other aspects (e.g. fleuncy) of caption.

[1] Y. Ogasa et al, "Hallucinated Span Detection with Multi-View Attention Features", SEM 2025.

[2] Y. Lee, et al, "FLEUR: An Explainable Reference-Free Evaluation Metric for Image Captioning Using a Large Multimodal Model", ACL 2024

Questions for authors

See the above weaknesses.

Reviewer_GppWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SpanProbe is a span-level hallucination detector for multimodal LLMs. For every response token, it builds a Token Evidence Profile (TEP) of 3LH3LH scalars: for each layer and head, the attention mass on visual tokens, the attention entropy, and the norm of the head output. Each TEP is standardized and projected with a tanh⁡\tanh layer to obtain ete_t. A learned linear scorer produces softmax weights over the tokens of a span, and a linear head applied to the weighted sum gives the span's hallucination risk. For mitigation, a span whose risk exceeds τ\tau is rolled back and regenerated, with its exact token sequence banned. The paper reports an average AUPRC of 77.43% on M-HalDetect (+8.05 over VIB-Probe) across four backbones, and a relative CHAIR reduction of more than 60% on LLaVA-v1.6.

Critical strengths
  1. Careful experimental hygiene. Splits are image-disjoint, thresholds and normalization statistics use only training folds, and the key comparisons use five seeds with paired image-level bootstrap confidence intervals (Tables 11 and 12).
  2. Strong control experiments. The appendix includes shortcut probes based on position, length and lexical features with a matched subset (App. F), counterfactual image pairs in which only the image changes (PairAcc 94.6%, App. G), boundary-jitter and automatic-sentence transfer (App. D), and remove-and-retrain on top-weighted tokens (App. H.3).
  3. Fair detector comparison under a shared rollback policy. Table 9 separates detector quality from the rollback mechanism.
  4. Low overhead. End-to-end latency increases by 12% with no additional peak memory.
  5. Consistent detection gains on M-HalDetect across four backbones.
Critical weaknesses

W1. Limited novelty: the pooling is standard attention-MIL pooling.

Eqs. 10 to 12 are the attention-based multiple-instance-learning pooling of [1] and closely match the span-attention representation of [2]; neither is cited. Because the risk head is linear and ∑tαt=1\sum_t \alpha_t = 1, the span logit decomposes exactly as

us=wr⊤∑tαtet+br=∑t=ab−1αt,ut,ut=wr⊤et+br,αt=softmaxt(wa⊤et).u_s = w_r^{\top} \sum_t \alpha_t e_t + b_r = \sum_{t=a}^{b-1} \alpha_t \\, u_t, \qquad u_t = w_r^{\top} e_t + b_r, \qquad \alpha_t = \mathrm{softmax}_t \big( w_a^{\top} e_t \big).

SpanProbe is therefore a softmax-weighted average of per-token logits. It interpolates between mean pooling (wa→0w_a \to 0) and max pooling (wa∝wrw_a \propto w_r with a large scale) of token logits. The span labels have "at least one part is hallucinated" semantics (an MSCOCO sentence is positive if any object is hallucinated), so the principled baselines are max, log-sum-exp or noisy-OR pooling of token logits, none of which is compared. StatPool, which takes the maximum over representations rather than over token logits, is already within 2.27 AUPRC (95% CI [1.09, 4.29], Table 12).

W2. The motivating claim relies on the method's own attention weights.

Fig. 1(a) uses SpanProbe's own pooling weights as evidence that hallucination signals concentrate on a few tokens, but attention weights are not faithful explanations [3]. The independent analysis in App. H.2 clips and renormalizes noisy differences (vt=max⁡(δt,0)v_t = \max(\delta_t, 0)) before computing the Gini coefficient and top-20% share. This procedure produces concentration even when the true signal is uniform, so comparing against a "uniform baseline" is not a valid null; a noise null is required, for example the same statistics computed between two images that both support the object. The remove-and-retrain effect in App. H.3 is small (1.86 and 2.51 AUPRC), uses one teacher per fold, and has no confidence intervals, while random removal improves AUPRC on LLaVA-v1.5, indicating a noise level of about 0.5 points.

W3. The mitigation comparison is supervised versus training-free, and the supervision matches the metric.

The MSCOCO mitigation detector is trained on COCO val2014 images with the same prompt and with labels produced by the CHAIR object-matching procedure (App. A.4), and the span constructor uses the same CHAIR vocabulary. In effect, the detector learns the evaluation metric's labeling function on the same distribution. On POPE, the detector is trained on 400 of POPE's 500 images with the same question templates, and because the answer is binary, a rollback with the rejected answer banned simply flips the answer, so the mitigation reduces to a supervised answer-correctness classifier. PAI, VCD, EAZY and beam search use no labels. A fair comparison requires supervised detector-guided baselines, such as HaloProbe-guided decoding [4] (cited but not compared), VIB-Probe mitigation under the same protocol (Table 10 is cross-study only), Lookback-Lens-guided decoding [5], M-HalDetect rejection sampling [6], and REVERSE [7], together with evaluation on benchmarks whose labels are not defined by CHAIR (e.g., AMBER, Object HalBench, MMHal-Bench).

In addition, banning only the exact rejected sequence (Eq. 15) leaves the model free to emit a synonym or a plural, or to omit the object, and nothing in the procedure favors a grounded replacement. The results suggest that omission contributes substantially: object recall drops from 65.19 to 61.10 on LLaVA-v1.6 and from 55.84 to 50.65 on Qwen2.5-VL, and objects per caption drop from 5.81 to 5.28 and from 5.26 to 4.57 (Table 7). Fewer mentions mechanically lower CHAIR, yet no CHAIR-versus-recall curves over τ\tau are reported.

W4. Baseline fairness and missing baselines.

MetaToken, DHCP and VIB-Probe are matched reimplementations with span-adaptation rules chosen by the authors; in particular, VIB-Probe is restricted to endpoint readout, the very strategy the paper argues against. No reproduction of published numbers is shown, and SVAR and Internal Confidence perform near chance on M-HalDetect and MSCOCO (AUROC 51 to 56), which suggests poor adaptation. Strong simple baselines are missing: a probe on middle-layer hidden states with mean or attention pooling (the "w/o TEP" ablation uses only the final layer, which Table 15 suggests is the weakest depth), ContextualLens (cited but not compared), HaloProbe [4], Lookback Lens features [5], and HalLoc [8]. The POPE detection results add little evidence, because each question is scored with both "Yes" and "No", exactly one of which is wrong, so detection reduces to answering the question, involves a single token (the pooling is never active), and is near ceiling.

W5. Additional concerns.

Table 1 and all mitigation results come from a single run (mitigation on 500 images) without confidence intervals. Detectors are trained per backbone and dataset; transfer is tested only on LLaVA-v1.5 and cross-model transfer is not tested. The efficiency comparison uses eager attention for both methods because TEP requires materialized attention probabilities, so the overhead relative to a realistic FlashAttention or SDPA deployment is understated. Finally, detection is framed around arbitrary semantic spans, but mitigation acts only on COCO object mentions of one or two tokens, where adaptive pooling has little effect; the mitigation gains therefore come mainly from the TEP features and the rollback, not from the paper's central contribution.

Questions for authors
  1. Pooling baselines (W1). Given that the span logit equals ∑tαtut\sum_t \alpha_t u_t, how does SpanProbe compare with max, log-sum-exp and noisy-OR pooling of the same per-token logits utu_t, trained under the same protocol? In practice, are the learned weights αt\alpha_t closer to mean pooling or to max pooling (e.g., what is their average within-span entropy)?
  2. Timing of the token evidence. For token tt, is the TEP computed from the forward pass that predicts yt+1y_{t+1} or from the pass that emitted yty_t? Is the step at position a−1a-1, which emits the first span token, ever used? Prior work finds that the first hallucinated token is the most detectable [9].
  3. Motivating evidence (W2). How are the relative token bins in Fig. 1(a) computed, and how were the 50 spans selected? Can you report the App. H.2 concentration statistics under a noise null (e.g., between two images that both support the object), and confidence intervals with several teacher checkpoints for App. H.3?
  4. Label-independent mitigation (W3). How does SpanProbe perform on a mitigation benchmark whose labels are not produced by CHAIR matching (e.g., AMBER, Object HalBench or MMHal-Bench), and on images outside COCO?
  5. Supervised baselines (W3). How does SpanProbe compare with HaloProbe-guided decoding [4] and with VIB-Probe mitigation run under your protocol, i.e., with detector-guided methods that also use supervision?
  6. Rollback outcomes (W3). After a rollback, how often is the mention (a) replaced by a grounded object, (b) replaced by another hallucination or a synonym of the rejected one, or (c) dropped? Please also report CHAIR-versus-object-recall curves obtained by sweeping τ\tau.
  7. Baseline reproduction (W4). Do your reimplementations of MetaToken, DHCP and VIB-Probe reproduce their published numbers in their original settings? Which adaptation of SVAR and Internal Confidence to multi-token spans and sentences produced the near-chance results in Table 1?
  8. Hidden-state baseline (W4). How does a probe on middle-layer hidden states (or on hidden states concatenated across layers) with the same adaptive pooling compare with the TEP representation?
  9. Transfer and cost (W5). Does a detector trained on one backbone transfer to another backbone of the same family? What is the latency overhead relative to a vanilla model that uses FlashAttention or SDPA rather than eager attention?
  10. Clarification. For MSCOCO detection, are the captions each backbone's own generations? Appendix A.1 does not specify the generating model.

References

[1] M. Ilse, J. Tomczak, and M. Welling. Attention-based Deep Multiple Instance Learning. ICML, 2018.

[2] K. Lee, L. He, M. Lewis, and L. Zettlemoyer. End-to-end Neural Coreference Resolution. EMNLP, 2017.

[3] S. Jain and B. C. Wallace. Attention is not Explanation. NAACL, 2019.

[4] R. Zohrabi et al. HaloProbe: Bayesian Detection and Mitigation of Object Hallucinations in Vision-Language Models. ICML, 2026.

[5] Y.-S. Chuang et al. Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps. EMNLP, 2024.

[6] A. Gunjal, J. Yin, and E. Bas. Detecting and Preventing Hallucinations in Large Vision Language Models. AAAI, 2024.

[7] T.-H. Wu et al. Generate, but Verify: Reducing Hallucination in Vision-Language Models with Retrospective Resampling. NeurIPS, 2025.

[8] E. Park, M. Kim, and G. Kim. HalLoc: Token-level Localization of Hallucinations for Vision Language Models. CVPR, 2025.

[9] Snel and Oh. First Hallucination Tokens Are Different from Conditional Ones. arXiv:2507.20836, 2025.

Reviewer_My26OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Internal-signal hallucination detectors for multimodal LLMs usually read a fixed position, such as the span-final token (VIB-Probe) or the mention-start token (MetaToken), or they use a fixed aggregation like mean pooling. The paper argues that the most informative token varies from span to span, so the readout should be learned. To mitigate this, SpanProbe has two parts: a detector that scores a span of generated text for hallucination risk (which to my understanding primarily uses visual-attention mass, action entropy and head-output norm), and a mitigation loop that uses that score to roll back and regenerate risky spans during generation. The multimodal model itself is frozen throughout; only a small detector head is trained.

Critical strengths

The detection gains survive controls. Five-seed runs with paired image-level bootstrap intervals separate the feature and pooling contributions: TEP with mean pooling (72.60) is roughly equal to VIB features with adaptive pooling (72.34), and combining both gives 77.12. Each component contributes about 4.5 AUPRC points, and both intervals exclude zero.

Shortcut controls are unusually thorough for this area. The appendix includes position-only, lexical-only and combined probes, a matched subset paired on text, length and position, and a length-normalized entropy variant.

The counterfactual image-pair test is a strong design. Holding the text fixed and swapping the image gives 94.6% paired accuracy, including when the number of visual tokens is matched. This is good evidence that the detector responds to visual content rather than to text alone.

The shared-rollback comparison (Table 9) holds the policy fixed and varies only the detector, which isolates the detector’s contribution to mitigation. Boundary robustness is tested. The paper evaluates jitter, merging, function-word expansion and automatic sentence segmentation.

Critical weaknesses

To accurately express my concerns, I may need some clarifying details.

It appears that mitigation can substitute a hallucination, but not remove the start of a false claim:

Rollback goes only to the start of the flagged span, so the prefix that sets up the claim stays. In “There is also a truck parked nearby,” the model is still committed to naming something after “There is also a.” It can swap “truck” for another object, which may also be hallucinated, but the method does not appear able to drop the start of a claim. The method appears to assume that you already know where each span starts and ends.

Detection might be a bit slower than other methods that propose to prevent

I don't believe this is a insurmountable concern, but as the method is written, it might justify a high evaluation of cost trade offs. For example, it appears that the method requires all attention probabilities, and all layers, as well as not having the ability to detect before any generation was made.

Novelty might be limited

The scores used are fairly known: Visual-attention mass, attention entropy and head-output norms, (and are actually mentioned to some extent in the VIB-Probe paper), which suggests that most of the novelty comes from the rollback loop. Would any score be compatible? Is there any motivation that these scores would be useful in a feedback loop paradigm?

Questions for authors

See weaknesses

Comments on the paper

2 comments

Response to authors

Thank you for the detailed response. Some of my concerns are resolved; others remain.

Span boundaries. Thank you for clarifying that candidate boundaries are determined online during generation. I withdraw my comment that the method assumes span boundaries are known in advance. I note, however, that the online span constructor identifies only object mentions matching the CHAIR vocabulary, so the mitigation applies only to that type of span.

Rollback. My main concern remains. Because rollback restores generation to the start of the flagged span, the prefix that introduces the claim (e.g. "There is also a") is retained, and the model must still produce an object in that position. Re-detection only helps when the replacement is itself recognized as an object mention. If it falls outside the vocabulary, it is never re-evaluated, and it is also not counted by CHAIR. The example in Figure 2 ("traffic light" → "roadside pole") appears to be such a case. Could the authors report:

  1. what fraction of regenerated spans are (a) COCO-vocabulary objects, (b) out-of-vocabulary objects, and (c) hallucinated under a broader check (e.g. human judgement or an open-vocabulary detector); and
  2. results for a variant that rolls back to the start of the clause or sentence, allowing the claim to be omitted?

Cost. Thank you for the latency comparison. Two questions:

  • Since TEP requires full attention probabilities, the method appears incompatible with FlashAttention/SDPA kernels. Was the vanilla baseline in Table 8 run with eager attention? What is the overhead relative to vanilla decoding with FlashAttention, and did all methods in the new table use the same attention implementation?
  • In the new table, VIB-Probe has CHAIRi 9.52 and CHAIRs 42.00 on LLaVA-v1.6, worse than vanilla (8.12 / 29.40). VIB-Probe's own paper reports improvements on the same backbone (11.8 → 8.7 CHAIRi, 40.7 → 32.1 CHAIRs, cited in your Table 10). Could the authors explain this discrepancy and confirm that the baseline follows its published configuration?

Novelty. I appreciate the clarification that other detectors can be plugged into the rollback loop, and that Table 9 compares them under a shared policy.

Perhaps my largest disagreement from the authors response is the claim that hallucination evidence is concentrated within spans. Figure 1 shows SpanProbe's own learned softmax weights, which are non-uniform by construction. A trained softmax will always favor some items over others, so I find Figure 1 uninformative. The independent evidence suggests modest concentration: in the remove-and-retrain analysis, removing the top-weighted token costs 1.9–2.5 AUPRC points. In the qualitative examples, the top-weighted token is usually the object noun. Could the authors compare against a fixed readout at the head noun of the span, and against max pooling? This would show whether the adaptive weighting learns more than "attend to the object word."

I am maintaining my score for now and will reconsider in light of answers to these questions.

Reviewer_GppWOpenReview ↗

Official Comment by Reviewer GppW

Thank you for the response. It majorly does not resolve my two central concerns. On W1, the argument that useful evidence appears at different positions in different spans does not distinguish the proposed attention pooling from max, log-sum-exp or noisy-OR pooling of token logits. Those alternatives are equally sample-adaptive and better match the "any part is hallucinated" label semantics. Yet the paper compares only against uniform averaging and fixed summary statistics, and the response neither runs these baselines nor acknowledges attention-based multiple-instance pooling as prior work. On W3, the shared-rollback comparison cannot address my concern, because every detector in it is trained on the same CHAIR-derived labels. The issue is not image overlap. It is that the detector is trained to reproduce the labeling function of the evaluation metric itself, which the response confirms. The reported gains over label-free methods therefore largely reflect supervision on the metric's own labels. An evaluation on a benchmark whose labels do not come from CHAIR matching is still missing, as are an analysis of what replaces a rejected mention and comparisons with supervised mitigation baselines. I therefore maintain my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.