acceptodds
Under review as a conference paper at ICLR 2027

Calibrated Vision-Language Models Must Hallucinate

Abstract

Calibrated language models must hallucinate rare facts at a rate set by the missing mass of the training sample, yet this result is open domain and does not apply when a source document determines the answer. An image looks like such a source, so a naive transplant would predict a floor of zero. This paper shows when that closed domain caveat fails for object hallucination in vision language models. A vision language model does not observe the image directly but rather a feature produced by an encoder, a connector, and softmax attention, which we model as a bounded evidence channel with retention . For any generatively calibrated captioner, the object hallucination rate is at least the visual monofact rate times the chance the image is unread, minus miscalibration. The bound recovers the text theorem at , vanishes at perfect vision, and is matched by an explicit generator. Forecast calibration on yes or no existence questions does not imply the same necessity on captions, and instruction tuned decoders occupy the abstention vertex of a linear coverage hallucination frontier, so low false Yes on a binary probe need not imply reliable free form description. Softmax attention converts visual token count into , so adding unrelated visual tokens raises the generative floor. The identities are attained in a Zipf visual fact world and are checked on frozen Qwen3-VL models with official POPE questions and CHAIR style captions, confirming the theory within one percent relative error.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Vision-Language Models, Object Hallucination, Generative Calibration

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_cWZaOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies why vision-language models may still hallucinate objects even when an image is provided. It extends the framework of Kalai and Vempala (2024) by introducing a visual evidence channel: the relevant visual information is either available to the model or erased. Under the paper’s calibration and distributional assumptions, the proposed hallucination lower bound depends on both the probability of losing visual evidence and the visual monofact rate, which measures the frequency of training facts that appear only once. The paper also presents a generator intended to match this bound. The paper further discusses attention dilution from unrelated visual tokens, the trade-off between answering questions and avoiding hallucinations, and the distinction between calibration on yes/no existence questions and calibration in free-form caption generation. Evaluation includes a controlled synthetic setting and experiments with frozen Qwen3-VL models using POPE questions and CHAIR-style caption evaluation.

Critical strengths

S1. The paper addresses an important question through a useful distinction between having an image and having usable visual evidence. The motivation is compelling: providing an image does not necessarily mean that the model’s internal representation retains the information needed to verify a statement. Connecting this limitation with incomplete coverage of training facts offers an interesting perspective on object hallucination. I find this a worthwhile direction for theoretical investigation, even though the applicability of the proposed model needs further justification. S2. The distinction between yes/no probing and free-form generation is valuable for interpreting evaluation results. The paper explains that reliable existence predictions do not automatically imply reliable object mentions in generated captions. It also separates abstention from incorrect answers, making clear that reducing errors can come at the cost of answering fewer questions. These distinctions are useful when interpreting claims about hallucination reduction across different evaluation settings.

Critical weaknesses

W1. The connection between the theoretical assumptions and practical vision-language models is not sufficiently established. The central erasure assumption is quite strong: the visual representation either determines whether a statement is true or provides no information about its truth. I understand why this assumption makes the analysis more tractable. However, I am less clear about how the resulting conclusions apply when the visual representation provides useful but incomplete evidence. The discussion of alternative channels in Appendix E is helpful, but the paper should more clearly distinguish conclusions specific to hard erasure from those that extend to these less idealized settings. W2. The empirical evidence does not adequately support the strength of the quantitative claims. The synthetic experiments use the proposed matching generator, whose probability of drawing from the false pool is explicitly set using the monofact rate. These experiments are useful for illustrating the construction and checking its numerical behavior. However, because the mechanism being tested is built into the generator, the close agreement should be distinguished from independent evidence that trained vision-language models follow the same quantitative relationship. For the real-model experiments, Table 8 on page 32 reports a CHAIR-style hallucination rate of 0.276 against a proxy lower bound of 0.026 for the 224-pixel condition. The corresponding values are 0.279 versus 0.019 for the 448-pixel condition, and 0.307 versus 0.037 for the four-image condition. These observations are above the estimated lower bound, but that is different from showing that the bound closely describes the measured hallucination rates. I do not regard this gap as a contradiction of a lower bound, and a bound that is attainable by a constructed generator need not be attained by a deployed model. My concern is instead with the presentation: the abstract and conclusion appear to extend the claim of agreement within one percent to the frozen-model experiments. I could not identify the corresponding real-model slope analysis supporting that statement. The paper should clearly separate numerical agreement in the synthetic setting, qualitative consistency in real models, and evidence that real models approach the proposed bound.

Questions for authors

Q1. What are the main new technical contributions beyond the existing text-only result? Section 4 describes the main argument as applying Kalai and Vempala’s result when visual evidence is erased and then combining that case with the visible case. The paper also introduces attention and coverage analyses. It would help to distinguish more explicitly which steps require new arguments, which follow from the existing result, and which depend on additional modeling assumptions. This would make the significance of the visual extension easier to assess.
Q2. How does the matching construction account for familiar objects that are absent from the current image? Appendix H notes that replaying “cat” or “dog” on an image containing only a toaster is still an error, even though these objects appeared in training. I would appreciate an explanation of how such errors are included in the matching-generator analysis. In particular, how should readers reconcile these errors on observed facts with the claim that the generator matches a bound based on the monofact rate? A concrete explanation using the existing worked example would be helpful. Q3. Can the distractor-image comparison be reported on a matched set of target images? Table 8 uses 200 images for the single-image conditions but 60 for the four-image condition. Please clarify whether the reported increase in hallucination also holds when comparing the same 60 target images with and without distractors. Reporting uncertainty estimates would help readers assess the reliability of this difference. Q4. Could you provide some specific case studies to help readers better visualize how hallucinations change (for example, how hallucinations change or appear when visual information is removed)?

Reviewer_f3RKOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper extends the result of Kalai & Vempala (2024), which shows that calibrated language models must hallucinate (https://arxiv.org/pdf/2311.14648), to object hallucination in vision-language models. It models the visual pathway as an erasure channel with retention η and proves that a calibrated captioner must hallucinate at a rate of at least (1 − η) times the visual monofact rate. It also gives further results on visual token dilution, the trade-off between coverage and hallucination, and the gap between POPE and CHAIR. The authors test their theory on a synthetic dataset and only on Qwen3-VL-8B model, using POPE and CHAIR-style captions on COCO images.

Critical strengths

The paper asks an interesting and important question - 'how low can the hallucination rate of a calibrated VLM go? Most prior work on object hallucination focuses on mitigation methods, so a theoretical view of the lower limit is a useful direction. The observation that the closed-domain caveat of Kalai & Vempala does not directly apply to VLMs. The main bound is simple and easy to follow. When η = 0 (the model gets no information from the image), it reduces to the text bound of Kalai & Vempala, and when η = 1 (the model fully sees the image), it becomes zero. This makes it a natural bridge between the case where the model has no visual information and the case where it has full visual information.

Critical weaknesses
  1. The paper is hard to follow because many notations are used before they are defined. For example, L051-053: 'If the encoder recovered I, false objects could be filtered and the floor would vanish. A naive transplant of their bound would predict a floor of zero, since I would determine y.' I can assume I is the image, but y is only defined later in Section 3. Similarly, Equation 1 uses MFvMF_v and Mis, which are defined only in Section 3, and M in L071 is defined only later in Lemma 2. Introducing these notations briefly where they first appear would make the paper much easier to read.

  2. The abstract claims the theory is confirmed on Qwen3-VL within one percent error, but if I understand correctly, this number (0.42% slope error in Figure 2b) comes from the synthetic Zipf world, where the matching generator g* is built to follow the bound anyway. For the real Qwen3-VL experiments, the floor is computed from rough proxies. η is taken from a last-layer attention snapshot, which the authors themselves say is subject to the bias in Lemma 3, so the floor is probably underestimated. MF_v is estimated from only 2000 captions, so it may say more about the dataset specific study. I also could not find where the miscalibration term is measured.

  3. The empirical validation is limited to a single model family only on Qwen3-VL-8B, while the claims concern VLMs in general. Since Lemma 2 depends on architecture-specific factors such as the connector design and visual token count, the authors should evaluate at least two new architecturally different VLM families. All experiments also use COCO (POPE and 200 COCO captions); results on an additional benchmark such as AMBER, CHAIR dataset, with larger caption samples, would strengthen the claims. And, also the paper states that existing mitigation methods (VCD, OPERA, CMI-based decoding) occupy interior points of the coverage-hallucination line and cannot go below the floor, but none of them is evaluated.

  4. In the abstract 'A vision language model does not observe the image directly but rather a feature produced by an encoder, a connector, and softmax attention, which we model as a bounded evidence channel with retention η.' seems very strong claim. Current VLMs perform very much capable to image understand and also many methods proved that by visualizing the attention maps. It might creates problems in very long video-understanding types of task.

Questions for authors

see the weakness section.

Reviewer_tgmPOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies whether generatively calibrated vision-language models must incur object hallucinations when visual evidence is limited. The authors model visual input as an erasure channel with retention probability eta, propose a hallucination lower bound governed by the product of the visual monofact rate and the erasure probability, and claim to provide a matching generator.

It addresses a worthwhile research question, but I do not recommend acceptance in its current form. The main reasons are gaps in the proofs of the main lower bound and the matching upper bound, as well as clear errors in the scoring analysis and the conversion from mention-level to sentence-level CHAIR.

In particular, the proof of the central lower bound relies on an insufficiently justified transition from overall calibration to conditional calibration. The decision-theoretic results contain explicit algebraic inconsistencies, and the sentence-level CHAIR inequality does not appear to hold under the stated generality.

Critical strengths

The paper analyzes attention dilution, the relationship between coverage and hallucination rate, and the distinction between binary object-existence questions and free-form captioning. Experiments include a constructed Zipf world and POPE and CHAIR-style evaluations of frozen Qwen3-VL models.

Critical weaknesses
  1. The conditional calibration required for the main lower bound has not been established. D.2 does not establish compatibility between the generator-dependent calibration partitions before and after conditioning on the erasure branch. The total variation (TV) upper bound for mixtures in C.2 also does not imply control of the component-wise discrepancies. Moreover, the allocation of probability mass to the missing-mass bin required by the average-calibration version does not follow from correctness-based expected calibration error (ECE). The claimed scope of Theorem 1 is therefore not adequately justified.

  2. D.6 allocates the remaining probability mass to training facts, but being observed during training does not imply being true of the current image. Appendix H acknowledges these replay errors, yet the matching upper bound does not control this contribution. The current construction therefore does not establish the tightness claimed in Theorem 2.

  3. The necessary connection between attention mass and erasure probability is missing. Lemma 2 computes the softmax mass assigned to relevant tokens, whereas Assumption 2 requires either full revelation of the truth value or independence between the features and the truth value on the erasure branch. The former does not automatically imply the latter, and the discussion of proxy bias in Lemma 3 does not establish this connection. Interpreting Theorem 4 for real models therefore requires additional assumptions about the mechanism or error control.

  4. The Yes and No regions stated in Theorem 6 overlap when 0 < c < 1, and the comparison between these two actions is omitted. Theorem 7 does not rule out the model continuing to answer No, so increasing the false-positive penalty alone does not imply vanishing coverage on the erasure branch. This weakens the interpretation that instruction-tuned models occupy the gating vertex.

  5. The sentence-level CHAIR lower bound in Theorem 8 does not hold under the stated conditions. The main text does not assume independent mention errors. Even under independent mention errors with a common error probability independent of sentence length, replacing random sentence length by its mean through Jensen's inequality gives a bound in the opposite direction.

  6. The experiments do not establish tightness or the claimed one-percent accuracy for real models. The frozen-model experiments use caption-based estimates of the monofact rate and an attention proxy, without measuring the training-distribution statistics and generative miscalibration required by the theorem. The reported 0.42% relative slope error corresponds to a synthetic sweep. The CHAIR_i value of 0.276 and proxy lower bound of 0.026 in Table 8 do not establish tightness.

Questions for authors
  1. Can the authors clarify the common fact space, the conditional calibration partitions, and the observability of G, and prove that overall generative miscalibration controls miscalibration on the erasure branch? What additional assumptions are required for the average-calibration version based on ECE?

  2. After evaluating every replayed object against the current image, do the hallucination and miscalibration upper bounds for the matching construction still hold? What criterion for truth is used in the Zipf experiments, and how is it consistent with ECE = 0 in G.2?

  3. Which experiments and fits support the claimed one-percent relative error? How do the proxy quantities used for frozen models correspond to the theorem's parameters, and how is the unmeasured Mis_G handled?

If the authors can repair the main proof and matching construction within a consistent image-conditioned fact space, and correct the derivations concerning scoring and evaluation metrics, I would be willing to reassess the theoretical contribution.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.