acceptodds
Under review as a conference paper at ICLR 2027

Hidden States, Open Secrets: Recognizing VLM Failures During Visual Prefill

Abstract

Vision-language models sometimes answer incorrectly despite retaining sufficient visual information in their internal representations. Such failures demonstrate the importance of how a model uses its representations, but also raise the question of when unsuccessful answers are preceded by distinguishable representations in the first place. We investigate this question across multimodal tasks by examining visual prefill, the image-token states formed even before access to any subsequent textual question. Prior work has shown that learned hidden states have geometric structures which reflect properties of the information they represent. As such, by collecting internal representations which usually lead to correct model responses, we obtain a reference for what favourable visual representations should geometrically be. By measuring the angular distance of other states to their nearest reference neighbors, we are able to find strong separation between usually correct and usually incorrect inputs on ScienceQA and ChartQA, with unsuccessful inputs tending to lie farther from successful references. However, on tasks like POPE and CV-Bench, the separation is much weaker. These results extend across distinct visual architectures and are visible within the first quarter of visual-token positions.

Primary area:
foundation or frontier models, including LLMs
Keywords:
vision-language models, visual reasoning, multimodal hallucination, representation geometry, manifold-based error detection, visual prefill, failure localization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Bq9FOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether visual-prefill hidden states, formed before a VLM observes the question, can predict subsequent answering failures. It introduces a reference-based geometric method that measures the angular distance between test representations and a “success reference cloud” constructed from usually correct samples. Experiments across six VLMs and eight benchmarks reveal strong task dependence: success–failure separation is pronounced on ScienceQA and ChartQA but weak on POPE and CV-Bench, and emerges within the first quarter of the visual sequence across multiple layers. The authors validate these findings using repeated-response majority labels, Hedges’ gg, image-group bootstrapping, comparisons with HALP, alternative distance metrics, deduplication analyses, and exploratory representation interventions.

Critical strengths

This paper presents a careful and relatively broad empirical study showing that hidden-state geometry during the visual prefill stage can distinguish downstream successes from failures on some tasks; evaluation across six VLMs and eight benchmarks is a clear strength.

Critical weaknesses
  1. Limited methodological contribution and empirical gains: The method essentially constructs a reference bank from successful samples and computes standard angular distance/kNN. The comparison against HALP is restricted to ScienceQA and ChartQA on Qwen3-VL. Compared to HALP, AUROC improvements are merely 0.003 and 0.051, with 95% confidence intervals for both differences containing zero. Moreover, the geometric method additionally uses reference samples with correctness labels and features from multiple layers, thus failing to demonstrate a clear advantage over existing latent-state probes.

  2. Core explanation lacks mechanistic evidence: The paper attributes task differences to the "incompressibility" of visual information, but neither operationalizes nor quantifies this concept, nor validates it through controlled experiments. Crucially, after directly reducing the target angular distance, no errors were corrected and no correct answers were altered across 64 ScienceQA inputs and 960 responses. Therefore, the observed geometric distance may merely be a dataset-specific statistical correlate rather than a behaviorally meaningful deficiency in visual representations.

Questions for authors

After matching the number of correctness labels, reference samples, and model layers, does the proposed geometric method still outperform HALP or other latent-state probes? If not, what is its substantive contribution over existing methods?

Reviewer_HoJ9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether a vision-language model’s image representations of the vision encoder already signal whether it will answer correctly. It scores each input by its mean angular distance to hidden states of reference samples where the model answers correctly. Across six architectures, incorrectly answered samples are far from these correct references on ScienceQA and ChartQA, but barely differ on POPE and CV-Bench. The authors attribute this to “incompressibility”: diagrams and charts require preserving many fine details before the question is known, while natural photos can be summarized safely. The signal appears within the first quarter of visual tokens and across decoder layers, and a small MLP on these distances predicts errors about as well as a hidden-state probe.

Critical strengths
  • The question is interesting; examining visual states before the question arrives helps isolate failure from the image part.
  • The paper covers 6 models and 8 benchmarks, showing consistent model behavior.
  • The paper discusses negative results in POPE and CV-Bench.
Critical weaknesses
  • Although interesting, the question seems to be weird to ask. How can we know if the model can solve the task without knowing the task? For example, given the same ScienceQA question in Fig 2, if we ask "How many capital F’s appear?", the model may not be able to answer. We cannot decide if the input is incompressible or not without seeing the question. In HALP, they also suggest that hidden states of query tokens are the most informative for the probe.
  • The method in the paper is more like outlier detection, and along with the above point, the contribution seems to be showing that failed samples in ScienceQA and ChartQA are outliers instead of detecting correctly answered samples.
  • The novelty of this paper seems limited. HALP already studies probing with hidden states in the vision encoder.
Questions for authors

What is the implication of this paper? Can we use this method to improve performance in practice?

Reviewer_136jOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether the correctness of a VLM's response can be predicted directly from its internal input representations prior to generation. Through empirical analyses across multiple VLMs and benchmarks, the authors observe divergent trends depending on the dataset and task.

Critical strengths

Exploring whether a VLM's internal hidden states encode a reliable pre-generation signal of response correctness is interesting with relevance to model interpretability.

Critical weaknesses
  • Inconclusive Findings and Lack of Deeper Explanation: Because the empirical results show divergent trends across different benchmarks, the paper ultimately leaves its central research question unanswered. It remains unclear whether response correctness is genuinely predictable from input representations as a fundamental model property, or if the observed probe performance is merely an artifact of specific dataset characteristics, task formulations, or model behaviors. A deeper investigation into why certain benchmarks exhibit predictability while others do not is needed.
  • Unclear Practical Utility: Given the inconsistent predictability across tasks and benchmarks, it is unclear how these findings can be practically leveraged in real-world developments and deployments.
Questions for authors

What is the single-sentence conclusion from this paper? Is recognizing VLM failures during visual prefill feasible or not? What practical guidance can readers learn from this paper?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.