- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper investigates the Mirage phenomenon in vision-language models (VLMs), where models can generate confident responses to visual questions even when they cannot access the corresponding images. The authors propose Mirage Probes, a contrastive probing framework that aims to study how this behavior is represented in the internal activations of VLMs.
This framework generates rephrased versions of the same question and identifies Mirage-like and Non-mirage-like responses to construct contrastive samples. Subsequently, the authors extract activations from residual streams, MLP outputs, post-attention outputs, and individual attention head outputs, and train different types of probes.
Experiments on multiple VLMs and visual question answering benchmarks show that the models' internal activations can be used to predict Mirage-related behaviors, and probes based on activation differences achieve higher accuracy. The authors further propose that Mirage may involve two distinct regimes, namely Textual Biases and Spurious Images, and study these phenomena through additional probing and intervention experiments.
Critical strengths
-
The research motivation is meaningful. This study examines an important issue in VLMs: whether the models' seemingly correct visual responses are truly based on visual evidence. Studying the Mirage phenomenon from the perspective of the internal representations of models is a worthwhile direction that can help us further understand the limitations of current VLMs.
-
The experimental design is reasonably sound and the evaluation scope is relatively broad. The authors attempt to reduce the differences in content between positive and negative samples by constructing contrastive samples, which is a valuable design. In addition, the authors study various activation positions and probing strategies, and conduct evaluations on multiple models and benchmarks. The supplementary intervention experiments further indicate that some of the identified internal representations are indeed related to Mirage-related behaviors.
Critical weaknesses
-
Prompt changes may have an impact on Mirage behavior. In Section 3.2, the authors construct contrastive samples by generating four rewritten versions of each original question. However, the same image and semantically similar questions can produce two different types of responses, Mirage-like and Non-mirage-like, which raises questions about the influence of prompt changes. It is not yet clear whether these differences reflect different internal representations related to Mirage or whether the model is merely sensitive to different question formulations. In particular, even if the meaning of the question remains unchanged, certain rewrites may alter the model's confidence in its answers or its tendency to refuse to answer. Therefore, I believe it is necessary for the paper to further analyze how prompt changes affect the Mirage labels and whether the constructed contrastive samples can reliably reflect the behavior that the authors intend to study.
-
The authors do not adequately explain why model-generated answers need to be included during activation extraction. As shown in Figure 3, the authors feed the complete conversation, including the model-generated answers, back into the VLM to extract internal activations. However, the paper does not adequately explain the motivation for doing so. Since the Mirage labels are also determined by comparing model answers, the extracted activations may contain information related to the answers, such as answer content, confidence, or refusal patterns. Therefore, higher probing accuracy does not necessarily mean that the probe has identified the true internal mechanism that causes Mirage. I think the authors should add experiments that use only activations extracted before answer generation to further clarify this issue.
-
The generalization ability of the trained probes is insufficiently evaluated. The paper mainly uses data from different splits of the same benchmark to train and evaluate the probes. Although the authors use held-out test samples and extend the experiments to more models and datasets, such an evaluation may not be sufficient to prove that the probes have learned general Mirage-related representations rather than patterns specific to certain benchmarks. In particular, it is not clear whether a probe trained on one benchmark can remain effective on an unseen benchmark. I suggest that the authors add cross-benchmark evaluations, for example, by training a probe on one dataset and then directly testing it on other datasets without retraining. Such experiments can provide stronger evidence for the generalization ability of the proposed framework.
Questions for authors
Q1. Influence of prompt variations. In Section 3.2, the authors construct contrastive samples by rewriting the same question in different ways. Have the authors analyzed whether the observed Mirage/Non-Mirage differences are mainly caused by prompt sensitivity? For instance, could certain rewrites potentially alter the model's confidence or tendency to refuse to answer, rather than truly changing the model's reliance on visual information? An additional analysis of the impact of different prompt rewrites on the final labels would help clarify this issue.
Q2. Motivation for response-conditioned activation extraction. In Figure 3, why do the authors choose to include the model's generated answer during activation extraction instead of only using the image and question? Have the authors tried using only activations extracted before answer generation for probing and comparing the results of the two settings? Such a comparison would help determine whether the probe captures Mirage-related internal representations or mainly reflects the information already contained in the generated answer.
Q3. Cross-benchmark generalization. Can the authors further explain whether the trained probes can generalize across different benchmarks? For example, can a probe trained on VQA-RAD be directly used on MMMU-Pro or MedXpertQA without retraining? Although the paper includes evaluations on multiple benchmarks, conducting cross-benchmark transfer tests on the main activation probes would provide stronger evidence that the identified representations are not specific to a particular dataset. Additionally, could the authors clarify whether all the rewrites of the same original question are strictly kept within the same data split?