- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates when visual information influences an answer, can an intervention also elicit that information in words? In exploring this question, it identifies a use-elicitation dissociation (termed the Umbra effect): visual relations influence answers under interchange yet respond weakly to elicitation with the tested payloads. Furthermore, the paper discovers a vision-to-answer handoff and payload-dependent elicitation across five checkpoints from two distinct model families.
Critical strengths
The mathematical formulation in the methodology section is academical.
Critical weaknesses
-
Does the Umbra effect identified in the paper possess generalizability? The paper presents Qwen2.5-VL-3B as a detailed case study, while other checkpoints are utilized to examine model variations. However, the detailed case analysis is conducted on only a single model, and merely five models are used for the overall evaluation. Consequently, the number of observed models is highly limited.
-
Regarding Figure 4, it is necessary to present the "Vision and answer positions exchange their relative influence" plots for Qwen2.5-VL-7B, Qwen2-VL-2B/7B, LLaVA-OneVision-7B, and additional models, in order to uncover universally existing patterns.
-
The rationale behind selecting the 25 specific layers out of the total 36 layers is not elaborated.
Questions for authors
Do the Umbra effect and the vision-to-answer handoff identified in the paper possess generalizability, rather than being confined to a single model? For specific details, please refer to the "Critical Weaknesses" section.