- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Layered-VQA to investigate how visual composition and multi-turn delivery affect VLM performance. Experiments show that fragmenting visual scenes substantially reduces accuracy, while recomposition largely restores it. The authors further find that oracle-selected evidence can underperform complete scenes, and that correct answers do not necessarily indicate reliable visual grounding.
Critical strengths
- The paper offers a novel perspective by examining how visual evidence is composed and presented, rather than simply whether it is available. This provides useful insights into VLMs' limitations in integrating fragmented visual information.
Critical weaknesses
- The paper mainly attributes the performance degradation caused by visual fragmentation to the loss of scene composition. However, splitting a complete image into multiple layers also increases the number of visual inputs and may introduce many redundant visual tokens. Without controlling the visual token budget, it is difficult to determine whether the degradation primarily results from the loss of scene composition or the increased complexity of visual inputs.
Questions for authors
- Could the performance drop under visual fragmentation be partly attributed to the increased number of visual tokens and the dilution of relevant visual evidence, rather than the loss of scene composition itself?
2 comments
Reply
Thank you for your response. I have no further questions about this paper. Overall, I have decided to maintain my current score.
Response to Reviewer mdzy
Response to W1/Q1
We thank Reviewer mdzy for the review. We are glad the reviewer finds our perspective novel. We respond below. We show two new conditions, FULL-PAD (full image, padded) and CONCAT-V-TM (concatenated visual layers, token-matched), which separate token budget from decomposition.
So yes, the loss follows the decomposition, not the token count. Thank you again, this control makes the paper's central claim stronger.
We hope these new experiments - "FULL-PAD, CONCAT-V-TM, ORACLE-COMPOSED" address the reviewer's concerns and would be grateful if the reviewer would reconsider their score in light of these results. We are happy to answer any further questions.