- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes Second-Look Prompting (SLP), a training-free approach that repeats the image-question pair to improve multimodal understanding in vision-language models. The authors argue that causal attention limits cross-modal information exchange during initial encoding, while repeated inputs enable question-guided visual representations. Extensive experiments across multiple models and benchmarks demonstrate consistent average improvements, supported by attention analysis and scratchpad probing.
Critical strengths
1.The paper is well written and presents a clear motivation.
2.The experimental evaluation is comprehensive and covers diverse models and benchmarks.
Critical weaknesses
1.Limited methodological novelty. The proposed method primarily relies on repeating existing inputs, which is a relatively straightforward modification of the prompting strategy. Although the authors provide interesting interpretations, the methodological contribution remains incremental compared with prior studies on prompt repetition and input ordering.
2.Insufficient evidence for the proposed mechanisms. The attention visualizations and scratchpad probing provide useful insights, but they do not fully establish that the proposed mechanisms are responsible for the observed improvements. In particular, the effectiveness of blank-image tokens varies across model architectures, raising questions about the generality of the scratchpad interpretation.
3.Limited performance gains and practical applicability. Despite consistent average improvements, the overall gains are relatively modest, and some tasks, such as visual grounding, experience performance degradation. Furthermore, duplicating the input increases token consumption and training costs, while the evaluation is largely restricted to single-image scenarios, leaving its applicability to more complex multimodal inputs unclear.
Questions for authors
1.Can the authors provide stronger causal evidence that question-guided visual attention directly contributes to the performance improvements, rather than merely correlating with them?
2.Beyond the existing repetition and blank-image controls, how does SLP compare with token- or compute-matched baselines? To what extent can the improvements be attributed to additional computation rather than the proposed encoding mechanism?
5 comments
Response to Reviewer hzAc (2/2)
To Q1:
Q1: Can the authors provide stronger causal evidence that question-guided visual attention directly contributes to the performance improvements, rather than merely correlating with them?
We intervene on V₂'s attention during standard generation on LogicVista with Qwen2.5-VL-32B (Table R4), masking only V₂'s attention to (V₁, T₁) (V₂-no-read) using FlexAttention [1]. This lowers accuracy from 59.28% to 55.92% and removes the entire SLP gain over ⟨V, T, T⟩ (56.82%), showing that question-guided attention in V₂ directly contributes to the improvement. The ⟨V, V, T, T⟩ order control in Q2 provides converging evidence without altering the attention mask. We have added this result to Appendix E.4.
Table R4. Attention-blocking intervention on LogicVista (Qwen2.5-VL-32B, %)
| ⟨V, T, T⟩ | SLP | V₂-no-read |
|---|---|---|
| 56.82 | 59.28 | 55.92 |
To Q2:
Q2: ... how does SLP compare with token- or compute-matched baselines? To what extent can the improvements be attributed to additional computation rather than the proposed encoding mechanism?
We compare SLP with two token- and prefill-compute-matched baselines (Table R5).
- ⟨V, T, Spaces, T⟩ replaces V₂ with length-matched spaces. It keeps the additional computation but provides no question-conditioned second view of the image nor a visual scratchpad, and yields no average gain.
- ⟨V, V, T, T⟩ places V₂ before the question, which is intra-modal repetition: V₂ attends to V₁ but not to T₁. Its average gain of +0.29 points is consistent with the +0.4-point gain of ⟨V, V, T⟩ over ⟨V, T⟩ (Table 2).
SLP outperforms both baselines on all three models, with an average gain of +0.93 points. Additional computation alone thus explains little of the improvement, and resolving the cross-modal partial-observation problem brings further gains beyond intra-modal repetition. Per-benchmark results are added to Appendix E.4.
Table R5. Gains over ⟨V, T, T⟩ of token- and prefill-compute-matched baselines
| Model | ⟨V, T, T⟩ (%) | ⟨V, T, Spaces, T⟩ | ⟨V, V, T, T⟩ | SLP |
|---|---|---|---|---|
| MiniCPM-o 4.5 | 67.90 | +0.03 | +0.58 | +0.89 |
| Gemma-3-12B-IT | 55.73 | +0.13 | +0.45 | +0.91 |
| Qwen2.5-VL-32B | 68.76 | −0.16 | −0.16 | +0.99 |
| Average | 64.13 | 0.00 | +0.29 | +0.93 |
Thank you again for your support and thoughtful suggestions, which have strengthened both our mechanism evidence and the practical case for SLP. We would be happy to discuss any remaining questions.
References
[1] Dong et al. Flex Attention: A Programming Model for Generating Optimized Attention Kernels. MLSys 2025.
Dear reviewer hzAc,
We hope our responses have addressed your questions. Please let us know if there is anything else we can clarify or improve before the discussion ends. Your feedback is very important to us, and we welcome any further discussion.
Thanks again for your time and effort in reviewing our submission.
Response to Reviewer hzAc (1/2)
We appreciate your valuable feedback and support of our work. We detail our response below and please kindly let us know if our response addresses your concerns.
To W1:
We appreciate your positive assessment of our interpretations and agree that the prompt modification itself is simple. Our goal is to identify and analyze the cross-modal partial-observation problem in causal VLMs, where ⟨V, T⟩ prefill leaves visual tokens unconditioned on the question. A minimal, parameter-free intervention lets us isolate this encoding gap cleanly, just as simple interventions like zero-shot chain-of-thought [1] and register tokens [2] helped reveal underlying model behaviors.
Practically, this simplicity makes SLP an easy plug-in across closed- and open-source VLMs (as noted by Reviewer 5JNq), and our new results on Qwen3.8-27B at xhigh (+2.91 points, Table R2) show that recent model advances have not closed this gap. Beyond the prompt rule itself, our mechanism and training analyses offer concrete guidance for future multimodal designs. We therefore believe that the methodological simplicity of SLP strengthens rather than diminishes its value.
References
[1] Kojima et al. Large Language Models are Zero-Shot Reasoners. NeurIPS 2022.
[2] Darcet et al. Vision Transformers Need Registers. ICLR 2024.
To W2:
We respectfully note a misunderstanding of the blank-image experiment. Blank images do not need to work across every vision encoder for the scratchpad mechanism of SLP to hold. In actual SLP (Copy V), all evaluated architectures, including Qwen2.5-VL-32B, show strong scratchpad readout and generation gains (Table R1). Although Appendix E.3 offers one possible architectural explanation for Qwen2.5-VL's near-zero blank-image readout, our argument does not rely on it. Crucially, blank images improve accuracy precisely where blank-image readout succeeds (MiniCPM-o 4.5 and Gemma-3-12B-IT) and yield no gain where readout fails (Qwen2.5-VL-32B), serving as positive-and-negative control evidence for the scratchpad mechanism (as also noted by Reviewer 5JNq). Per-benchmark results are added to Appendix E.4.
Table R1. Scratchpad probe gains (Table 3) and normal-generation Δ over ⟨V, T, T⟩ (six-benchmark average, points)
To W3:
Thank you for raising these practical points. We agree on the scope boundaries discussed in our Limitations section, where image duplication can interfere with fine coordinate grounding on RefCOCOg and extending SLP beyond single-image ⟨V, T⟩ inputs remains future work.
Regarding practical value and cost, SLP is applied directly at inference without any need for training (our continued-training experiment only studies how the capability develops). Even on Qwen3.8-27B at its highest effort (xhigh), SLP improves all five challenging benchmarks by +2.91 points on average (Table R2, Appendix K). We additionally provide a lightweight variant, SLP-quarter, which downsamples only V₂ to a quarter of its visual-token budget. It slightly improves accuracy over full SLP while keeping the additional visual tokens within 28% of ⟨V, T⟩ (Table R3). Per-benchmark results are added to Appendix I.2.
Table R2. Qwen3.8-27B (xhigh) accuracy on five challenging benchmarks (%)
Table R3. Cost and accuracy of SLP-quarter