- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates how vision-language models (VLMs) transfer answer-relevant visual information into text representations across decoder depth and how this schedule governs model compression. To overcome the non-additivity of standard activation patching and the non-causal nature of attention mass, the authors propose a reciprocal nested intervention protocol on matched image pairs that swaps clean donor and recipient visual hidden states at layer while preserving text states, yielding an exact telescoping decomposition of the total visual effect into block-level writer shares () and a causal closure boundary. Validating this framework primarily on Qwen3.5 and Qwen3-VL models, the authors demonstrate that visual-to-text transfer is heavily concentrated in just two dominant full-attention blocks (carrying 71%–83% of the total visual effect), which subsequently guides a "Phase Rule" for compression: preserving BF16 precision solely at the K/V projection inputs of top writer blocks under low-bit activation quantization. Finally, the paper reveals that transfer schedules are task-dependent and that causal closure calibrated on single-token likelihood margins fails under multi-step generative reasoning due to ongoing visual re-access during autoregressive decoding.
Critical strengths
- Mathematically rigorous causal accounting: The reciprocal nested intervention cleanly resolves the non-additivity flaw of conventional activation patching via a telescoping sum (). Using matched clean visual states instead of zero ablation convincingly avoids out-of-distribution intervention artifacts.
- Sharp isolation of the K/V reading port: By controlling for layer depth, module type, and quantization-error energy on A-OKVQA, the authors convincingly prove that protecting the K/V input of dominant writer L7 (only 6.8% of window error energy) recovers 94% of full-layer protection gains, ruling out trivial error-magnitude confounders.
- Candid diagnosis of the prefill-decode readout mismatch: The decode-only visual blocking experiment cleanly isolates why static-depth token eviction collapses on multi-step math and logical reasoning, offering a valuable cautionary insight for token pruning methods.
Critical weaknesses
- Reliance on aligned counterfactual image pairs: Computing the ledger requires paired images with opposite semantics and identical token grids (requiring synthetic pairs in Appendix F or forced padding in Appendix K). The authors should test whether unpaired random donor images can approximate under native dynamic resolution.
- Missing related work on layer-wise visual truncation and quantization: The finding that single-token VQA tolerates early truncation while multi-step reasoning requires deeper visual access closely mirrors prior work that must be cited and discussed: (1) Ghosh et al., "Do Vision Language Models Need to Process Image Tokens?", CVPR 2026 Workshop Oral (arXiv:2604.09425); (2) Zhu & Feng, "From Attenuation to Attention: Variational Information Flow Manipulation for Fine-Grained Visual Perception", arXiv:2604.12508; and (3) Ye et al., "ImpQuant: Fine-Grained Importance-Aware Quantization for Large Vision-Language Models", ICML 2026.
- Idealized compression setup without system-level metrics: Main experiments only simulate quantization on pre-closure visual-row activations while leaving text states and weights untouched. Reporting wall-clock latency, KV-cache memory savings, and end-to-end PTQ comparisons is needed.
Questions for authors
See the weaknesses. I will improve my score after the author addresses my concerns. I hope the author can supplement the corresponding experimental results before the end of the rebuttal and mark them in different colors in the updated article for my review and score improvement.