- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Align and Bridge, a visual-token pruning framework for VLMs. It consists of two components: VTA, which learns query-conditioned relevance from pre-decoder visual and textual embeddings without running the LLM decoder, and PDR, which progressively prunes visual tokens inside the decoder by combining the VTA prior, text-to-vision attention, and diversity-aware selection. The method is evaluated across multiple VLM backbones and multimodal benchmarks. It shows particularly strong grounding retention under aggressive pruning on LLaVA-1.5, LLaVA-NeXT, and Qwen3-VL-8B, while being less competitive on Qwen2.5-VL. The paper also provides extensive ablations and efficiency measurements.
Critical strengths
- Strong preservation of query-relevant visual evidence. The main empirical strength is the substantial improvement on grounding-sensitive tasks. For example, on Qwen3-VL-8B at the nominal 22.2% budget, the method preserves 87.78% relative grounding performance, compared with 71.04% for LearnPruner, while maintaining similar relative QA performance. This suggests that the method is effective at avoiding the loss of localized visual evidence under aggressive pruning.
- Broad and transparent evaluation. The paper evaluates multiple backbones, budgets, task types, and efficiency metrics, and includes extensive ablations of the aligner, prior fusion, diversity term, and scoring strategy. It also reports negative results, including weaker performance on Qwen2.5-VL and mixed contributions from several VTA components. This makes the empirical analysis relatively convincing and transparent.
Critical weaknesses
- The full-system gains are not cleanly attributable to the proposed “Align and Bridge” mechanisms. At matched per-sample token counts on Qwen3-VL-8B, VTA does not significantly improve RefCOCOg [email protected] over saliency or raw cosine, with gains mainly appearing in mean IoU. Moreover, encoder-only VTA is weaker than saliency on Qwen2-VL and Qwen2.5-VL, and on Qwen2.5-VL, diversity-only pruning outperforms the full PDR configuration across all reported metrics.These results make it difficult to determine whether the large full-pipeline gains primarily come from VTA and prior propagation, or from adaptive budgeting, progressive pruning, diversity selection, and backbone-specific schedules.
- The practical efficiency advantage is limited. Despite substantial theoretical FLOP reductions, wall-clock acceleration is modest. On Qwen3-VL-8B, the reported end-to-end speedup is 0.95$\times$, i.e., slightly slower than dense inference. In addition, selector overhead is excluded from the reported FLOP ratios, and the main tables compare nominal rather than strictly matched compute budgets. Therefore, the method demonstrates a favorable accuracy/grounding trade-off on several backbones, but its practical inference-efficiency advantage is less clear.
Questions for authors
- Contribution attribution. Could the authors isolate the effect of adaptive encoder budgeting from VTA and PDR, for example by comparing fixed-budget and adaptive-budget variants under the same full pipeline?
- Robustness of prior propagation. Why does the encoder prior help on LLaVA backbones but degrade performance relative to diversity-only pruning on Qwen2.5-VL? Clarifying whether this stems from the fixed schedule, backbone architecture, or prior quality would materially affect the assessment of the proposed “Bridge” mechanism.
- Matched-compute comparison. Could the authors report comparisons against the strongest relevant baselines at matched measured token retention or decoder FLOPs rather than nominal budgets? This would make the practical accuracy-efficiency trade-off substantially clearer.