- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes AdaVTP, a training-free adaptive visual token pruning framework for efficient inference in diffusion-based multimodal large language models (dMLLMs). After the first denoising step, AdaVTP estimates visual token relevance using prediction confidence and attention from question tokens and masked response positions. It then combines cumulative relevance screening with semantic coverage-based selection to determine a sample-specific visual token budget.
The authors evaluate AdaVTP on three backbones—LLaDA-V, LaViDA-LLaDA, and LaViDA-Dream—across 13 image and video benchmarks. They report improved task performance and inference efficiency compared with existing visual token pruning methods.
Critical strengths
- A practical training-free approach to visual token pruning.
Adaptively adjusting the visual token budget without additional model training is a practical approach to improving inference efficiency. In particular, performing pruning only once and reusing the selected visual tokens throughout subsequent denoising steps is well suited to reducing repeated visual token processing in dMLLMs.
- Promising performance and efficiency improvements over existing methods.
AdaVTP demonstrates improvements over directly relevant diffusion MLLM pruning methods, including RedVTP and D³ToM, across multiple benchmarks. Its ability to preserve relatively strong performance under aggressive visual token compression, together with measured end-to-end latency reductions, is encouraging.
Critical weaknesses
- The limited diffusion decoding configurations in the main experiments make it difficult to assess the general effectiveness of the proposed method.
The central motivation of this paper is to reduce the computational overhead caused by repeatedly processing visual tokens throughout the denoising process of dMLLMs. However, according to Table 7 in the appendix, most image understanding benchmarks use generation length, block length, and denoising steps of only 2/2/2 or 4/4/4.
Since AdaVTP performs pruning after the first denoising step, under the 2-step configuration, the pruned visual tokens are used for only one subsequent denoising step. It is questionable how representative such a setting is of general multi-step diffusion inference.
Although additional experiments with 16–32 denoising steps are provided on DocVQA, InfoVQA, and MathVerse, these experiments are restricted to LLaDA-V. More importantly, the paper lacks a systematic analysis of how performance and inference efficiency change when the number of denoising steps and block size are varied on the same backbone and benchmark.
Given that the effectiveness of visual token pruning can depend substantially on decoding configurations, the current results do not sufficiently establish whether AdaVTP provides consistent advantages across different dMLLM inference settings.
- The independent effectiveness and necessity of adaptive token allocation, the central contribution of the paper, are not sufficiently established.
AdaVTP combines confidence-guided relevance estimation, relevance mass screening, and semantic coverage selection to determine sample-specific visual token budgets. However, the independent improvements provided by these components are relatively modest compared with the overall performance gains.
For example, in Table 4, adaptive allocation improves performance over fixed allocation using identical relevance scoring and token selection by only 1.2 points on InfoVQA and 0.6 points on POPE. Although the appendix reports larger improvements under more aggressive compression, the paper does not directly demonstrate that the allocated token budgets accurately reflect the visual evidence requirements or question complexity of individual inputs.
Similarly, the improvement from confidence-guided relevance fusion over a simple combination of question and masked-response attention is relatively small. It therefore remains unclear whether the proposed adaptive mechanism genuinely captures input-specific visual complexity and allocates tokens accordingly.
Overall, the experiments demonstrate improvements from the complete framework, but provide insufficient evidence explaining why adaptive visual evidence allocation is necessary or how it fundamentally differs from existing relevance-based pruning approaches.
Questions for authors
1.Why do most of the main benchmarks use only 2–4 denoising steps? Can the authors compare AdaVTP and RedVTP under different denoising step counts and block sizes on the same backbone, reporting both task performance and inference efficiency?
2.Why are the longer-generation experiments restricted to LLaDA-V? Can the authors evaluate the proposed method on LaViDA-LLaDA and LaViDA-Dream under longer denoising trajectories?
3.How can the authors demonstrate that adaptive token allocation actually reflects question difficulty or the amount of visual evidence required by individual inputs? Can they analyze the relationship between the number of retained visual tokens and input characteristics such as visual complexity or question type?
4.What fundamental advantages does confidence-guided relevance fusion provide over simpler pruning strategies based on question attention or masked-response attention? How should the relatively modest performance improvements be interpreted?