- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces DAPrune, a dynamic token pruning method that selects which tokens to prune or store for each input. It uses lightweight modules to predict pruning decisions, trained with an auxiliary loss while freezing the LVLM backbone. Experiments on LLaVA-1.5, LLaVA-NeXT, and Qwen-2.5-VL show that the trade-off between performance and efficiency is well-balanced.
Critical strengths
-
DAPrune achieves promising results across various scenarios by optimizing the balance between performance and computational cost. For instance, with LLaVA-1.5, it reduces FLOPs by 64.9% while retaining 98.7% of the baseline performance. Further experimental results using LLaVA-NeXT and Qwen2.5-VL also substantiate the method's effectiveness.
-
Section 3 and Figure 1 highlight the limitations of manually configured pruning methods and demonstrate the need for dynamic pruning approaches. The method addresses this challenge by optimizing pruning positions and the number of retained tokens at each decoder stage based on the input. This approach offers an effective way of reducing redundant tokens while preserving model performance.
Critical weaknesses
-
The research motivation and novelty are somewhat limited. In lines 48–50, the authors claim that existing methods typically rely on pruning configurations preset for all inputs, such as uniform pruning rates or fixed pruning layers. However, existing works have already addressed the heterogeneity of input data and variations in token redundancy across layers. For instance, SparseVLM (ICML 25) and AutoPrune (NeurIPS 25) incorporate adaptive token retention mechanisms, while PDrop (CVPR 25) explicitly leverages redundancy characteristics across layers. Therefore, the authors should more clearly articulate the distinctions between DAPrune and these existing methods, and provide a more compelling justification for the necessity and novelty of simultaneously adapting both the pruning layers and retention rates.
-
Although DAPrune aims to reduce reliance on manual and pre-defined pruning configurations, it still requires several hyperparameters, e.g., number of pruning stages, candidate pruning layers, target retention ratios, and loss weights. While this paper examines the impact of sparsity weights and retention budgets, the sensitivity to the number of pruning stages and the selection of candidate layers has not been thoroughly investigated. It remains unclear how these configurations are determined and to what extent the reported performance gains depend on how well these parameters are developed. More importantly, reliance on multiple manually set hyperparameters appears to contradict the paper's primary objective, i.e., overcoming fixed, manual pruning schemes.
-
Comparisons with existing methods are insufficient and potentially unfair. Table 3 primarily compares DAPrune with existing methods under training-free conditions, yet DAPrune actually utilizes training data to train its routing and retention modules. Representative training-based methods cited in related literature, such as ATP-LLaVA (CVPR 25), are absent from these comparisons. This weakens the evidence supporting the claim that the proposed method outperforms existing approaches.
Questions for authors
-
Given that the depth of the decoder varies across model architectures (e.g., 32 layers for LLaVA-1.5-7B and 28 layers for Qwen2.5-VL-7B), do DAPrune hyperparameters (such as the number of pruning stages and target candidate layers) need to be manually reset or tuned for each model? If so, the complexity and cost associated with deploying DAPrune on new architectures would increase dramatically, potentially limiting its practicality and scalability in real-world applications.
-
Recent studies, such as LLaVA-Mini, have demonstrated that efficient multi-modal processing can be achieved using a single visual token. Since DAPrune and LLaVA-Mini are both training-based approaches, is it possible to compare the two to clearly demonstrate the advantages of the proposed method? Furthermore, the results from LLaVA-Mini raise concerns regarding whether general multimodal benchmarks are sufficient for token pruning evaluations (as only a single visual token is enough for correct answering). To further validate the effectiveness of the proposed method, could DAPrune be evaluated using benchmarks that require fine-grained visual perception, such as V* Bench or HR-Bench (4K/8K)?
-
Can DAPrune be extended to other practical use cases, such as LVLMs other than 7B and video-based reasoning tasks, to further validate its robustness and generalizability?
-
Could the authors provide some failure cases or specific challenging scenarios where DAPrune’s performance degrades? Such an analysis would help readers gain a deeper understanding of the method's limitations, grasp the conditions under which it performs well, and offer insights for further improvements and future directions.