- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CiVi, a training-free visual token pruning method based on the concept of irreplaceability. The authors argue that traditional metrics for saliency and redundancy fail to adequately capture whether visual information can be compensated for by other tokens. To address this, CiVi introduces an adaptive feature-space perturbation strategy to estimate intervention responses, combining this with factual saliency and non-redundancy for token selection. Experiments across various image and video MLLMs demonstrate that the method is competitive in terms of both performance and inference efficiency, particularly in high-intensity pruning scenarios.
Critical strengths
-
The paper explores visual token pruning from the perspective of irreplaceability, offering an interesting angle that goes beyond traditional concepts of token importance.
-
Experiments cover a wide range of backbone networks, pruning ratios, and image/video benchmarks, showing excellent performance at high compression rates.
-
The paper includes component ablation studies, validation of decoder interventions, and inference latency measurements, providing strong empirical support for the proposed method.
Critical weaknesses
-
The paper emphasizes that a token's irreplaceability depends on whether its information can be compensated for by other retained tokens. However, CiVi employs a static Top-K selection strategy without explicitly accounting for the dynamic changes in the retained set. Furthermore, while decoder interventions rely on text queries, the actual selection scores are derived entirely from visual features. Consequently, the method appears more like a combination of saliency, perturbation sensitivity, and non-redundancy rather than a direct estimate of irreplaceability.
-
The method assigns higher perturbation ratios to visual features with more concentrated distributions, yet the rationale for this choice is not fully explained. For instance, images with uniform backgrounds containing only a few key objects might be subjected to stronger perturbations, potentially introducing additional noise into the token scoring process. Although the paper evaluates various perturbation types and sample sizes, it lacks a direct comparison against a fixed perturbation ratio, and the rationale behind several heuristic design choices is insufficiently explained.
-
According to the formulation, unperturbed tokens receive an intervention response score of zero, and these scores feed directly into the log-domain score fusion process. Given that only three perturbation samples are used by default, token rankings may depend to some extent on the specific random masks and the frequency with which each token is perturbed. While deterministic seeds ensure reproducibility and the paper explores different sample sizes, it reports neither performance variance across independent random seeds nor the stability of the selected token sets.
-
A strong response in the feature space to random perturbations does not necessarily imply a significant impact on the decoder's predictions. Although Figure 4 demonstrates some consistency between proxy rankings and decoder intervention results, that comparison relies on specific perturbation mechanisms and measures the impact of individual token replacements; it does not assess whether the retained token set can compensate for the removed visual evidence. Furthermore, a direct comparison with existing perturbation-based sensitivity analysis methods (such as ZOO-Prune) would help clarify the unique contribution of the proposed intervention response scores.
Questions for authors
Please see the weaknesses above.