- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes UniPrune, a training-free method for pruning visual tokens in multimodal large language models. After an early LLM layer, UniPrune constructs a positive support matrix from visual–textual relevance scores and defines the value of a retained token set through a Log-Sum-Exp objective. Tokens are selected greedily by repeatedly updating their marginal gains. The authors interpret this formulation as jointly modeling relevance, textual semantic coverage, and redundancy suppression. They establish that the regularized objective is monotone submodular, enabling the classical greedy approximation guarantee under a cardinality constraint. The performance evaluation covers three backbone models, nine image benchmarks, and six video benchmarks, with additional efficiency measurements on LLaVA-NeXT-7B. UniPrune achieves the highest average performance retention among the compared methods across the main evaluated settings and reports approximately 1.76× end-to-end acceleration. The supplementary material includes analyses of temperature, pruning position, token budget, and fine-grained capabilities.
Critical strengths
-
A compact formulation with clear mathematical properties. For a fixed support matrix, the regularized objective has a straightforward monotone-submodular structure and a closed-form marginal gain. The method requires no additional training, explicit attention maps, or token reconstruction, which makes it practically attractive to integrate.
-
Broad empirical coverage and competitive average performance. Tables 1–3 evaluate multiple backbones across image and video benchmarks, with consistent average improvements under tight token budgets. The supplementary budget, temperature, position, and capability analyses provide useful additional evidence. The efficiency evaluation also includes pruning overhead, making the reported acceleration more informative than measurements that exclude selection cost.
Critical weaknesses
-
The motivation does not sufficiently establish the advance over existing set-based methods. MMTok and CoIn already jointly consider complementary selection criteria, and MMTok also provides a submodular formulation with a greedy approximation guarantee. The paper needs to demonstrate more clearly why its LSE objective provides a substantive advantage beyond simplifying existing formulations. Moreover, equally weighting text tokens does not necessarily correspond to covering distinct, task-relevant semantics.
-
The experimental evidence does not fully establish robust accuracy–efficiency superiority. Average improvements conceal substantial task-specific weaknesses, including lower POPE performance than CoIn/MMTok and severe TextVQA degradation under aggressive pruning. Some gains are small and lack uncertainty estimates. Table 6 also combines latency from LLaVA-NeXT with average accuracy from other backbones, limiting conclusions about the overall tradeoff. Matched accuracy–latency evaluations and scaling measurements are needed, particularly because selection costs , which becomes quadratic in visual token count at a fixed retention ratio.
Questions for authors
-
Implementation consistency in Table D6: Why is the text-independent subtraction in “Negative redundancy” not canceled by the subsequent row-wise z-score normalization? Please provide the exact operation order and compare its normalized score matrix and selected indices with those of default cosine, including the numerical precision used.
-
Interpretation of Frozen-Marginal Top-K: What is the distribution of initial marginal gains under the actual temperature and ε settings? How frequent are exact or near ties, and how sensitive are the results to token order, numerical precision, and tie-breaking? How does this variant compare with random selection and raw-cosine Top-K?
-
Evidence for the core mechanism: Under matched pruning positions, normalization, and temperature settings, do the benefits of LSE over Hard-Max also hold on Qwen and InternVL? How sensitive is selection to prompt templates, function words, and repeated text tokens?
-
Robustness of the performance gains: Can the authors provide paired confidence intervals against the strongest baselines and explain the weaker results on POPE, TextVQA, and motion-related tasks? Does the contribution of Video-MMMU to the video average indicate a task-specific preference?
-
Matched efficiency evaluation: Can the authors report accuracy–end-to-end latency curves under the same backbone, input lengths, and token budgets, together with timing variability? How does selection overhead scale with visual token count, text length, and video frame count?
-
Reproducibility details: Please specify which text tokens participate in scoring, the order of normalization and temperature computation, the numerically stable implementation of exponential support and marginal gains, and the textual representations used for per-frame video selection.
Thank you for the detailed response. The matched LSE versus Hard-Max comparisons, bootstrap interval, and timing breakdown address most of my empirical concerns. The side-by-side objectives also clarify UniPrune’s differences from MMTok and CoIn. My remaining concern is whether equal-weight aggregation over text tokens reliably captures distinct semantic aspects, especially when prompts contain repeated or low-information tokens. I would keep my current rating while noting that the rebuttal has increased my confidence in the empirical results.