- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SGPrune, a new training-free method for multi-view visual language model (VLM) visual token pruning in autonomous driving. SGPrune consists of two offline processing stages and one online inference stage. The experimental results in the paper have, to some extent, verified its effectiveness and efficiency.
Critical strengths
The article is easy to understand.
This article proposes a visual token-pruning framework that requires no training.
Critical weaknesses
It seems that the innovation of this article is a simple combination of the previous pruning schemes based on attention and diversity. What is the essential difference between this article and the previous methods?
In Table 1, the method proposed in this article does not show significant performance improvement compared to the previous SOTA method. The existing ablation experiments cannot convince me of the effectiveness of the three stages of SGPrune. Suggest adding visual results to demonstrate the effectiveness of SGPrune at each stage.
Questions for authors
What is the computational complexity of this pruning method now? What about the comparison with the previous method? Do we need to consider the offline operations of the first two stages of SGPrune when comparing computational efficiency?
4 comments
Weakness 2: Significance of the improvement in Table 1 and effectiveness of each stage
We thank the reviewer for this comment and for suggesting visual results for each stage. Your comment prompted us to add a significance analysis for Table 1 and a stage-wise qualitative figure, which helped us present the strengths of the paper more clearly.
Statistical significance. The revised PDF adds a paired frame-level bootstrap (10,000 resamples) of the judge scores on DriveLMM-o1 at 90.2% token reduction, for the configuration reported in Table 1 (appendix, “Statistical Significance on DriveLMM-o1”). The difference in Overall Reasoning between SGPrune and each baseline is as follows.
| Baseline | Difference | 95% CI | 99.5% CI |
|---|---|---|---|
| SparseVLM | +5.31 | [4.72, 5.94] | [4.45, 6.18] |
| HAWK | +2.81 | [2.28, 3.34] | [2.06, 3.58] |
| PruneSID | +1.14 | [0.63, 1.65] | [0.41, 1.86] |
| ZOO-Prune | +1.20 | [0.69, 1.71] | [0.47, 1.93] |
| Prune2Drive | +1.13 | [0.62, 1.66] | [0.42, 1.87] |
All intervals exclude zero, including the Bonferroni-adjusted 99.5% intervals. At this budget SGPrune scores 76.77 with 150 of 1,536 visual tokens, against 76.57 for the unpruned model.
Effectiveness of each stage. For each stage we report the end-task effect of removing or replacing it. Scores are Overall Reasoning on DriveLMM-o1 and the mean of FRM and QNS on MAPLM; percentages in the column headers are token reduction rates. Table 4 was part of the original submission; Tables 8 and 9 are new.
Stage 1, attention-based token selection. We fitted the subspaces on randomly chosen tokens instead of attention-selected tokens (Table 4).
| Calibration tokens | DriveLMM-o1 75.0% | DriveLMM-o1 90.2% | MAPLM 75.3% | MAPLM 90.1% |
|---|---|---|---|---|
| Random | 76.34 | 75.31 | 63.81 | 60.58 |
| Attention-selected (SGPrune) | 77.11 | 76.77 | 69.41 | 67.41 |
Attention-based selection is higher at every budget, by 0.77 and 1.46 points on DriveLMM-o1 and by 5.60 and 6.83 points on MAPLM.
Stage 2, subspace relevance scoring. Removing the relevance score (diversity-only selection) lowers the score by 0.93 and 1.22 points on DriveLMM-o1 and by 5.94 and 8.03 points on MAPLM (Table 4). Replacing it with simpler scores computed from the same tokens lowers the MAPLM score by 1.37 to 8.00 points (Table 8). On the full MAPLM test split (6,000 frame-question pairs), the 288 tokens with the highest relevance score hold 21.5% of the question's attention mass, against 9.8% for 288 random tokens, although the score does not read the question.
Stage 3, adaptive multi-view token pruning. Table 9 compares SGPrune with a variant that assigns equal token quotas to all views under the same total budget. Both use the same subspaces, relevance-weight strength and selection rule.
| Variant | DriveLMM-o1 75.0% | DriveLMM-o1 90.2% | MAPLM 75.3% | MAPLM 90.1% |
|---|---|---|---|---|
| Fixed equal allocation | 76.96 | 76.76 | 69.26 | 67.37 |
| SGPrune | 77.11 | 76.77 | 69.41 | 67.41 |
SGPrune is 0.01 to 0.15 points higher without any per-view quota being set in advance.
Visual results. Following your suggestion, we added a stage-wise qualitative comparison to the appendix section “Qualitative Visualization” (Figure 5, “Stage-wise qualitative comparison”). It shows (i) the tokens kept by random and by attention-based selection for the same image-question pair, (ii) the relevance scores of the same visual tokens under subspaces fitted on random and on attention-selected calibration tokens, and (iii) the tokens kept by fixed allocation with diversity-only selection and by SGPrune under the same budget of 150 tokens. The additions are highlighted in blue in the revised PDF.
Question 1: Computational complexity, comparison with prior methods, and offline operations
We thank the reviewer for this question. Your comment helped us report the online complexity, the measured cost and the offline cost separately.
Online complexity. The revised PDF states the online complexity in the appendix (“Efficiency: Timing Protocol, Complexity and Offline Cost”). For pooled visual tokens of width , subspace dimension and budget , SGPrune costs for the projections, for ranking, for the cosine-distance matrix and for the farthest-point steps. The vision encoder and the decoder are unchanged. Selection plus the projector on the kept tokens costs 30 GFLOPs on DriveLMM-o1 and 32 GFLOPs on DriveMM, less than the projector pass over all tokens in the unpruned model (103 and 125 GFLOPs). Counted the same way, PruneSID costs 122 and 195 GFLOPs, HAWK 195 and 262 GFLOPs, and ZOO-Prune 13.3 and 16.2 TFLOPs.
Measured comparison. The revised Table 5 reports the measured cost of every method at about 90% token reduction on one NVIDIA H200 at batch size 1, now with selection time, TTFT and estimated end-to-end latency. Below we reproduce all of its rows and columns. Latencies are in ms; parentheses give speedups over the unpruned model. Score is Overall Reasoning for DriveLMM-o1 and the mean of FRM and QNS for MAPLM.
DriveLMM-o1: 90.2% token reduction
| Method | Tokens | Sel. | Prefill | TTFT | end-to-end | FLOPs (%) | KV (MB) | Score |
|---|---|---|---|---|---|---|---|---|
| Vanilla DriveLMM-o1 | 1,536 | — | 171.5 | 205.8 (1.00×) | 6694 (1.00×) | 100.0 | 232.4 | 76.57 |
| SparseVLM (ICML 2025) | 150 | 3.8† | 55.3 | 89.5 (2.30×) | 6559 (1.02×) | 21.8 | 50.7 | 71.46 |
| HAWK (CVPR 2026) | 150 | 0.9 | 46.1 | 81.1 (2.54×) | 6567 (1.02×) | 22.0 | 50.7 | 73.96 |
| PruneSID (ICLR 2026) | 150 | 17.8 | 45.8 | 97.6 (2.11×) | 6678 (1.00×) | 50.7 | 75.63 | |
| ZOO-Prune (CVPR 2026) | 150 | 40.8 | 46.1 | 121.1 (1.70×) | 6595 (1.01×) | 73.1 | 50.7 | 75.57 |
| Prune2Drive (CVPR 2026) | 150 | 6.0† | 56.7 | 91.0 (2.26×) | 6592 (1.02×) | 24.0 | ||
| SGPrune | 150 | 45.9 | (2.53×) | 6626 (1.01×) | 21.3 | 50.7 | 76.77 |
DriveMM / MAPLM: 90.1% token reduction
| Method | Tokens | Sel. | Prefill | TTFT | end-to-end | FLOPs (%) | KV (MB) | Score |
|---|---|---|---|---|---|---|---|---|
| Vanilla DriveMM | 2,916 | — | 301.6 | 326.9 (1.00×) | 343.9 (1.00×) | 100.0 | 410.4 | 72.61 |
| SparseVLM (ICML 2025) | 288 | 8.2† | 49.6 | 75.0 (4.36×) | 91.6 (3.76×) | 16.0 | 65.9 | 50.58 |
| HAWK (CVPR 2026) | 288 | 0.8 | 27.8 | 53.6 (6.09×) | 70.4 (4.88×) | 15.8 | 65.9 | 49.48 |
| PruneSID (ICLR 2026) | 288 | 32.0 | 27.8 | 84.9 (3.85×) | 101.5 (3.39×) | 65.9 | 63.46 | |
| ZOO-Prune (CVPR 2026) | 288 | 60.6 | 27.9 | 113.5 (2.88×) | 130.2 (2.64×) | 50.3 | 65.9 | 63.83 |
| Prune2Drive (CVPR 2026) | 288 | 2.1† | 36.3 | 61.5 (5.31×) | 78.2 (4.40×) | 18.2 | ||
| SGPrune | 288 | 30.6 | (5.68×) | (4.62×) | 15.3 | 65.9 | 67.41 |
† Selection occurs inside the decoder and is already included in Prefill.
Time to first token (TTFT) includes the vision encoder, selection, the projector and LLM prefill; end-to-end adds decoding of the mean answer (192 and 2 tokens). SGPrune has the second lowest selection time and TTFT on both models after HAWK, and the highest score among the pruned methods.
Offline operations. The first two stages run once for a given model and camera setup and are not repeated at inference, so we report them separately from per-input latency. For DriveLMM-o1 on one NVIDIA H200, attention-based token selection takes 19 min and PCA subspace construction for all six views takes 1 min 45 s, 20 min 45 s in total. No model weights are updated. SparseVLM, HAWK, PruneSID and ZOO-Prune have no offline stage. Prune2Drive has one: its per-view ratio search is reported to converge within 3 H100 GPU hours [1]. The same appendix section also gives the timing protocol and the offline cost.
[1] Prune2Drive: A Plug-and-Play Framework for Accelerating Vision-Language Models in Autonomous Driving. CVPR 2026. arXiv:2508.13305v2, Section 3.2.
We Are Looking Forward to Your Reply
Thank you for your thoughtful and constructive review. In response to your comments, we clarified how SGPrune differs from prior attention-based and diversity-based methods and added the following experiments and analyses:
- Statistical significance: A paired frame-level bootstrap analysis with 10,000 resamples to assess the improvements over the compared pruning methods on DriveLMM-o1 at 90.2% token reduction.
- Additional ablations: Comparisons with alternative relevance scores and fixed per-view allocation under matched total token budgets (Tables 8 and 9).
- Stage-wise visualization: Qualitative comparisons illustrating all three stages of SGPrune, following your suggestion (Figure 5).
- Computational efficiency: Expanded latency reporting, including selection overhead, TTFT, and estimated end-to-end latency (Table 5), together with online complexity and offline preparation costs.
As the discussion period approaches its end, we would appreciate your feedback on whether these responses and additional results address your concerns. We would be happy to clarify any remaining points.
Thank you again for your time and consideration.
Best regards, The Authors
Weakness 1: Essential difference from prior attention-based and diversity-based pruning
We thank the reviewer for this question on novelty. Your comment helped us state more precisely where SGPrune departs from prior attention-based and diversity-based methods.
SGPrune differs from prior methods in three respects.
Attention is used only offline. Attention-based pruning methods compute text-to-visual attention for every input at inference time, for example inside the decoder as in SparseVLM (Table 5, †). SGPrune uses question-conditioned attention only during offline calibration to select tokens for subspace construction. Online selection needs no decoder pass and no attention map, and it runs before the visual projector. The selection overhead is 1.2 ms on DriveLMM-o1 and 1.6 ms on DriveMM (Table 5).
Relevance is a reusable subspace, not an attention score. The attention-selected tokens are summarized by a PCA subspace for each view, and a new token is scored by its normalized projection magnitude onto that subspace. This score is not interchangeable with simpler summaries of the same tokens: replacing it with the cosine similarity or distance to their mean, the unnormalized projection magnitude, or the distance to the fitted affine subspace lowers the MAPLM score by 1.37 to 8.00 points (Table 8).
Relevance and diversity are combined in one selection over all views. The relevance score enters as a rank-based weight on the farthest-point sampling distance, and selection runs once over the pooled tokens of all views under a shared budget, so the number of tokens kept in each view is determined for each input. Prune2Drive, the closest diversity-based method, applies farthest-point sampling with per-view pruning ratios that are searched offline; its paper reports that this search converges within 3 H100 GPU hours [1]. SGPrune requires no ratio search, and its offline step takes 20 min 45 s on one H200 for DriveLMM-o1. SGPrune determines the per-view counts without any per-view ratio being searched or set in advance; under matched budgets it scores 0.01 to 0.15 points above a variant with a fixed equal allocation (Table 9).
Neither component alone reaches the combined method. At 90.1% token reduction on MAPLM, relevance-only selection scores 58.95 and diversity-only selection scores 59.38, against 67.41 for SGPrune (Table 4).
[1] Prune2Drive: A Plug-and-Play Framework for Accelerating Vision-Language Models in Autonomous Driving. CVPR 2026. arXiv:2508.13305v2, Section 3.2.