- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
ERASE is a training-free, two-stage visual-token pruner. Stage 1, before the LLM, smooths a luminance map and keeps a patch if any pixel's Sobel gradient exceeds τE = (1 - τ)·median(Ỹ) or if its 256-bin intensity entropy exceeds the mean patch entropy H̄ (Eqs 1-5), so its retention r1 varies per image. Stage 2 ranks survivors by instruction-to-vision attention a_v (Eq 7), adding the Stage-1 cues with weight w at layer 2 (Eq 8) and using attention alone near two-thirds depth; Algorithm 1 sets both keep ratios from r1 and the target layer-averaged retention. With τ = 0.45 and w = 0.2 fixed, ERASE is tested on five Qwen and InternVL3 backbones, eight benchmarks and up to nine baselines at 45/35/25% retention, reporting the best average at every tier.
Critical strengths
-
ERASE avoids the encoder-attention memory cost at native resolution. Because Stage 1 scores tokens from luminance statistics rather than encoder attention, ERASE prunes Qwen2.5-VL-7B inputs of up to 16K visual tokens at 19.15 GB peak memory, while the evaluated implementations of AgilePruner, VScan and Nüwa need 37.38 to 39.59 GB even on downscaled inputs (Table 8). On encoders without a [CLS] token, like Qwen2.5-VL's, that is the clearest practical separation from [CLS]-dependent pruners such as these three.
-
The accuracy lead over the [CLS]-dependent baselines survives the one comparison free of the resolution confound. On InternVL3-8B, whose [CLS] token lets AgilePruner, VScan and Nüwa run without OOM (Sec 4.2), ERASE has the best average at all three tiers, 72.39 against 69.51 for Nüwa at 25% retention, a 2.88-point margin (Table 3). This is the decisive reference point for me, because on Qwen those baselines see downscaled inputs, which mixes pruning quality with input resolution.
-
The evaluation is broad and its numbers check out. ERASE has the top average at every tier on all five backbones against up to nine baselines (Tables 1-3, 6-7), and every average and stated margin I recomputed matches the printed value to rounding. The result holds at the most aggressive tier: at 25% retention on Qwen2.5-VL-7B, ERASE keeps 95.70% of base accuracy (Table 1). A lead this consistent and this checkable is what makes the efficiency-accuracy claim credible.
Critical weaknesses
-
The novelty is a combination of known parts, and the new Stage-2 piece adds under a point in the full system. Stage 1 builds on Sobel and Canny-style thresholding (Eqs 1-4), and iLLaVA, VScan and Nüwa already prune before and inside the LLM (Sec 2.2.2). PruneSID, a compared baseline, already adapts its compression ratio to image complexity. The 3.14 points credited to visual guidance (Sec 3.3) are retained accuracy with Stage 1 disabled (Fig 7 caption). With Stage 1 on, task accuracy moves from 70.10 to 71.01, +0.91 (Table 9). Moving Table 9 into the main text and scoping the claim costs no experiments.
-
The paper claims matched retention, but ERASE's per-tier realized retention is never reported and sometimes falls below the nominal tier. Per A.1 (L675-677), Stage 2 is skipped whenever Stage 1 already meets the budget, so "the reported retention ratios serve as upper bounds for ERASE." On Qwen2.5-VL-7B, Stage 1 alone keeps 41.28% of DocVQA tokens on average (Table 4), so ERASE's 45% DocVQA cell in Table 1 must run below budget on average. This may understate ERASE rather than flatter it, but readers cannot compare the operating points at equal cost. Reporting realized retention per cell, and matching baselines to it, would close this.
-
The Qwen headline margin leaves out the strongest baseline, which the appendix shows is nearly tied. Sec 4.2 reports +3.95 over iLLaVA as the margin over the strongest Qwen2.5-VL-7B baseline. Nüwa, the strongest baseline on InternVL3 at every tier (Table 3), appears on Qwen only in appendix Table 8, on downscaled inputs, because its evaluated implementation needs full attention maps without a [CLS] token (A.3.2). Even so it reaches 74.85 at 25%, 0.72 below ERASE (Table 8). Reporting this margin in the main text, and scoring Nüwa without the full map as iLLaVA does (Sec 2.2.2), would show whether the gap survives.
Questions for authors
- Please report ERASE's realized layer-averaged retention per cell in Tables 1-3. How far below each nominal tier does it fall (for example DocVQA at 45%), and does the ranking hold when baselines run at the realized value?
- Did Table 8 follow Nüwa's own Qwen2.5-VL setup (its paper reports Qwen2.5-VL-7B results) or your adaptation? Could its attention scores be computed via the FlashAttention-2 normalizer, as for iLLaVA, to run it at full resolution?
- On Qwen3-VL-8B, ERASE trails PruneSID on MathVista at all three tiers (47.10 vs 50.40 at 25%, Table 2). Is there a systematic cost on math and reasoning tasks, and would you scope the OCR-gains statement in Sec 4.2 accordingly?
- SparseVLM (ICML 2025, arXiv 2410.04417) rates visual tokens from decoder self-attention and needs no encoder [CLS] token, so it fits both Qwen and InternVL3. It is neither cited nor compared; can you add it on one backbone?
- Algorithm 1 holds each image at the target unless Stage 1 falls below it (L714-720), so Tables 1-3 adapt only the stage split. Table 4's +0.52 adaptive gain omits Stage 2 (L406-407); can Table 9 test a fixed Stage 1?
7 comments
W2/Q1. Realized retention and comparison fairness (1/2)
We thank the reviewer for this careful observation. As the reviewer correctly notes, Stage 2 is skipped when Stage 1 already retains fewer tokens than the target, so ERASE's realized retention can fall below the nominal retention ratio. We agree that evaluation at these retention is needed to compare baselines at equal cost, and we address it below.
To clarify the actual operating points, Tables wrQy-2 to wrQy-4 report ERASE's realized layer-averaged retention for each benchmark and target ratio in Tables 1–3, respectively. Bold values indicate deviations of at least 2 percentage points from the target.
Table wrQy-2. Realized layer-averaged retention (%) per cell of ERASE in Table 1, Qwen2.5-VL-7B
| Target | TextVQA | ChartQA | DocVQA | InfoVQA | MathVista | MMBench | RWQA | MMStar |
|---|---|---|---|---|---|---|---|---|
| 45% | 44.70 | 43.89 | 37.08 | 44.41 | 40.21 | 43.49 | 44.87 | 42.23 |
| 35% | 34.93 | 34.84 | 32.26 | 34.90 | 33.69 | 34.61 | 34.97 | 34.28 |
| 25% | 24.96 | 24.96 | 24.29 | 24.99 | 24.78 | 24.91 | 24.97 | 24.88 |
Table wrQy-3. Realized layer-averaged retention (%) per cell of ERASE in Table 2, Qwen3-VL-8B
| Target | TextVQA | ChartQA | DocVQA | InfoVQA | MathVista | MMBench | RWQA | MMStar |
|---|---|---|---|---|---|---|---|---|
| 45% | 44.75 | 44.74 | 37.58 | 44.42 | 42.10 | 44.12 | 44.88 | 42.93 |
| 35% | 34.91 | 34.87 | 32.41 | 34.91 | 34.25 | 34.61 | 34.96 | 34.40 |
| 25% | 24.93 | 24.88 | 24.32 | 24.99 | 24.82 | 24.70 | 24.96 | 24.76 |
Table wrQy-4. Realized layer-averaged retention (%) per cell of ERASE in Table 3, InternVL3-8B
| Target | TextVQA | ChartQA | DocVQA | InfoVQA | MathVista | MMBench | RWQA | MMStar |
|---|---|---|---|---|---|---|---|---|
| 45% | 44.76 | 44.22 | 40.14 | 44.93 | 41.36 | 43.97 | 44.92 | 43.24 |
| 35% | 34.94 | 34.94 | 33.64 | 34.99 | 34.01 | 34.78 | 34.97 | 34.60 |
| 25% | 24.97 | 24.97 | 24.71 | 24.99 | 24.81 | 24.94 | 24.97 | 24.92 |
As anticipated by the reviewer, the largest deviation occurs on DocVQA at 45% target retention, where the realized retention is 37.08%, 37.58%, and 40.14% on Qwen2.5-VL-7B, Qwen3-VL-8B, and InternVL3-8B, respectively. Most other configurations remain close to their target ratios, with all cells at 25% retention deviating by less than 1 percentage point.
W2/Q1. Realized retention and comparison fairness (2/2)
To assess the impact of these differences on comparison fairness, we re-evaluated the baselines at ERASE's realized retention for all cells with deviations of at least 2 percentage points. Tables wrQy-5 to wrQy-7 summarize the results. Values in parentheses indicate the original nominal-tier results, while the other values represent the re-evaluated results. Cells marked with † retain their original values because ERASE's realized retention differs from the target by less than 2 percentage points. The average accuracy (Avg.) is recomputed across all eight benchmarks using the updated results.
Table wrQy-5. Re-evaluation of Table 1 with matched realized layer-averaged retention
| Retain | Method | DocVQA | MathVista | MMStar | Avg. |
|---|---|---|---|---|---|
| 100% | Base | 94.82 | 65.10 | 65.53 | 78.97 |
| 45% | FastV | 91.28 (93.19) | 63.20 (63.20) | 61.20 (62.00) | 76.00 (76.34) |
| DART | 79.68 (85.35) | 61.00 (62.00) | 62.67 (62.53) | 71.43 (72.25) | |
| CDPruner | 82.53 (85.41) | 56.20 (56.70) | 61.73 (63.07) | 69.76 (70.35) | |
| PruneSID | 85.88 (90.32) | 61.20 (61.30) | 63.80 (62.73) | 73.39 (73.82) | |
| iLLaVA | 91.38 (93.09) | 63.20 (64.30) | 62.47 (62.27) | 76.31 (76.64) | |
| SparseVLM | 87.95 (91.05) | 61.40 (62.90) | 61.73 (62.47) | 74.03 (74.70) | |
| ERASE | 94.22 | 64.20 | 64.13 | 77.63 | |
| 35% | FastV | 88.58 (90.27) | 62.30† | 60.20† | 73.85 (74.06) |
| DART | 75.25 (78.17) | 57.70† | 61.33† | 67.72 (68.09) | |
| CDPruner | 80.44 (81.57) | 55.40† | 62.00† | 67.68 (67.82) | |
| PruneSID | 83.85 (86.07) | 60.40† | 62.20† | 70.87 (71.14) | |
| iLLaVA | 90.20 (90.91) | 60.70† | 62.40† | 74.82 (74.91) | |
| SparseVLM | 84.23 (86.75) | 59.40† | 60.40† | 70.49 (70.80) | |
| ERASE | 94.12 | 63.70 | 63.93 | 77.18 |
Table wrQy-6. Re-evaluation of Table 2 with matched realized layer-averaged retention
| Retain | Method | DocVQA | MathVista | MMStar | Avg. |
|---|---|---|---|---|---|
| 100% | Base | 95.75 | 69.10 | 66.60 | 79.59 |
| 45% | FastV | 74.04 (81.92) | 47.10 (49.00) | 53.07 (55.07) | 62.93 (64.40) |
| DART | 75.91 (82.69) | 55.80 (56.10) | 56.00 (59.53) | 66.93 (68.26) | |
| CDPruner | 65.89 (72.63) | 40.50 (42.60) | 50.60 (52.93) | 60.06 (61.45) | |
| PruneSID | 91.53 (93.46) | 58.60 (59.90) | 61.20 (60.40) | 73.45 (73.76) | |
| iLLaVA | 91.65 (93.02) | 44.00 (45.40) | 60.07 (60.53) | 70.94 (71.35) | |
| ERASE | 94.77 | 54.70 | 61.53 | 74.88 | |
| 35% | FastV | 66.05 (70.07) | 43.60† | 49.00† | 57.05 (57.55) |
| DART | 69.35 (73.30) | 49.80† | 55.07† | 62.14 (62.64) | |
| CDPruner | 61.12 (63.57) | 38.00† | 49.33† | 55.92 (56.23) | |
| PruneSID | 88.59 (90.46) | 52.80† | 60.33† | 68.96 (69.19) | |
| iLLaVA | 90.64 (90.74) | 44.30† | 57.93† | 68.81 (68.82) | |
| ERASE | 94.29 | 51.00 | 59.07 | 72.82 |
Table wrQy-7. Re-evaluation of Table 3 with matched realized layer-averaged retention
| Retain | Method | DocVQA | MathVista | Avg. |
|---|---|---|---|---|
| 100% | Base | 91.98 | 68.80 | 78.77 |
| 45% | FastV | 77.83 (82.17) | 64.50 (64.00) | 74.06 (74.54) |
| DART | 73.84 (77.59) | 67.80 (66.30) | 71.51 (71.80) | |
| CDPruner | 61.25 (66.87) | 56.30 (57.80) | 64.53 (65.42) | |
| PruneSID | 69.40 (76.02) | 66.00 (66.10) | 70.42 (71.26) | |
| iLLaVA | 65.92 (68.65) | 59.00 (58.40) | 64.81 (65.08) | |
| VScan | 87.22 (88.73) | 60.30 (60.90) | 75.19 (75.45) | |
| Nüwa | 85.98 (87.13) | 65.00 (65.90) | 75.82 (76.07) | |
| AgilePruner | 80.97 (83.83) | 62.00 (63.40) | 73.31 (73.85) | |
| ERASE | 87.95 | 66.40 | 76.37 |
As expected, most baseline accuracies decrease under the reduced token budgets. Importantly, ERASE continues to achieve the highest average accuracy across all three models and retention tiers after accounting for the realized retention differences. The relative rankings among baselines also remain nearly unchanged, except for a minor swap between iLLaVA and CDPruner on InternVL3-8B. These results confirm that the main conclusions of Tables 1–3 remain valid when the larger retention discrepancies are accounted for. We have incorporated these analyses into Appendix A.3.8 (Tables 15–18) of the revised paper.
We appreciate the reviewer for the thoughtful feedback. We address each concern and suggestion below with additional experiments.
W1/Q5. Component contributions and image-adaptive pruning
We agree that Table 9 should be moved to the main text to clarify the contribution of each component in the full system. We have moved it to the main text as Table 6 (Table wrQy-1 below) and revised Sec. 3.3 to distinguish the two results: visual guidance improves retained accuracy by 3.14 points with Stage 1 disabled (Fig. 7c), while improving task accuracy by 0.91 points in the full system (L309-311). As shown in Table wrQy-1, visual guidance provides smaller gains when Stage 1 is enabled, likely because Stage 1 has already removed visually redundant tokens using the same image statistics. We now clarify this distinction in the revised manuscript (L512–514).
As the reviewer correctly notes, ERASE adapts how the token budget is distributed across pruning stages rather than the overall target budget, except when Stage 1 alone falls below the target. To evaluate the contribution of this adaptive allocation in the full system, we added a fixed Stage 1 variant (†) to Table 6 (Table wrQy-1). Adaptive Stage 1 improves average accuracy from 70.70% to 71.01% (+0.31 points). Together with the Stage 1-only improvement of 0.52 points in Table 4, these results demonstrate a consistent, albeit modest, benefit from image-adaptive retention. We have revised the manuscript to clarify this contribution (L519–524).
Regarding novelty, we agree that Stage 1 builds on established techniques such as edge detection and patch entropy. However, ERASE does not simply apply these operators independently. Instead, it combines complementary visual cues with image-relative thresholds to determine both which tokens and how many tokens to retain before the LLM. This adaptive selection is further integrated with hierarchical instruction-aware pruning, where Stage 1 retention determines the subsequent pruning schedule. Unlike iLLaVA, VScan, and Nüwa, which rely on vision-encoder attention or learned visual representations, ERASE uses lightweight raw-image statistics for early pruning. This avoids the additional latency observed with iLLaVA (2671 vs. 2941 ms, Table 5) and the encoder-attention memory overhead encountered by VScan and Nüwa on Qwen2.5-VL, which lacks an encoder [CLS] token (Table 9 of the revised manuscript). While PruneSID also adapts compression to image complexity, ERASE achieves higher Stage 1 accuracy at matched retention ratios (Table 4) with lower selection overhead (Fig. 8c and Table 5).
Table wrQy-1. End-to-end component ablation on Qwen2.5-VL-7B