- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes a training-free visual token pruning method for VLA models. The motivation is that, as the pruning ratio increases, attention-based global Top-k pruning tends to concentrate the retained tokens on a few dominant attention targets. The result is spatial collapse, so a small number of tokens must also be kept for the other task-relevant attention regions. ProtoP scores each visual token by the mean attention it receives from text and action query tokens at an early LLM layer. It then finds local attention maxima and uses watershed segmentation to partition the remaining foreground into regions. The token budget is allocated across regions in proportion to each region's peak score, and every region keeps at least one token. At 90% pruning on LIBERO, ProtoP reaches 93.3% success with OpenVLA-OFT and 86.0% with π0.5, outperforming VLA-Pruner and other methods. At 60–75% pruning, ProtoP performs on par with the baselines.
Critical strengths
Large gains at extreme pruning. At 90% pruning on LIBERO, ProtoP improves over the latest baseline (VLA-Pruner) by about 11.1 points on OpenVLA-OFT and about 13.2 points on π0.5. The high-compression regime is where pruning is most valuable for deployment, and it is also where existing methods break down.
Critical weaknesses
Effect of the scoring layer. FastV, SparseVLM and VLA-Pruner score tokens at layer 2, while ProtoP scores at layer 3. The ablation study shows how much this matters: scored at layer 2, ProtoP reaches 81.7%, below VLA-Pruner's 82.2%; at layer 1 it reaches 84.0%, and at layer 3 it jumps to 92.8%. Yet the paper's central claim is that coverage (the one-token-per-region floor), not the scoring, drives the improvement.
Limited novelty. The paper's contribution is a diagnosis plus a relatively simple fix, and the method is somewhat sensitive to hyperparameter choices.
Questions for authors
Layer indexing and compute. Is "layer 3" 0-indexed? Is FastV/VLA-Pruner's "K = 2" defined the same way? Please compare at matched TFLOPs or latency. More diversity-aware baselines (DivPrune, CDPruner, DART)
3 comments
Rebuttal by Authors #2
[W2] "Limited novelty. The paper's contribution is a diagnosis plus a relatively simple fix, and the method is somewhat sensitive to hyperparameter choices."
As mentioned, there are two contributions: the diagnosis reveals the issue, leading to our simple, but effective method on every suite. The method is also insensitive to most hyperparameters.
Diagnosis. We identify spatial collapse as the main failure mode of attention-based pruning at high compression. Every evaluated attention-based pruner, including the VLA-specific VLA-Pruner, concentrates its tokens on one or two hotspots and loses the rest of the scene. The revision confirms this diagnosis with ground-truth object masks that are independent of our method: removing the tokens on task objects lowers success by 33 points, whereas removing as many random tokens lowers it by 2 (Appendix G.6–G.7).
Simple, but effective. Not every simple fix works. At a matched score, layer, and token count (main-paper Table 5), each of the other simple rules fails on some suite:
- The fixed grid fails on Long (83.5 vs 91.5%, p = 0.007).
- Per-camera quotas and peaks + Top-k fail on Object (79.5 / 81.5 vs 93.0%, p < 0.001).
- Feature diversity without attention (DivPrune) collapses to 35.6% (Appendix G.10).
ProtoP is the only rule that holds on every suite. It needs no training and improves over VLA-Pruner by 11–13 points on two backbones.
Sensitivity. Most choices have little effect:
- The foreground threshold is flat: 93.0–93.4% over 0.25–1.0 (Appendix Table S4).
- Peak detectors are interchangeable (Appendix Table S6).
- d_min ∈ {1, 2} stays within 2 points (main-paper Table 4(b)).
The most sensitive choice is the scoring layer, which affects all attention-based methods alike.
[Q] "Is "layer 3" 0-indexed? Is FastV/VLA-Pruner's "K = 2" defined the same way?" Yes, all indices are 0-indexed over decoder layers:
- ProtoP scores at index 3.
- FastV with K = 2 scores at index 1.
- VLA-Pruner and single-step SparseVLM score at index 2.
Sec. 5.1 and Appendix A.3 now state this.
[Q] "Please compare at matched TFLOPs or latency."
At matched FLOPs, ProtoP is the most accurate method we evaluated. In latency it is not faster, so the paper now claims accuracy at a given budget rather than efficiency.
We added operating points with similar analytic LLM FLOPs: progressive SparseVLM, ProtoP's selection inside the same progressive schedule, and VLA-Pruner with a larger budget (Table R5).
Table R5. Accuracy at similar compute (Appendix Table S12). Δ is relative to ProtoP.
| Method | TFLOPs | Success (%) | Δ |
|---|---|---|---|
| ProtoP | 2.58 | 94.8 | (ref.) |
| Progressive SparseVLM | 2.65 | 89.9 (Object 75.5) | −4.9∗∗∗ |
| Progressive ProtoP | 2.65 | 92.3 (Object 86.0) | −2.5∗ |
| VLA-Pruner (85% pruning)† | 2.68 | 92.8 | −1.7 |
| Progressive SparseVLM | 2.90 | 95.0 | +0.2 |
† 100 episodes per suite, compared with ProtoP on the same episodes (94.5%). The other rows use 200 paired episodes per suite.
FLOPs. At matched FLOPs, ProtoP is significantly more accurate than progressive SparseVLM, which needs 12% more FLOPs to match ProtoP's accuracy. No evaluated operating point is both cheaper and more accurate than ProtoP.
Latency. ProtoP's benefit is accuracy at a given budget, not wall-clock time. VLA-Pruner at 85% pruning runs at 95 ms vs 109 ms for ProtoP, because our region construction runs on the CPU. The paper says this explicitly.
[Q] "More diversity-aware baselines (DivPrune, CDPruner, DART)"
We evaluated DivPrune, which collapses at 90% pruning. CDPruner and DART are now cited.
- DivPrune (feature diversity without attention) reaches 35.6% on average (Appendix G.10).
- Fixed grid (spatial diversity without attention) fails on Long.
Diversity helps only when it is aligned with task-conditioned attention, which is what ProtoP's regions provide. CDPruner and DART were not evaluated, and the paper says so.
Dear Reviewer gw1t,
Thank you again for your review. As the author-reviewer discussion period is coming to a close, we wanted to check whether our response addressed your concerns.
To separate the scoring layer from coverage, we evaluated every baseline at layer 3 as well as at its default layer. ProtoP still leads by 14 to 27 points, and at layer 2 the gap to VLA-Pruner is not significant, so we now present the layer and coverage as two separate factors rather than one. We also clarified that all layer indices are 0-indexed, added matched-FLOPs operating points, and evaluated DivPrune, which reaches 35.6%.
If any part of your concern remains open, we would be glad to answer it while there is still time.
Best regards, The Authors
Rebuttal by Authors #1
Thank you for the review. We appreciate your recognition of the large gains at extreme pruning (+11.1 and +13.2 points over VLA-Pruner), the regime where pruning matters most for deployment and where existing methods break down. Your point about the scoring-layer confound made our central claim more precise. Changes in the revised PDF are in blue.
Notation: Table R# denotes a table in this rebuttal (numbered across all responses). "Main-paper Table #" and "Appendix Table S#" refer to the revised PDF. New experiments use OpenVLA-OFT on LIBERO at 90% pruning, with 200 episodes per suite. In tables, Δ is the difference in success rate (percentage points) from the reference row; ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001 (exact McNemar test on paired episodes); unmarked differences are not significant.
With all baselines moved to layer 3, ProtoP still leads by 14–27 points, so the gain does not come from the layer alone. The layer and coverage are both needed, and only coverage lets a method benefit from the deeper layer.
To separate the scoring layer from the coverage floor, we evaluated every baseline at both its default layer and ProtoP's layer 3 (with exactly 52 tokens), and we also evaluated ProtoP at layer 2 (Table R4).
Table R4. Effect of the scoring layer (Appendix Table S11). Each row shows the average at the default layer and at layer 3; Δ compares the layer-3 average with ProtoP.
† For ProtoP, the left value is at layer 2 (100 episodes per suite).