- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a method named PropPrune, which is a training-free visual token pruning method for VLA models. The proposed method uses proprioceptive information to score the importance of visual tokens. It also uses Jaccard similarity between adjacent layers to select the pruning layer for each input. Experiments are conducted on LIBERO, LIBERO-Plus, and two real-world tasks to evaluate the performance of PropPrune compared to baselines. The results show higher success rates than several pruning baselines while reducing inference costs.
Critical strengths
-
The proposed method is simple and does not require additional training. It achieves a 96.3% average success rate on LIBERO at 25% token retention, compared to 93.7% for FastV.
-
The experiments include distribution shifts and real-world manipulation tasks. These results help show the performance of the method beyond the standard LIBERO setting.
Critical weaknesses
-
The contribution of adaptive layer selection is not fully clear. Table 3 shows that always pruning at the later layer achieves a 97.1% average success rate, compared to 96.9% for PropPrune. However, this also increases FLOPs from 2.483T to 2.582T. The paper does not compare with randomly delaying pruning or delaying it at a fixed frequency under the same computation budget. Therefore, it is not clear if the gains come from using Jaccard similarity to select the layer. The improvement may mainly come from keeping all visual tokens for more layers on some inputs.
-
Appendix B shows that the average success rate only improves by 0.50 and 0.25 percentage points over language-only scoring at 50% and 25% retention, respectively. The improvement mainly comes from LIBERO-Long. The other suites show little improvement or worse performance. Additionally, there is no direct evidence to show if the selected visual regions change in a meaningful way when the robot state changes.
-
The novelty compared to existing action-aware VLA pruning methods is not fully clear. Action-aware Dynamic Pruning (ICLR 2026) already uses robot motion information to decide how many visual tokens to retain. SpecPrune-VLA (ICML 2026) also uses action-related signals for dynamic pruning across layers. PropPrune uses different attention weighting and layer-selection rules, but the paper does not compare with these methods. All main experiments use OpenVLA-OFT, whose Transformer includes visual, language, and proprioceptive tokens. Therefore, it is not clear if the results will hold with other VLA architectures or fusion mechanisms.
-
The real-world evaluation only includes two tabletop tasks with 20 trials for each task. At 50% retention, PropPrune and the full model both succeed in 38 of 40 trials. However, this number of trials is too small to show that the two models have the same success rate. The success rates on LIBERO are also close to 100%. It would be helpful to provide uncertainty estimates to show if the small performance differences are reliable.
Questions for authors
-
Can the authors compare Jaccard-based deferral with random deferral under the same late-pruning frequency and inference cost? This would help show the contribution of using Jaccard similarity to select the pruning layer.
-
Can the authors show if proprioceptive guidance changes token selection in a meaningful way when only the robot state changes? It would be helpful to keep the visual observation and instruction the same to show the effect of robot state.
-
Can the authors provide results on another VLA architecture to show if the performance gains will hold? This could include a model with a different proprioceptive representation or action decoder.
-
Can the authors provide confidence intervals or compare the failure cases of the methods on the same trials? This would help make the success-rate differences more clear, especially on LIBERO-Long and the real-world tasks.
6 comments
Response to Weakness 3 & Question 3 (2/3)
Using the calibration rule described in our previous comment, we re-evaluated PROPPRUNE on OpenVLA-OFT and extended the evaluation to RynnVLA-002. The same procedure selected P4/L5 with Jaccard thresholds of 0.523 and 0.445 for OpenVLA-OFT at 50% and 25% retention, respectively, and P7/L8 with Jaccard thresholds of 0.605 for RynnVLA-002 at 50% retention.
All experiments were conducted using the corresponding model checkpoints and the same evaluation protocol, with 500 episodes per LIBERO suite.
The updated OpenVLA-OFT results are as follows.
| Method | Retention | Spatial | Object | Goal | Long | Avg. SR | FLOPs (T) | LLM latency (ms) |
|---|---|---|---|---|---|---|---|---|
| Full-token | 100% | 99.2 | 98.2 | 97.2 | 95.6 | 97.6 | 4.013 | 77.5 |
| PROPPRUNE (calibrated) | 50% | 99.0 | 97.6 | 97.4 | 94.2 | 97.1 | 2.545 | 62.1 |
| PROPPRUNE (calibrated) | 25% | 98.6 | 96.0 | 96.8 | 93.2 | 96.2 | 1.842 | 48.6 |
With the calibrated configurations, PROPPRUNE achieved average success rates of 97.1% and 96.2% at 50% and 25% retention, respectively, compared with 97.6% for the full-token baseline. The corresponding LLM FLOPs decreased from 4.013T to 2.545T and 1.842T, respectively. Although these results differ slightly from the original submission due to the updated pruning configurations, the overall accuracy-efficiency trends remain consistent.
We also re-evaluated LIBERO-Plus using the calibrated pruning configuration. Independently of the configuration changes, the original LIBERO-Plus evaluation environment was lost, requiring us to reconstruct the evaluation setup. We therefore re-ran all methods, including the full-token baseline and FastV, under the same reconstructed environment to ensure a consistent comparison.
| Method | Camera | Robot | Language | Light | Background | Noise | Layout | Overall |
|---|---|---|---|---|---|---|---|---|
| Full-token | 95.9 | 28.1 | 86.5 | 96.0 | 97.2 | 93.3 | 76.3 | 80.7 |
| PROPPRUNE (50%) | 94.0 | 26.7 | 85.0 | 96.8 | 96.3 | 94.1 | 75.9 | 80.0 |
| FastV (50%) | 92.0 | 25.5 | 82.8 | 94.0 | 94.1 | 90.4 | 75.1 | 78.0 |
| ADP (50%) | 90.5 | 24.8 | 80.1 | 92.9 | 92.3 | 87.1 | 72.2 | 75.9 |
Under the reconstructed evaluation setup, PROPPRUNE achieved an overall success rate of 80.0%, close to the full-token baseline (80.7%) and higher than FastV (78.0%) and ADP (75.9%). Despite the updated evaluation environment and pruning configuration, the overall performance trend remains consistent with the original submission.
We then evaluated PROPPRUNE and other pruning baselines on RynnVLA-002 at 50% retention.
| Method | Spatial | Object | Goal | Long | Avg. SR | FLOPs (T) | LLM latency (ms) |
|---|---|---|---|---|---|---|---|
| Full-token | 98.6 | 98.8 | 97.6 | 92.4 | 96.9 | 7.498 | 108.0 |
| FastV | 53.8 | 0.0 | 54.0 | 22.8 | 32.7 | 4.303 | 84.2 |
| EfficientVLA | 61.6 | 43.6 | 42.4 | 58.4 | 51.5 | 4.299 | 89.0 |
| DivPrune | 12.0 | 0.0 | 0.2 | 1.2 | 3.4 | 4.098 | 86.9 |
| ADP | 77.0 | 0.8 | 44.8 | 46.2 | 42.2 | 5.788 | 94.9 |
| PROPPRUNE | 94.6 | 84.8 | 89.0 | 83.0 | 87.9 | 4.863 | 91.5 |
PROPPRUNE achieved an average success rate of 87.9%, outperforming all evaluated pruning baselines, whose success rates ranged from 3.4% to 51.5%. Although PROPPRUNE showed a 9%p decrease relative to the full-token baseline, it preserved substantially higher task performance than the other pruning methods.
These results suggest that the core mechanisms of PROPPRUNE can be applied beyond OpenVLA-OFT.
To further examine the effectiveness of the calibration rule, we compared the original OpenVLA-OFT pruning configuration with the automatically calibrated configuration on RynnVLA-002.
| Configuration | Avg. SR (%) | FLOPs |
|---|---|---|
| OpenVLA-OFT Configuration (P4/L5, θ=0.35) | 76.6 | 4.497 |
| Calibrated RynnVLA-002 (P7/L8, θ=0.605) | 87.9 | 4.863 |
Directly transferring the original OpenVLA-OFT configuration to RynnVLA-002 resulted in 76.6% average success. In contrast, the calibrated configuration achieved 87.9%, an improvement of 11.3 percentage points. These results indicate that pruning configurations may not transfer directly across VLA architectures and that the proposed calibration rule can select a competitive configuration without evaluation-success-based tuning.
Response to Weakness 3 & Question 3 (3/3)
We also conducted additional comparisons with existing action-aware VLA pruning methods to clarify the methodological differences and empirical advantages of PROPPRUNE. ADP (ICLR 2026) combines text-driven visual-token selection with motion-based gating using end-effector trajectories. SpecPrune-VLA (ICML 2026) combines historical attention reuse, layer-wise token importance updates, and velocity-based control of pruning aggressiveness.
In contrast, PROPPRUNE directly incorporates proprioception-related attention into visual-token importance scoring. Specifically, we combine language-to-proprioception attention with language-to-visual attention to determine which visual tokens to retain. We then use inter-layer Jaccard similarity to select the pruning depth.
Thus, our main distinction lies in proprioception-guided visual-token ranking and Jaccard-based pruning-depth selection, rather than motion-based control of pruning aggressiveness or layer-wise token importance updates.
We evaluated these methods using their publicly released implementations. ADP produced results comparable to those reported in its original paper. We therefore compared ADP with PROPPRUNE under the same OpenVLA-OFT evaluation protocol, using 500 episodes per LIBERO suite.
| Method | Retention | Spatial | Object | Goal | Long | Avg. SR | TFLOPs |
|---|---|---|---|---|---|---|---|
| ADP | 50% | 98.8 | 97.4 | 96.4 | 91.6 | 96.1 | 3.236 |
| PROPPRUNE | 50% | 99.0 | 97.6 | 97.4 | 94.2 | 97.1 | 2.545 |
| ADP | 25% | 98.0 | 97.2 | 94.6 | 82.2 | 93.0 | 2.844 |
| PROPPRUNE | 25% | 98.6 | 96.0 | 96.8 | 93.2 | 96.2 | 1.842 |
At 50% retention, PROPPRUNE achieved a 1.0%p higher average success rate than ADP. At 25% retention, the improvement increased to 3.2%p. In particular, PROPPRUNE achieved 93.2% on LIBERO-Long at 25% retention, compared with 82.2% for ADP.
These results show that PROPPRUNE outperforms ADP at both retention budgets.
We also attempted to reproduce SpecPrune-VLA using its official public implementation with . The reproduced success rates on LIBERO-Spatial, Object, Goal, and Long were 92.0%, 96.8%, 94.4%, and 41.4%, respectively, with an average success rate of 81.2%.
However, these results did not reproduce the performance reported in the original paper, particularly on LIBERO-Long. A related reproducibility issue was also reported in Issue #1 of the official GitHub repository.
Given the limited rebuttal period, we could not fully resolve this discrepancy with the original authors. Therefore, we focus our main quantitative comparison on ADP, for which the reported results were reproducible. We will also cite both ADP and SpecPrune-VLA in Section 2.3 of the revised manuscript and clarify their methodological differences from PROPPRUNE.
Response to Weakness 3 & Question 3 (1/3)
We thank the reviewer for the valuable suggestions on comparing our method with existing action-aware pruning methods and evaluating its performance on other VLA architectures.
In the original paper, we evaluated fixed configurations for the probe layer, deferred pruning layer, and Jaccard similarity threshold only on OpenVLA-OFT. We acknowledge that these experiments provide limited evidence for generalization across VLA architectures and raises concerns about the use of heuristic configurations, as also noted by reviewers WxwJ and 2Dvs.
To address these concerns, we conducted additional experiments on RynnVLA-002 and introduced a calibration rule to select pruning configurations for each model using training data, rather than searching for hyperparameters based on evaluation success rates.
The calibration procedure consists of the following steps:
This calibration rule reduces the need for manually selecting model-specific pruning configurations without changing the core components of our method, including proprioception-guided importance scoring and adaptive pruning. We also re-evaluated OpenVLA-OFT using the calibrated configuration. Although the success rates and computational costs differ slightly from the original results, the overall trends in task success rate and computational efficiency remain consistent. We will incorporate these additional experiments and the calibration procedure into the revised paper as soon as possible.
The updated results for both OpenVLA-OFT and RynnVLA-002 are presented in the following comment, together with comparisons against action-aware pruning methods.