- Rating
- 4(+1)
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies architectural redundancy in VLA models via Drop-Then-Recovery (DTR), a method that removes redundant transformer blocks and fine-tunes the smaller model, paired with GateProbe, a one-shot virtual-gate metric that ranks blocks by their contribution to the downstream action loss. It finds that language backbones are highly redundant while vision and action pathways tolerate removal poorly. The paper concludes that current benchmarks under-test language grounding and calls for more capacity-balanced VLA designs.
Critical strengths
-
Block skipping yields substantial speedups without significant accuracy loss: DTR-30 achieves 2.94× action speedup at 90% success on LIBERO-Goal, and DTR-16 even improves both accuracyand task speedup.
-
The evaluation is comprehensive and convincing: four architectures, three simulation benchmarks, and real-robot experiments, including OOD robustness analysis.
-
The paper is clearly written and easy to follow.
Critical weaknesses
When determining the importance of different LLM layers, representative datasets are used, but the source and proportion of representative datasets are not very clear. For example, if the simulator or real-world environment is changed, do we need to use anthor representative dataset for calibration again? How much data is needed as a representative dataset?
Questions for authors
-
This article mainly argues that the understanding of instructions in embodied tasks is simple, and there is a lot of redundancy in the language model when processing these tasks. However, some datasets such as BEHAVIOR-1K have longer instructions. Do you think it is still possible to use such an aggressive block skipping strategy?
-
Some representative works are not cited, such as DySL-VLA [1] and Deer-VLA [2]. Especially DySL-VLA uses the input-output similarity of each block to determine the importance of each block. How do you compare it with your method?
-
Just for discussion, I noticed the robustness degradation under perturbations, while in VLA, the language model not only needs to understand the instructions, but also needs to handle the output of the vision encoder. Is it possible that aggressive layer skipping has caused a decrease in this ability of VLA, resulting in a decrease in accuracy?
[1] Yang Z, Qi Y, Xie T, et al. DySL-VLA: Efficient vision-language-action model inference via dynamic-static layer-skipping for robot manipulation[J]. arXiv preprint arXiv:2602.22896, 2026.
[2] Yue Y, Wang Y, Kang B, et al. Deer-vla: Dynamic inference of multimodal large language models for efficient robot execution[J]. Advances in Neural Information Processing Systems, 2024, 37: 56619-56643.
6 comments
Response - 2
Q2: Comparison with DySL-VLA and DeeR-VLA
We thank the reviewer for pointing out these two relevant works. They are closely related to our study, and we acknowledge that we overlooked discussing them in the original manuscript.
1. DySL-VLA: Comparison of block importance metrics.
DySL-VLA uses the similarity between a block's input and output representations to estimate its importance. This corresponds to the CosSim (Block Influence) metric already evaluated in our Table 3.
Under the same number of removed blocks and recovery fine-tuning settings:
| Model | Setting | CosSim | GateProbe |
|---|---|---|---|
| π0.5 | Drop-9 | 90.2 | 94.0 |
| π0.5 | Drop-12 | 89.4 | 92.1 |
| π0.5 | Drop-16 | 81.9 | 92.2 |
| OpenVLA-OFT | Drop-16 | 91.8 | 97.1 |
| OpenVLA-OFT | Drop-30 | 93.5 | 95.9 |
GateProbe achieves higher post-recovery success rates in all five settings, particularly under aggressive compression.
We emphasize that this comparison concerns the similarity-based layer selection criterion used by DySL-VLA, rather than a full reproduction of DySL-VLA. The latter also incorporates dynamic layer skipping, prior-post guidance, and knowledge distillation.
2. DeeR-VLA: Dynamic inference versus structural compression.
DeeR-VLA dynamically determines the inference depth based on the input to avoid unnecessary computation. DySL-VLA similarly improves efficiency by dynamically adjusting computation during inference. In contrast, DTR physically removes transformer blocks before recovery fine-tuning, producing a structurally smaller model and evaluating whether its performance can be recovered.
Therefore, DySL-VLA and DeeR-VLA primarily investigate inference-time computational redundancy, whereas DTR focuses on structural redundancy. These approaches are complementary.
We will discuss both works in Related Work and clarify the connection between DySL-VLA's similarity criterion and our CosSim baseline, together with the experimental comparisons, in the appendix. We appreciate the reviewer for helping us improve the discussion of related work.
Q3: Could robustness degradation be related to the language backbone's role in processing visual features?
We agree that the language backbone in VLAs not only processes instructions but also participates in visual feature processing and cross-modal integration. Therefore, removing language layers may affect more than language understanding.
Our LIBERO-Plus results are consistent with this explanation. For π0.5 Drop-16, the performance decreases relative to the full model under different perturbations are:
| Perturbation | Performance Change |
|---|---|
| Robot | −24.2 points |
| Noise | −15.0 points |
| Camera | −12.7 points |
| Language | −9.9 points |
| Light | −4.7 points |
Robot perturbations change the robot's initial configuration, while Noise and Camera perturbations primarily affect visual observations. Thus, the performance degradation is not limited to language perturbations.
Our real-robot experiments show a similar pattern. Under the green-light perturbation, the full model achieves 50% success, compared with 35% for Drop-16. When the container is removed, the corresponding success rates are 60% and 40%, respectively. The language instructions remain unchanged in both cases.
These results are consistent with the reviewer's explanation that the removed language layers may also contribute to visual integration and OOD robustness. However, our current experiments do not isolate the precise mechanisms responsible for the degradation.
Dear Reviewer UhLq,
Thank you very much for your encouraging feedback and valuable suggestions regarding our work.
As we approach the end of the discussion period, we wanted to briefly follow up and ask whether our responses have sufficiently clarified the questions you raised.
If there are any outstanding concerns or additional points you would like us to discuss, we would be more than happy to address them.
We truly appreciate your time and effort in reviewing our paper.
Best regards,
The Authors of Submission 3168
Response - 1
We thank the reviewer for the thoughtful questions regarding calibration data selection and cross-task applicability. We are happy to clarify our experimental setup and provide additional analyses.
1. Calibration data selection and computational overhead.
Under the DTR's setting, target-task training data are already available for recovery fine-tuning. Therefore, GateProbe can directly sample a small calibration subset from the existing training data, without requiring additional data collection or annotation.
For LIBERO, we combine the training data from all four suites and uniformly sample 64 batches × 8 = 512 samples (We use 512 samples for all benchmarks), with approximately the same suite proportions as the combined training set. Each batch requires only one forward and one backward pass, without parameter updates. For π0.5, calibration takes 24.9 seconds on a single H200 GPU (Table 8), representing a small overhead compared with recovery fine-tuning. We have not determined the minimum amount needed, but since the cost grows linearly with the number of batches, a larger calibration set remains inexpensive.
2. Calibration results show substantial consistency across tasks.
To investigate whether recalibration is necessary when the environment changes, we compare GateProbe rankings for π0.5 across nine calibration datasets, including LIBERO, LIBERO-Plus, and seven RoboTwin tasks. RoboTwin differs considerably from LIBERO in its environments, robot configurations, and manipulation tasks.
Across these datasets, Spearman correlations with LIBERO range from 0.76 to 0.92, and the Drop-9 configurations share 7–8 of the 9 retained blocks with the LIBERO selection. These results indicate that GateProbe importance rankings are largely consistent across different tasks.
Overall, GateProbe has low calibration overhead and exhibits substantial consistency in block selection across the evaluated environments. For new target tasks, especially under aggressive compression, we still recommend recalibration. We will include these experimental details and cross-task analyses in the revised paper.
We thank the reviewer for this valuable question. We agree that evaluating tasks with more diverse linguistic expressions is important for understanding language-layer redundancy.
We believe that structural redundancy may still exist in more language-demanding tasks, although the extent of recoverable compression may decrease as language requirements increase. To further investigate this, we analyze how different degrees of instruction reformulation affect recovery performance on LIBERO-Plus.
In this analysis, only the instruction wording is changed, while the scene, initial state, target task, and evaluation seed remain fixed. Within each suite, we divide instructions into three groups based on their degree of reformulation, characterized by instruction length ratio (L), lexical overlap with the original instruction (J), and the proportion of subordinate clauses or indirect expressions (SUB).
π0.5 on LIBERO-Plus language perturbations:
As the degree of instruction reformulation increases, the performance difference between Drop-9 and the full model changes from +3 to −2 and −9 percentage points. Drop-16 performs below the full model across all three groups (−11, −13, and −8 points), although the degradation is not monotonic.
These results suggest that the high recoverability of language layers observed on standard LIBERO does not necessarily translate into equally strong performance under greater linguistic variation.
For benchmarks such as BEHAVIOR-1K, we believe the achievable compression level should be evaluated on the target tasks, and DTR provides a framework for conducting such an analysis. We appreciate the reviewer for highlighting this direction, which also helps clarify the scope of our findings.