- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper applies a dense-to-MoE adaptation approach to vision-language-action (VLA) models. The experimental results demonstrate that this approach can reduce FLOPs and memory usage during inference while largely preserving task performance.
Critical strengths
The paper introduces dense-to-MoE adaptation to VLA models, demonstrating that the number of active parameters can be reduced with only a minor degradation in task performance.
Critical weaknesses
-
Limited novelty. The techniques employed in this paper have already been widely explored in the LLM literature. The paper primarily applies existing methods to VLA models without introducing substantial methodological innovations.
-
Lack of robotics-specific contributions. Neither the proposed method nor the experimental analysis provides significant insights specific to robotics. It remains unclear what new understanding or practical benefits this work offers beyond applying existing LLM compression techniques to VLA models.
-
Increased inference latency. Although the proposed method reduces FLOPs and memory usage, it actually increases inference latency.
Questions for authors
No.
Response to Reviewer xhCU
We thank Reviewer xhCU for the careful review. The additional results and corresponding changes are marked in blue in the revised PDF.
1. Long-horizon evaluation and variance
We evaluated the dense π0.5 baseline and AdaDE using three independent seeds under the same protocol.
The dense baseline obtains 91.8±0.7 on LIBERO-Long and 95.5±0.1 averaged over the four LIBERO suites. AdaDE obtains 91.7±0.5, 90.9±0.6, and 90.3±0.8 on LIBERO-Long at 0%, 40%, and 50% deactivation, with four-suite averages of 95.5±0.3, 94.6±0.1, and 94.1±0.2, respectively.
On all 50 RobotWin2.0 tasks, the corresponding averages are 46.59±0.69, 42.37±0.44, and 38.51±0.48. Thus, degradation is larger on RobotWin2.0 than on the near-saturated LIBERO suites. We now describe the result as competitive performance rather than uniform preservation.
The π0.5 baseline uses the public pretrained checkpoint and official codebase, evaluated under our fixed protocol.
2. Deactivation ratio and adaptive allocation
The 40% and 50% settings denote approximate deactivation ratios within selected LLM-side FFN channel groups. Table 1(b) reports the resulting counts: LLM-side parameters decrease from 2.043B to 1.266B / 1.044B, and total parameters from 3.618B to 2.841B / 2.619B.
Mmin=5 is a per-layer lower bound, not a uniform mask. Layers can remove different experts at different times according to router statistics, Wanda protection, and the update schedule.
We also evaluated a no-router baseline with the same ten channel slices and 40k-step budget. At 40%, Static Slice Pruning + Wanda obtains 92.5±1.7 on Spatial and 82.7±1.6 on Long, versus 97.5±0.2 and 90.9±0.6 for AdaDE. At 50%, it obtains 90.9±0.4 and 81.1±2.0, versus 96.9±1.0 and 90.3±0.8. This supports the benefit of adaptive routing beyond fixed channel partitioning.
3. Compact implementation and latency
The original multi-expert implementation introduces routing, aggregation, and small-matrix overhead. We therefore pack retained slices before deployment into a narrower gated FFN while preserving token-dependent router weighting.
Under a unified batch-size-one LIBERO protocol, the dense baseline takes 293.20 ms. The packed implementations at 20%, 30%, 40%, and 50% deactivation take 273.92, 262.39, 251.45, and 244.27 ms, respectively. We do not claim that the same speedup holds across all hardware or kernel implementations.
4. Other baselines
We agree that 8-bit quantization has a stronger memory-performance trade-off in the reported OpenVLA setting; we do not claim universal dominance.
LLM-Pruner and FLAP use additional SFT, so these are not equal-training-budget comparisons and are not used to claim optimization-budget superiority.
Efficient VLA targets a different efficiency axis; our claim is adaptive structured reduction of the deployed LLM-side FFN footprint while retaining competitive manipulation performance.
We thank the reviewer again for helping us clarify these points.