- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents VLA-ZO, a framework for accelerating forward-only, zeroth-order adaptation of VLA models under visual distribution shifts. Adaptation is restricted to an action-side LoRA module, allowing the frozen vision-language conditioning states to be reused across perturbation evaluations and optimization steps. A disk-backed cache and one-step-ahead prefetching further reduce recomputation and transfer overhead.
The evaluation uses π0.5 and OpenVLA-OFT on 40 LIBERO tasks, with five target-condition demonstrations, one per scene, covering viewpoint changes and compound visual shifts. For π0.5, the full system achieves 25.59× and 32.54× speedups over uncached action-side ZO at query budgets of 16 and 64, respectively, while maintaining similar adaptation quality. Average viewpoint-shift success improves from 48.27% without adaptation to 58.17% and 63.58%.
Critical strengths
-
A clear and practically important systems insight. Reusing the frozen prefix states within and across optimization steps is a simple, well-motivated intervention that preserves the intended optimization objective under the stated assumptions.
-
Substantial measured acceleration across two VLA architectures. Evaluation across π0.5 and OpenVLA-OFT strengthens the case that the reuse strategy is not specific to a single action-module implementation. The improvements in adaptation success over the unadapted policies further demonstrate the practical relevance of making ZO adaptation faster.
Critical weaknesses
-
Insufficient baseline comparisons. The proposed caching strategy could also be applied to first-order training of the action module. The experiments with FLA† and DART† show that the proposed method outperforms the evaluated zeroth-order implementations, but it remains unclear under what conditions it outperforms similarly execution-optimized first-order approaches.
-
Insufficient evidence for practical deployment. The experiments provide convincing evidence of acceleration under specific simulation settings and on high-performance hardware. However, direct evidence of the method’s applicability to real-world robotic systems and its practical deployment benefits remains limited.
-
Insufficient efficiency evaluation under a fixed time budget and limited experimental robustness. The current experiments fix the number of updates at 1,000 steps and adjust the learning rate according to the query budget , leaving the benefits of increasing perturbation directions versus optimization steps under the same time budget unclear. The authors should report success rate versus adaptation time curves.
Questions for authors
-
On a Matched First-Order Baseline: What trainable parameters and optimizer are used for the first-order results in Figure 1? Can the authors compare the success rate, adaptation time, and peak memory usage of first-order training and the proposed method under the same action-side LoRA configuration and prefix caching strategy?
-
On the FLA and DART Baselines: Why were FLA and DART reimplemented using zero-order gradient estimation and SGD? How does this modification affect comparability with the original methods? Can the authors disentangle the effects of system optimizations from those of optimization configurations?
-
On Performance Under a Fixed Time Budget: The current experiments fix the number of updates at 1,000 and adjust the learning rate according to the query budget (). Can the authors jointly vary the query budget and number of updates, and report success rate versus end-to-end adaptation time curves to verify whether increasing the number of perturbation directions is more effective than increasing the number of optimization steps under the same time budget?