- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes BELT-VLA, an online RL framework for adapting pretrained VLA models. It uses human interventions to identify failure-prone states and applies orthogonal transformations to edit latent action features. The method combines RND novelty and failure risk for optimizaiton, showing some improvments on four real-world manipulation tasks.
Critical strengths
I think the motivation is reasonable, and localized policy editing is an intresting direction. The paper also provides real-world robot experiemnts, and the reported results are promising.
Critical weaknesses
-
Limited novelty and questionable methodology. The method mainly combines existing techniques, including PPO, RND, Failure Critic and orthogonal transformations. I don't see much methodological novelty here. More importantly, why can optimizing novelty and intervention risk improve task success without directly using task rewards? This is quite confusing and needs more explaination and evidance.
-
The experiments are not very convincing. Although the success rate improves from 33.5% to 82%, the improvement over HIL-SERL is only 4.5 points. Considering the additional complexity, I don't think this improvment is very impressive. Also, only four tasks are evaluated, without sufficient statistical analaysis or more comprehensive comparisions.
-
The preservation claim is not well supported. The authors mainly use latent and action deviations to measure policy preservation. However, smaller deviations do not neccessarily mean the original VLA capabilities are preserved. I think the authors should directly evaluate the adapted policy on other tasks. The reliability and generalizaiton of the Failure Critic also need further evaluation.
Questions for authors
No more further question. Please see the weakness part.
We thank for your careful evaluation and constructive feedback. We respond to the main concerns below.
Q1: Limited Novelty and Questionable Methodology
Our key contribution is an intervention-guided editing framework that couples where to edit (risk-conditioned regularization) with how to edit (orthogonal latent transformations). The same continuous risk estimate ρt guides both reward shaping and edit regularization:
rt=αrtcur−βρt,
=\mathbb E\left[(1-\rho_t)\|v_t\|_2^2\right].$$ Here, $r_t^{cur}$ is the RND novelty reward, $\rho_t\in[0,1]$ is the intervention risk predicted by the Failure Critic, and $v_t$ is the latent editing vector. The risk penalty discourages intervention-prone states, while edit regularization suppresses unnecessary changes in low-risk states. Unlike LAE [1] and ZPRL [2], BELT-VLA jointly uses intervention-derived risk to guide online optimization and state-conditioned orthogonal editing. Our reward design was motivated by an observation when reproducing real-world RL methods: human interventions led to successful episodes with a terminal reward of ‘1’. Thus, terminal success rewards may not reflect the reliability of autonomous execution. The pretrained VLA provides task priors, while RND encourages exploration and intervention risk discourages unreliable actions. Ablations support both components: removing RND novelty or the risk penalty reduces success rates from 82.0% to 70.5% and 66.0%, respectively. Editing ablations further validate this design: BELT-VLA achieves 82.0% success, compared with 72.0% without boundary regularization and 68.5% with unconstrained editing. We also provide more detailed reward comparisons in the new **Effects of Reward Design** subsection. **Q2: The Experiments Are Not Very Convincing** We acknowledge the modest average improvement over HIL-SERL. However, on the most challenging task (screwdriver assembly), BELT-VLA improves success from **64.0% to 72.0%** (+8.0 percentage points). Moreover, BELT-VLA updates fewer than 1M parameters while keeping the pretrained VLA frozen, offering lightweight adaptation under the same interaction budget. To provide additional statistical evidence, we report 95% confidence intervals. Across 200 test trials, BELT-VLA achieves 82.0% success (95% CI: 76.1–86.7%), compared with 77.5% (95% CI: 71.2–82.7%) for HIL-SERL. The 4.5-point improvement has an approximate 95% CI of −3.4 to 12.3 percentage points. BELT-VLA outperforms HIL-SERL on three tasks and matches it on the fourth. We appreciate the reviewer's suggestion for broader evaluation. Although limited to four tasks, they cover diverse manipulation skills, such as object transport, precise alignment, and small-object grasping, providing an initial evaluation across different manipulation challenges. We are working to add further baselines and simulation benchmarks to assess robustness and generalizability. **Q3: The Preservation Claim Is Not Well Supported** We agree that smaller action deviations do not directly demonstrate cross-task capability preservation. BELT-VLA primarily targets **within-task behavioral preservation**, minimizing changes in reliable states while correcting failures. We keep the pretrained VLA frozen and use risk-conditioned regularization. Experiments show much smaller action deviations in reliable states (0.07) than near failure boundaries (0.29). Removing boundary regularization reduces the success rate from 82.0% to 72.0%. For the Failure Critic, we further evaluate its reliability on held-out trajectories. Using intervention labels with $K=3$, we obtain an average **AUROC of 0.82** and **AP of 0.40**. These results support reliable intervention-risk discrimination on the evaluated tasks. We acknowledge that cross-task generalization remains insufficiently evaluated, particularly under limited intervention data. Improving the robustness of failure-risk estimation is an important direction for future work. **References** [1] S. Das et al., “Latent Activation Editing: Inference-Time Refinement of Learned Policies for Safer Multirobot Navigation,” arXiv:2509.20623, 2025. https://arxiv.org/abs/2509.20623 [2] D. Yu et al., “Beyond Action Residuals: Real-World Robot Policy Steering via Bottleneck Latent Reinforcement Learning,” arXiv:2605.19919, 2026. https://arxiv.org/abs/2605.19919