- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes VLSA-RL, a method for adapting a pretrained VLA policy to safety-critical manipulation tasks without updating the VLA itself. It freezes the VLA backbone and trains a small residual policy that corrects the VLA's actions before they pass through a control barrier function (CBF) safety filter. The residual policy is optimized with a safety-aware variant of GRPO that uses feedback from the CBF filter and adjusts the update size based on trajectory-level safety, and the authors also provide an analysis of how the residual actions relate to the CBF condition. Experiments on SafeLIBERO show improved safe success rates over the CBF-filtered AEGIS baseline and other VLA policies, along with ablations of the main design choices.
Critical strengths
The motivation is clear, and the proposed method is lightweight. It avoids the cost of end-to-end RL fine-tuning by training only a small residual policy on top of a frozen VLA. The paper also establishes a connection between the residual policy and the CBF formulation, showing how residual actions can compensate for dynamics mismatch and effectively modulate the CBF convergence rate.
Critical weaknesses
- The novelty of the method seems somewhat incremental. The framework mainly combines existing components, including residual RL on a frozen policy, the CBF-QP safety filter from VLSA, GRPO, and the use of safety-filter corrections as a learning signal, which has also been explored in prior work cited in the paper. As a result, the contribution lies mainly in how these components are combined, and it would help if the paper more clearly highlighted what is new relative to prior work.
- The proposed method introduces several hyperparameters that may require tuning, including , , , and , among others. Although the ablation study shows the performance when each component is removed, it does not show how performance changes as these values vary. As a result, the sensitivity of the method to these hyperparameters remains unclear.
Questions for authors
- Since the proposed method builds on VLSA, a natural alternative is to adapt the VLA within VLSA using parameter-efficient fine-tuning methods such as LoRA, which also update only a small number of parameters. Could the authors compare the proposed residual policy with such a LoRA-based variant of VLSA, to clarify the advantages of the residual design?
- In Eq. (6), an unsafe success receives a higher reward (0.5) than a safe failure (0.1). This seems to prioritize task completion over safety, and may encourage the policy to accept collisions in order to complete the task. How do the authors prevent this kind of reward hacking? In addition, how are in Eq. (7) and in Eq. (9) defined?
- Several recent safe-VLA methods, including KNOWS [1] and constrained flow matching [2], are also evaluated on SafeLIBERO and appear to be directly relevant baselines. Could the authors include comparisons with these methods, which would provide a clearer assessment of VLSA-RL relative to recent approaches developed for the same safety setting?
- Could the authors clarify how task success is defined? Since collisions do not terminate an episode, a trajectory can still count as a task success even after colliding with an obstacle. In this case, TSR may overestimate practical task performance, while SSR appears to be the more meaningful metric for evaluating safe task completion.
[1] https://arxiv.org/abs/2606.09749
[2] https://arxiv.org/abs/2607.01378
6 comments
Response to Reviewer k86c (Part 2)
Response to Q1: Comparison with LoRA-based Adaptation
We thank the reviewer for this valuable suggestion. Following this suggestion, we have added a new Appendix I, which presents a comparison between VLSA-RL and a LoRA-based VLSA variant on SafeLIBERO Spatial I. We have also added a new Figure 12 to illustrate the full evaluation curves.
Specifically, we apply rank-9 LoRA () to the query, key, value, and output projections of the action expert. The LoRA variant trains 1.41M parameters, comparable to the 1.36M trainable parameters in VLSA-RL. Both methods are trained for 300 updates and employ the same CBF-QP safety filter during execution.
An important distinction lies in how the two methods incorporate safety information. The LoRA variant retains the original VLA input interface and conditions action generation on visual observations and task instructions. Standard LoRA adaptation updates model parameters but does not inherently introduce additional input channels for explicit safety information. In contrast, VLSA-RL keeps the VLA backbone frozen and employs a separate residual policy that additionally receives a 79-D context vector containing robot states, obstacle geometry, CBF information, and previous QP corrections. This allows the residual policy to explicitly exploit safety-related information when adapting the nominal VLA actions, without modifying the original VLA architecture.
For a fair comparison, we select the best checkpoint of each method based on SSR, using TSR to break ties. The results are summarized below:
| Method | Best Step | TSR (%) | SSR (%) | ETS (steps) |
|---|---|---|---|---|
| VLSA-RL | 275 | 93.0 | 74.0 | 143.1 |
| LoRA | 250 | 70.5 | 50.5 | 197.05 |
The newly added Figure 12 presents the full evaluation curves. VLSA-RL exhibits consistent improvement during training, whereas LoRA fails to improve SSR beyond its pre-adaptation performance. These results demonstrate that, with comparable parameter budgets and training iterations, the proposed safety-aware residual adaptation achieves substantially better task and safety performance than the evaluated LoRA-based alternative.
Overall, the advantage of the residual design is that it provides a dedicated pathway for incorporating explicit safety information into policy adaptation, rather than relying solely on modifying the parameters of the original VLA policy.
Response to Q2: Reward design and thresholds
We thank the reviewer for raising this important concern. Our tiered reward design reflects the consideration that safety without task completion is insufficient for meaningful behavior. For example, a policy that never takes action can trivially avoid collisions but is unable to accomplish the task. This issue is particularly pronounced in challenging long-horizon tasks (e.g., LONG), where successful trajectories themselves are scarce during early training, and safe successes are even rarer. Assigning partial credit to unsafe successes (0.5), rather than treating them equally with safe failures (0.1), preserves informative learning signals from task-completing trajectories and encourages the policy to develop task competence. Meanwhile, safe successes receive the highest reward (1.0), progressively encouraging collision-free task completion. Thus, our design aims to establish task competence while promoting increasingly safe behaviors, rather than sacrificing task completion for trivial safety. We also acknowledge that alternative reward designs may offer different trade-offs between task completion and safety, which would be valuable to investigate in future work.
We apologize for the omission of these parameter values in the original manuscript. We now report them in Training Details (Section 4.1). Specifically, we set to 150 steps for the Spatial, Goal, and Object suites, and to 350 steps for the Long suite. We set for all suites and both safety levels.
Response to Reviewer k86c (Part 3)
Response to Q3: KNOWS and constrained flow matching
We thank the reviewer for this valuable suggestion. Following this suggestion, we have added new comparisons with KNOWS and constrained flow matching to Tables 1 and 2, reporting their results on SafeLIBERO Level I and Level II, respectively. We have also cited both works in the revised manuscript. Their CAR results have also been added to the new Tables 9 and 10 in Appendix F. The suite-averaged results are summarized below.
| Method | Level I TSR (%) | Level I SSR (%) | Level I ETS (steps) | Level II TSR (%) | Level II SSR (%) | Level II ETS (steps) |
|---|---|---|---|---|---|---|
| VLSA-RL | 80.0 | 63.1 | 225.17 | 82.8 | 72.8 | 217.73 |
| KNOWS | 67.5 | 59.9 | NR | 71.1 | 53.8 | NR |
| Constrained FM | 84.6 | NR | 275.87 | 78.6 | NR | 324.07 |
KNOWS does not report ETS. Constrained flow matching reports TSR, CAR, and ETS, but does not report SSR. We therefore mark its SSR as NR, since the separately reported task success and collision avoidance rates do not determine the joint rate of safe task completion.
We note that the source code of these methods is not publicly available, making direct reproduction under our experimental settings difficult. Furthermore, constrained flow matching uses simulator-provided obstacle positions, and its episode step limits are not specified. Some of its reported task-level ETS values also exceed our evaluation limits. Therefore, we retain the results reported in the original papers under their respective evaluation protocols and present them as cross-paper references rather than strictly controlled comparisons.
Response to Q4: Task success
We thank the reviewer for pointing out this important clarification. We agree that TSR alone may overestimate practical task performance because collisions do not terminate an episode. We have therefore clarified the task-success definition in Section 4.1. Specifically, an episode counts toward TSR if it satisfies the benchmark's task-success condition within the specified episode step limit, which is 300 steps for the Spatial, Goal, and Object suites and 550 steps for the Long suite. A collision marks the episode as unsafe but does not terminate the rollout. Thus, an episode that completes the task after a collision still counts toward TSR, whereas SSR only counts episodes that complete the task without any detected collision.
Summary of Revisions
- Added a LoRA-based comparison in new Appendix I and the full evaluation curves in new Figure 12.
- Added comparisons with KNOWS and constrained flow matching to Tables 1 and 2, and added their CAR results to the new Tables 9 and 10 in Appendix F.
- Reported the values of and in Training Details (Section 4.1).
- Clarified the task-success definition and episode step limits in Section 4.1.
Closing remarks
We would be happy to address any further questions you may have and hope that our revisions and responses will lead to a more favorable assessment. Thank you again for your time and constructive feedback.
Response to Reviewer k86c (Part 1)
Thank you for your thoughtful review and for suggesting comparisons that help clarify the contribution of the residual design.
Response to W1 - Contribution
We thank the reviewer for this constructive comment. We agree that several individual components of VLSA-RL, including residual reinforcement learning, CBF-QP safety filtering, GRPO, and the use of safety-filter interventions as learning signals, have been studied in prior work. Our contribution is not to introduce these components independently, but to develop a unified safety-aware adaptation framework that integrates them specifically for pretrained VLA policies. We have further clarified the distinctions from related work in the revised manuscript. Specifically, our contributions lie in three aspects:
Safety-aware residual adaptation of frozen VLA policies. Unlike VLSA/AEGIS, which relies on execution-time CBF-QP corrections without adapting the nominal policy, VLSA-RL trains a lightweight residual policy alongside the frozen VLA. The residual policy incorporates CBF-related state information and previous QP corrections, enabling it to learn safety-aware action refinements while retaining the original VLA and CBF-QP formulation. This also differs from existing residual RL approaches such as ResiP and PLD in its explicit integration of CBF-derived safety feedback into VLA adaptation.
CBF-informed policy optimization at both action and trajectory levels. Rather than simply treating safety-filter interventions as generic penalties or reward signals, we introduce a differentiable QP correction loss to guide nominal actions toward requiring less safety intervention. Meanwhile, the minimum CBF margin over an entire trajectory determines the GRPO clipping range, explicitly coupling policy-update magnitude with trajectory-level safety risk. Together with graded task-safety rewards and fixed-scale group-relative advantages, these mechanisms establish a safety-aware optimization objective tailored to residual VLA adaptation.
Theoretical characterization of residual actions in CBF-based control. We derive a sufficient condition under which residual actions compensate for dynamics mismatch in the CBF constraint. We further show that the residual contribution is algebraically equivalent to a state-dependent modulation of the CBF convergence-rate coefficient, providing a theoretical interpretation of how residual adaptation affects safety behavior.
The ablation results further support the contributions of these designs. For example, removing the CBF-QP correction loss reduces TSR and SSR from 93.0% and 74.0% to 87.5% and 69.5%, respectively. Replacing margin-dependent clipping with the best fixed clipping bound also reduces TSR and SSR to 90.0% and 72.5%. These results demonstrate the complementary contributions of action-level CBF feedback and trajectory-level risk-aware optimization.
We acknowledge that the novelty lies in the proposed integration and its resulting learning and theoretical mechanisms, rather than in any individual building block. We have revised the Introduction and Related Work to make these distinctions more explicit.
Response to W2 - Hyperparameter sensitivity
We thank the reviewer for this valuable suggestion. We acknowledge that a systematic hyperparameter sensitivity analysis has not been conducted, as it would require substantial additional training runs that we could not complete within the limited rebuttal period. Nevertheless, the hyperparameter configuration reported in the paper has been validated through our experiments and demonstrates effective task and safety performance. We recognize the importance of further investigating hyperparameter sensitivity and would be happy to conduct additional experiments and incorporate the results in a subsequent revision if there is an opportunity for further revision.