- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes Dualized-SafeVLA, a safety-alignment framework for vision-language-action models that addresses changes in state visitation as robot policies evolve. Its main contribution is a Bellman-aware dual formulation and a practical local update procedure that freezes the current occupancy distribution, solves a low-dimensional safety dual, and distills the resulting closed-form target policy under a trust-region constraint. The paper provides theoretical derivations and a bound connecting local constraint satisfaction to actual safety, alongside evaluations on Safety-CHORES, component ablations, and analyses of difficult tasks and solver convergence. It reports a 9.04-percentage-point success-rate improvement and a 79.9% reduction in cumulative safety cost over SafeVLA at approximately matched training steps, although differences in training settings and checkpoint selection limit causal attribution of these gains.
Critical strengths
- The paper addresses a concrete and consequential problem: improving task performance while controlling explicit object and robot safety costs during VLA post-training. Accounting for changes in the states visited by an evolving policy is relevant to maintaining safety in closed-loop robotic execution.
- The method is accompanied by theoretical analysis connecting the constrained formulation to a Bellman-aware dual, an explicit convex local dual, and a closed-form target policy. The trust-region analysis also states conditions under which local constraint satisfaction implies satisfaction of the original safety constraints.
Critical weaknesses
Summary of major concerns. I find it difficult to identify the precise contribution of the proposed method, primarily because of its presentation. The problem being addressed is not explained clearly enough, the figures are difficult to interpret, and the notation and newly introduced concepts lack sufficient definition and context. These issues obscure the distinction between the proposed approach and existing methods. I strongly recommend a thorough rewrite that clearly states the unresolved problem, explains the key technical ideas, and establishes what the method contributes beyond prior work.
- The distinction from prior constrained VLA training needs sharper justification. The introduction motivates the method by arguing that a globally fixed dual variable is generally insufficient under policy-induced occupancy shift. However, SafeVLA already updates its Lagrange multiplier iteratively. This argument therefore explains a limitation of directly transferring static one-shot alignment, rather than a limitation of SafeVLA itself. Although the paper identifies Bellman consistency and local dualization as its contributions, it should more clearly distinguish these elements from existing adaptive primal–dual updates and explain what additional capability or efficiency they provide.
- The central formulation and its practical implications are insufficiently explained in the main text. Several presentation choices make the contribution difficult to follow:
- Figure 1(a) illustrates a change in occupancy but does not clearly demonstrate why the proposed fixed-dual transfer fails. The figure also uses occupancy notation before its formal introduction in Section 4.2. A compact example, such as the counterexample in Appendix E, would make the issue more concrete.
- The discussion around lines 154–160 would benefit from a clearer connection between occupancy shift and specific robotic safety failures (especially the concept of "policy-induced occupancy shift"). Table 2 provides relevant ablations, but the operational definitions of “remove Bellman correction” and “freeze occupancy” need clarification to establish what those comparisons isolate.
- Section 4.1 repeats much of the preceding problem formulation: Eqs. (6), (7), (8), and (9) restate Eqs. (1), (2), (4), and (5), respectively. That space could instead explain Bellman flow consistency and why it is needed.
- Around lines 208–210, the paper announces an occupancy reformulation and a dual over safety multipliers and state-flow potentials, but presents only the resulting policy in Eq. (10). Because this formulation is central to the claimed contribution, the main text should include the occupancy constraints and explicit dual optimization problem currently deferred to Appendices D and F.
Minor Comments
- Please use parenthetical citations where required by the sentence structure; several citations currently appear as detached author–year phrases. (\citep instead of \cite)
- I could not locate a dedicated AI-use disclosure section in the manuscript. Please ensure that the submission includes the statement required by the ICLR 2027 author policy.
- The caption of Table 3 contains an unresolved reference, “Table ??”. Please correct it.
Questions for authors
- Algorithms 1 and 2 substantially overlap. Please consolidate them or clearly explain their distinct purposes.
- SafeVLA already updates its multiplier and uses reward and cost advantage estimates. What specific operation in your implemented update constitutes the additional Bellman correction, and how does it differ from SafeVLA beyond solving the frozen-occupancy dual before policy distillation? In particular, what exactly is changed in the “remove Bellman correction” ablation?
- I understand that the proposed procedure retains outer occupancy-refresh iterations while replacing repeated policy–multiplier interleaving within each frozen-occupancy region with a dual solve followed by policy distillation. What are the actual numbers of dual iterations and policy optimization steps per region, and what computational savings does this achieve relative to SafeVLA under comparable conditions?