acceptodds
Under review as a conference paper at ICLR 2027

Look Before You Flow: Inference-Time Risk Steering for Frozen Flow-Matching VLAs

Abstract

Existing vision-language-action (VLA) policies execute natural-language instructions in physical environments but lack mechanisms to avoid scene hazards during inference, including fragile objects, bystanders, and obstacles. Recent safety methods for VLAs either require retraining the policy, depend on known object geometry and dynamics models, or specify constraints in natural language without modifying the continuous action output. In this work, we introduce **MARS** (*Multi-risk Action Repulsion and Steering*), a training-free inference-time framework that steers frozen flow-matching VLAs away from scene risks by translating semantic risk knowledge from a vision-language model into bounded action-space corrections. MARS applies *Grounded Risk Verbalization* (GRV) to identify scene risks via a frozen VLM and express each as a counterfactual action prompt in the VLA's native instruction format, and *Trajectory-Anchored Flow Repulsion* (TAFR) to steer the flow-matching ODE by accumulating clipped velocity residuals in a correction state decoupled from the nominal trajectory, preventing guidance perturbations from compounding across integration steps. These designs enable MARS to bridge semantic risk recognition and bounded action-space intervention without retraining, geometric models, or learned critics. In comprehensive evaluations across three safety-critical simulation benchmarks and real-world manipulation tasks on two robot platforms, MARS improves safety across four VLA backbones ranging from 0.45B to 3B parameters. It reduces safety cost by approximately 40%, improves task success rate by up to 9.9 percentage points, and increases real-world safe success rate by up to 20 percentage points, all without modifying any model parameters. Anonymous project page: [https://mars-iclr-2026.github.io/](https://mars-iclr-2026.github.io/)

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied Intelligence, Vision-Language-Action Models, Inference-Time Safety, Robot Manipulation Safety

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Ei1GOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MARS, a training-free, inference-time safety intervention framework designed for frozen Vision-Language-Action (VLA) models based on flow matching. The method first employs "Grounded Risk Verbalization" (GRV) to identify scene-specific risks using a vision-language model, translating undesirable interactions into counterfactual action instructions. Subsequently, through "Trajectory-Anchored Flow Repulsion" (TAFR), the framework compares the velocity fields—derived from the original task instructions versus the counterfactual instructions—along a shared nominal sampling trajectory. The resulting clipped and weighted residuals are aggregated into a corrective term, which is then used to adjust the final action sequence. The authors evaluate MARS across three simulated safety benchmarks, four VLA backbones, and two real-world robotic platforms, while also providing analyses on component ablation, hyperparameter sensitivity, and computational overhead. The results indicate that, when applied to the π0.5 model, MARS simultaneously improves both task completion rates and safety; however, its benefits are more limited when applied to other backbones, in some cases even leading to performance degradation.

Critical strengths

This paper focuses on protecting surrounding objects and hazardous zones during task execution, explicitly distinguishing between task success and safe execution—a distinction that aligns with the practical requirements of deploying general-purpose robots.

MARS does not require updating VLM or VLA parameters, nor does it rely on explicit geometric models or separately trained evaluators. Its overall design is clear, bridging semantic risk identification with continuous action correction through counterfactual action instructions.

By decoupling nominal sampled trajectories from cumulative corrections, the approach prevents the correction terms from continuously shifting the evaluation points within the subsequent velocity field. The authors provide experimental evidence to support this design by comparing it against an approach that integrates corrections into the state.

The paper evaluates the method across three simulation benchmarks, four model backbones, and two real-world robotic platforms, while analyzing factors such as prompting strategies, clipping magnitudes, guidance strength, and the frequency of VLM invocations. The authors also discuss the limitations of the method when applied to less capable model backbones.

Critical weaknesses

Appendix J suggests that modifying the state integral invalidates the upper bound on the cumulative correction derived in Equation (14); however, a similar upper bound can still be derived for the cumulative correction itself, provided that dimension-wise clipping and weight constraints are maintained. It is important to distinguish between the cumulative correction and the total deviation of the final output from the original nominal trajectory. Furthermore, the fact that deviations in the action space are bounded does not, in itself, guarantee physical safety.

The collision rate reported in Table 7 is 162.50%, which exceeds the range permitted by its own definition. Additionally, the description of the numerical values ​​and the direction of change for the prompt-injection method "CC" (in Appendix H.1) is inconsistent with the data in Table 7. These issues undermine the credibility of the conclusions drawn from the relevant experiments.

The primary comparisons focus on the original policy, prompt injection, and internal integral variants. The absence of direct comparisons with other safety-filtering methods or simpler action-space correction schemes makes it difficult to fully substantiate the distinct advantages of TAFR.

The prompt for GRV requires the exclusion of "operation targets"; however, in "careful grasping" tasks, the hazardous regions are often part of the target object itself. The paper does not clearly explain how the current object-level risk representation identifies and avoids component-level risks—such as the metal shaft of a screwdriver.

The latency results presented in Tables 10 and 11 are difficult to reconcile with the described parallel execution mechanism. Moreover, the primary safety gains are concentrated in the π0.5\pi_{0.5} setting, while other backbone models fail to show consistent improvement; thus, the current results are insufficient to support a claim of broad, generalizable effectiveness.

The main results lack confidence intervals or inter-seed variance. Given that each experiment consists of only 20 trials per task for each method, it is difficult to determine whether the observed performance differences are stable.

Questions for authors
  1. Why does Equation (14) no longer hold for the cumulative correction δK\delta_K, given that the same clipping and weight constraints are applied to the correction-state integral? Is the authors' intent to constrain this cumulative correction, or rather the total output deviation relative to the unguided policy?
  2. Please verify the 162.50% CR value in Table 7, and address the discrepancy between the CC values ​​in the table and those in Appendix H.1. Following any necessary corrections, does the explanation regarding performance degradation due to prompt injection still hold?
  3. Could the authors include a baseline—such as a simple action-correction method—that utilizes the same risk information and a comparable computational budget, along with an appropriate geometric safety-filtering approach, to clearly demonstrate the incremental benefits of TAFR?
  4. In the "cautious grasping" task, how does GRV represent the hazardous regions of a target object itself? Please provide examples of actual risk records and counterfactual instructions, and clarify whether task-specific rules are employed.
  5. Are the timing conditions consistent between Table 10 and Table 11? Please explain the discrepancy (102 ms vs. 114 ms) when L=1L=1, as well as why the latency reaches 193 ms when N=1N=1 under parallel execution.
  6. Could the authors provide confidence intervals based on multiple random seeds, paired statistical analyses for real-world experiments, and an assessment of the impact of risk-cache lag in dynamic scenarios? Furthermore, for backbones that exhibit unstable performance, are there established criteria for enabling or rolling back these interventions?
10 comments

We thank Reviewer Ei1G for the thorough and constructive review. Your questions on the theoretical bound, the Table 7 typo, the missing baselines, and the CautiousGrasp scope were all well-placed, and the paper is improved as a result. We address each point below.

1. Clarification on the Theoretical Bound in Appendix I & J (W1 & Q1)

Q1: Why does Equation (14) no longer hold for the cumulative correction δK\delta_K, given that the same clipping and weight constraints are applied to the correction-state integral? Is the authors' intent to constrain this cumulative correction, or rather the total output deviation relative to the unguided policy?

Thank you for pointing out this distinction. You are right on both points, and the submitted Appendix J was imprecise. It stated that the bound of Equation (14) "no longer holds" for the cumulative correction in the corrected-state variant. That statement is wrong as written: with the same component-wise clipping bab_a and weights waw_a applied to G^k\hat{\mathbf{G}}_k, the telescoping argument of Appendix I applies verbatim to δ^K\hat{\boldsymbol{\delta}}_K, so ∣δ^K,h,a∣≤qˉ,wa,Lmax⁡,ba|\hat{\delta}_{K,h,a}| \le \bar{q}\\,w_a\\,L_{\max}\\,b_a holds in both variants. What fails in the corrected-state variant is the identity Pcorr−Pnom=δKP^{\mathrm{corr}} - P^{\mathrm{nom}} = \boldsymbol{\delta}_K, not the bound on δK\boldsymbol{\delta}_K.

  1. What Equation (14) is meant to bound. Our intent is to bound the total output deviation of the final action chunk relative to the unguided nominal policy, ∥Pcorr−Pnom∥∞\Vert P^{\mathrm{corr}} - P^{\mathrm{nom}}\Vert _\infty, where Pnom=xKnomP^{\mathrm{nom}} = \mathbf{x}_K^{\mathrm{nom}} is the endpoint of the unguided ODE integrated from the same initial noise.

    • TAFR evaluates all velocities on the unperturbed anchor trajectory, so xk≡xknom\mathbf{x}_k \equiv \mathbf{x}_k^{\mathrm{nom}} and ∥Pcorr−Pnom∥∞=∥δK∥∞≤D:=qˉ,wa,Lmax⁡,ba,\Vert P^{\mathrm{corr}} - P^{\mathrm{nom}}\Vert _\infty = \Vert \boldsymbol{\delta}_K\Vert _\infty \le D := \bar{q}\\,w_a\\,L_{\max}\\,b_a, independent of the Lipschitz properties of vθv_\theta.
    • In the Corrected-State Variant (Appendix J), evaluating v^k+=vθ(xk+δk,τk;ot,ct+)\hat{\mathbf{v}}_k^+ = v_\theta(\mathbf{x}_k + \boldsymbol{\delta}_k, \tau_k; o_t, c_t^+) at the shifted state alters every subsequent nominal velocity evaluation, so the terminal state xK\mathbf{x}_K itself drifts away from xKnom\mathbf{x}_K^{\mathrm{nom}} and ∥Pcorr−Pnom∥∞=∥(xK+δK)−xKnom∥∞≤∥xK−xKnom∥∞+∥δK∥∞.\Vert P^{\mathrm{corr}} - P^{\mathrm{nom}}\Vert _\infty = \Vert (\mathbf{x}_K + \boldsymbol{\delta}_K) - \mathbf{x}_K^{\mathrm{nom}}\Vert _\infty \le \Vert \mathbf{x}_K - \mathbf{x}_K^{\mathrm{nom}}\Vert _\infty + \Vert \boldsymbol{\delta}_K\Vert _\infty. The second term is still at most DD. The first term is not controlled by Equation (14). Writing ek=xk−xknom\mathbf{e}_k = \mathbf{x}_k - \mathbf{x}_k^{\mathrm{nom}} and letting Λ\Lambda denote the ℓ∞\ell_\infty-Lipschitz constant of vθv_\theta in its state argument (uniform over τ\tau), the Euler recursion gives ∥ek+1∥∞≤(1+Λ/K)∥ek∥∞+(Λ/K),D\Vert \mathbf{e}_{k+1}\Vert _\infty \le (1+\Lambda/K)\Vert \mathbf{e}_k\Vert _\infty + (\Lambda/K)\\,D, and since e0=0\mathbf{e}_0 = \mathbf{0}, ∥xK−xKnom∥∞≤((1+Λ/K)K−1),D≤(eΛ−1),D.\Vert \mathbf{x}_K - \mathbf{x}_K^{\mathrm{nom}}\Vert _\infty \le \big((1+\Lambda/K)^K - 1\big)\\,D \le (e^{\Lambda} - 1)\\,D. The corrected-state variant is therefore not unbounded, but its guarantee is eΛDe^{\Lambda} D rather than DD. Because vθv_\theta scales as 1/τk1/\tau_k near the end of integration, Λ\Lambda is large and not known a priori, which makes this bound vacuous in practice.
    • This is also what the trajectory-anchoring ablation measures empirically: on VLA-Arena with π0.5\pi_{0.5}, the corrected-state variant reaches 43.8% SR versus 49.2% for TAFR (Figure 5(b), Section 4.3; Table 13, Appendix H.7), consistent with the drift of the first term.
  2. Action-Space Bound vs. Physical Safety. We agree that bounding action-space deviation does not, by itself, guarantee physical safety. Equation (14) is an action-locality guarantee: it states that the steered chunk lies within a hyperparameter-controlled ℓ∞\ell_\infty neighborhood of the nominal chunk in the backbone's normalized action coordinates. Whether that neighborhood is collision-free depends on the backbone's action scale and on the scene. We have revised Section 3.2, Appendix I, and Appendix J to (i) correct the misstatement about δK\boldsymbol{\delta}_K, (ii) add the eΛDe^{\Lambda} D comparison as a new equation in Appendix J, and (iii) describe Equation (14) as action locality rather than safety.


2. Correction of the Transcription Typo in Table 7 (W2 & Q2)

Q2: Please verify the 162.50% CR value in Table 7, and address the discrepancy between the CC values in the table and those in Appendix H.1. Following any necessary corrections, does the explanation regarding performance degradation due to prompt injection still hold?

We sincerely apologize for this proofreading oversight in Table 7 and thank you for catching it. You are right that 162.50% is invalid for a collision rate in [0,100[0, 100\\%] and that 35.82 in Table 7 directly contradicted our text six lines below in Appendix H.1.

  1. Verification & Internal Manuscript Cross-Checks: Across all 5 evaluation seeds, the true values for Prompt injection are CC = 5.82 ± 0.58 on VLA-Arena and CR = 62.50 ± 1.76% on SafeLIBERO:

    • Cause of the typo: During table transcription, stray leading digits were inadvertently typed into the two safety-cost cells (5.82 → 35.82 and 62.50 → 162.50), and a subsequent column-wise formatting pass (bold for best, underline for second-best) mechanically underlined 12.55 and 79.38 based on those mistyped numbers.
    • Consistency with the submitted manuscript: Notably, the submitted prose in Appendix H.1 (six lines below Table 7) already reported the true value—"The lower CC (12.55→5.82) confirms that the robot largely fails to initiate meaningful motion rather than completing tasks more safely"—and Figure 5(b) in Section 4.3 plotted the exact same Prompt injection runs (15.7% VLA-Arena SR, 22.6% LIBERO-Safety SSR, and 7.5% SafeLIBERO SSR).
  2. Corrected Table 7 with 5-Seed Statistics: Below we report the corrected Table 7 together with SafeLIBERO SR:

Strategy (π0.5\pi_{0.5} Backbone, 5 Seeds)VLA-Arena SR (↑\uparrow)VLA-Arena CC (↓\downarrow)LIBERO-Safety SSR (↑\uparrow)SafeLIBERO SR (↑\uparrow)SafeLIBERO SSR (↑\uparrow)SafeLIBERO CR (↓\downarrow)
No VLM guidance39.3 ± 2.3%12.55 ± 0.9156.4 ± 2.5%59.69 ± 1.74%18.75 ± 1.35%79.38 ± 1.47%
Prompt injection15.7 ± 1.5%5.82 ± 0.5822.6 ± 1.9%24.38 ± 1.52%7.50 ± 0.93%62.50 ± 1.76%
Counterfactual (GRV, Ours)49.2 ± 2.4%7.53 ± 0.6860.8 ± 2.1%61.44 ± 1.78%21.25 ± 1.51%47.63 ± 1.93%
  1. Does the Explanation in Appendix H.1 Still Hold? Yes—correcting the table to CC = 5.82 and CR = 62.50% directly restores the explanation in Appendix H.1 (which had been drafted from the true 5.82 value):
    • Appending explicit negative avoidance clauses (e.g., "pick up the bowl and place it on the plate, while avoiding the wine bottle") produces compound instructions with linguistic negation that lie outside π0.5\pi_{0.5}'s single-objective fine-tuning distribution.
    • Just as SmolVLA in Tables 1 and 4 incurs low CC (8.43) only because of very low task completion (18.0% SR), Prompt injection's lower raw contact count (CC = 5.82, CR = 62.50%) reflects motion failure rather than safe execution (SR drops from 39.3% → 15.7% on VLA-Arena and 59.69% → 24.38% on SafeLIBERO; SSR drops from 18.75% → 7.50%). By contrast, Counterfactual (GRV) + TAFR keeps each prompt as an in-distribution, single-objective affirmative command (e.g., "pick up the wine bottle") and applies repulsion in velocity space, simultaneously achieving the highest task completion and safe success rate (49.2% VLA-Arena SR, 60.8% LIBERO-Safety SSR, 61.44% SafeLIBERO SR / 21.25% SSR) and the lowest overall collision rate (47.63% SafeLIBERO CR). We have fixed Table 7 in Section 4.3 and Appendix H.1.

Reviewer_wqBcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MARS, a training-free framework for improving the safety of frozen flow-matching VLAs through counterfactual action guidance. The paper addresses an important problem and presents a reasonably motivated approach with extensive empirical evaluation. However, the methodological novelty relative to existing guidance techniques, the validity of the proposed repulsion mechanism, and its generalizability across different VLA backbones warrant further investigation.

Critical strengths
  1. Important and well-motivated problem. The paper addresses inference-time safety enhancement for frozen VLAs without requiring policy retraining or additional safety critics. This is a practically relevant problem, particularly for adapting robotic policies to dynamically changing scene-level risks.

  2. Intuitive and interesting methodological design. The idea of converting semantic risks into affirmative counterfactual action instructions is insightful. By leveraging the VLA's existing language-action grounding capabilities, MARS provides a simple way to translate high-level semantic constraints into continuous action-space corrections.

  3. Well-motivated trajectory-anchored guidance. TAFR decouples the nominal trajectory from the correction state, preventing guidance perturbations from affecting subsequent velocity evaluations. The design is conceptually clear, and the theoretical correction bound and ablation results provide supporting evidence.

  4. Extensive empirical evaluation. The paper evaluates MARS across three simulation benchmarks, four VLA backbones, and two real-world robotic platforms. The improvements on π0.5 are particularly encouraging, demonstrating the potential to improve both task completion and safety without modifying model parameters.

Critical weaknesses
  1. Insufficient justification for the safety interpretation of velocity residuals. The method assumes that subtracting the counterfactual velocity produces a meaningful risk-avoidance direction. However, differences between action instructions do not necessarily correspond to safer trajectories. Moreover, the theoretical correction bound limits action perturbations rather than guaranteeing safety. This central assumption deserves more direct theoretical or empirical validation.

  2. The most substantial improvements are concentrated on π0.5, while the gains on other backbones are limited or occasionally negative. In particular, collision rates on SafeLIBERO slightly increase for the other three backbones. These results raise concerns about the general applicability of MARS, despite its plug-and-play design.

  3. The ablations mainly compare prompt injection, corrected-state steering, and TAFR. Comparisons with standard CFG-style guidance or direct counterfactual repulsion in the final action space would help isolate the benefits of the proposed ODE-level steering mechanism. Additionally, confidence intervals for the main results would strengthen the empirical conclusions.

Questions for authors
  1. Under what conditions does the velocity difference between nominal and counterfactual instructions provide a meaningful safety-improving direction? Could the authors directly evaluate its alignment with ground-truth risk-reduction directions in simulation?

  2. What is the fundamental advantage of TAFR over standard negative guidance or CFG-style flow guidance? Could the authors provide comparisons using identical counterfactual instructions and comparable inference budgets?

  3. Can repulsion completion rate reliably predict whether MARS will improve safety on a given backbone? Could an adaptive mechanism suppress unreliable guidance to prevent the performance degradation observed on weaker VLAs?

  4. How does the proposed framework handle safety constraints involving specific contact regions or context-dependent interactions rather than simply avoiding non-target objects? Can the counterfactual prompting mechanism generalize to these more complex semantic constraints?

9 comments

We thank Reviewer wqBc for the thoughtful and constructive review. Your questions on the safety interpretation of velocity residuals, the CFG-style comparison, backbone-adaptive gating, and part-level constraints all identified genuine gaps, and the paper is stronger for it. We address each point below.

1. Why Velocity Residuals Provide a Risk-Avoidance Direction & Direct Alignment Validation (W1 & Q1)

Q1: Under what conditions does the velocity difference between nominal and counterfactual instructions provide a meaningful safety-improving direction? Could the authors directly evaluate its alignment with ground-truth risk-reduction directions in simulation?

  1. Theoretical Conditions for Valid Repulsion Directions: Under our flow-matching parameterization xτ=(1−τ)x0+τx1\mathbf{x}_\tau = (1-\tau)\mathbf{x}_0 + \tau\mathbf{x}_1 integrated backward from τ0=1\tau_0 = 1 to τK=0\tau_K = 0 with step Δt=−1/K<0\Delta t = -1/K \lt 0 (Section 3, Eqs. 1–10), the optimal conditional velocity field satisfies vθ(xk,τk;ot,c)=E[x1−x0∣xτk=xk,ot,c]=(xk−E[x0∣xk,ot,c])/τkv_\theta(\mathbf{x}_k, \tau_k; o_t, c) = \mathbb{E}[\mathbf{x}_1 - \mathbf{x}_0 \mid \mathbf{x}_{\tau_k} = \mathbf{x}_k, o_t, c] = \big(\mathbf{x}_k - \mathbb{E}[\mathbf{x}_0 \mid \mathbf{x}_k, o_t, c]\big)/\tau_k. Because TAFR evaluates both the nominal task instruction ct+c_t^+ and each counterfactual risk instruction ct,i−c_{t,i}^- at the exact same anchor state xk\mathbf{x}_k, the xk/τk\mathbf{x}_k/\tau_k term cancels in the velocity residual Ri,k=vk+−vi,k−\mathbf{R}_{i,k} = \mathbf{v}_k^+ - \mathbf{v}_{i,k}^-, and multiplying by Δt=−1/K<0\Delta t = -1/K \lt 0 in Eq. (10) (δk+1=δk+Δt,qk,Gk\boldsymbol{\delta}_{k+1} = \boldsymbol{\delta}_k + \Delta t\\,q_k\\,\mathbf{G}_k) yields an unclipped one-step action shift proportional to the difference of expected clean target chunks: Δt,Ri,k=1K,τk(E[x0∣xk,ot,ct+]−E[x0∣xk,ot,ct,i−]).\Delta t\\,\mathbf{R}_{i,k} = \frac{1}{K\\,\tau_k}\Big(\mathbb{E}[\mathbf{x}_0 \mid \mathbf{x}_k, o_t, c_t^+] - \mathbb{E}[\mathbf{x}_0 \mid \mathbf{x}_k, o_t, c_{t,i}^-]\Big). Each step therefore moves the correction along the difference between the predicted task chunk and the predicted risk-interaction chunk. The factor 1/τk1/\tau_k grows to KK at the final step (up to 10×10\times for K=10K = 10), which is one reason the per-component clip bab_a in Eq. (4) is needed: without it (ba=∞b_a = \infty), VLA-Arena CC rises from 7.53 to 10.31 (Table 8). Whether this action-space shift produces a physical risk-avoidance direction in workspace is an empirical property of the learned policy that holds under two concrete conditions:
    • Condition 1 (Counterfactual Grounding Competence): The backbone vθv_\theta must ground ct,i−c_{t,i}^- into an action distribution whose expected trajectory moves toward interacting with risk object ii (E[x0∣xk,ot,ct,i−]≠E[x0∣xk,ot,ct+]\mathbb{E}[\mathbf{x}_0 \mid \mathbf{x}_k, o_t, c_{t,i}^-] \neq \mathbb{E}[\mathbf{x}_0 \mid \mathbf{x}_k, o_t, c_t^+]).
    • Condition 2 (Shared-Anchor Locality): Both expectations must be conditioned on the same intermediate state xk\mathbf{x}_k along the nominal trajectory so that their difference measures the local displacement vector pointing away from the risk-contact mode E[x0∣xk,ct,i−]\mathbb{E}[\mathbf{x}_0 \mid \mathbf{x}_k, c_{t,i}^-] and toward the task-completion mode E[x0∣xk,ct+]\mathbb{E}[\mathbf{x}_0 \mid \mathbf{x}_k, c_t^+].
    • Role of the Theoretical Bound (W1): We also agree with your point in W1 that the bound in Equation (14) constrains action-coordinate locality relative to the nominal chunk (∥Pcorr−Pnom∥∞≤qˉ,wa,Lmax⁡,ba\Vert P^{\mathrm{corr}} - P^{\mathrm{nom}}\Vert _\infty \le \bar{q}\\,w_a\\,L_{\max}\\,b_a) rather than certifying physical collision-free execution; we have stated this distinction explicitly in Appendix I.
  1. Direct Empirical Evaluation Against Ground-Truth Risk Positions in Simulation:
    • In Appendix E (Eq. 11) and Section 4.3 (Figure 6(a)), we used the simulator's ground-truth 3D object coordinates prisk,i\mathbf{p}_{\mathrm{risk},i} to test whether conditioning on ct,i−c_{t,i}^- moves the action chunk endpoint closer to the risk object than ct+c_t^+ (∥Pi,H−−prisk,i∥2<∥PH−prisk,i∥2\Vert P_{i,H}^- - \mathbf{p}_{\mathrm{risk},i}\Vert _2 \lt \Vert P_H - \mathbf{p}_{\mathrm{risk},i}\Vert _2). On π0.5\pi_{0.5}, this holds in 73.0% of risk instances, confirming that the counterfactual branch consistently moves toward the hazard and the integrated TAFR correction δK\boldsymbol{\delta}_K points away from it.
    • Direct Cosine Alignment & Surface-Clearance Verification in Simulator Replay: To directly test whether the accumulated anchored correction δK\boldsymbol{\delta}_K points along the ground-truth risk-reduction direction in physical 3D workspace, we replayed identical simulator states and executed matched 5-step action branches for the nominal anchor (xKnom\mathbf{x}_K^{\mathrm{nom}}) and TAFR (xKnom+δK\mathbf{x}_K^{\mathrm{nom}} + \boldsymbol{\delta}_K), measuring the cosine similarity between the induced end-effector displacement Δpee=peeTAFR−peenom\Delta\mathbf{p}_{\mathrm{ee}} = \mathbf{p}_{\mathrm{ee}}^{\mathrm{TAFR}} - \mathbf{p}_{\mathrm{ee}}^{\mathrm{nom}} and the ground-truth outward surface normal usafe=(peenom−psurf∗)/∥peenom−psurf∗∥2\mathbf{u}_{\mathrm{safe}} = (\mathbf{p}_{\mathrm{ee}}^{\mathrm{nom}} - \mathbf{p}_{\mathrm{surf}}^{*}) / \Vert \mathbf{p}_{\mathrm{ee}}^{\mathrm{nom}} - \mathbf{p}_{\mathrm{surf}}^{*}\Vert _2 from the closest point psurf∗\mathbf{p}_{\mathrm{surf}}^{*} on the obstacle's MuJoCo collision mesh:
      • Across active obstacle-approach replanning steps (StaticObstacles steps 30–70 and DynamicObstacles pass window steps 55–95), Δpee\Delta\mathbf{p}_{\mathrm{ee}} lies in the risk-reducing half-space (cos⁡(Δpee,usafe)>0\cos(\Delta\mathbf{p}_{\mathrm{ee}}, \mathbf{u}_{\mathrm{safe}}) \gt 0, increasing true surface clearance Δdsurf>0\Delta d_{\mathrm{surf}} \gt 0) in 83.3% (15/18) of steps (7/9 = 77.8% in static approach and 8/9 = 88.9% in dynamic pass).
      • At critical close-proximity states, alignment is near-collinear with the ground-truth outward normal: cos⁡θ=+0.999\cos\theta = \mathbf{+0.999} (step 55) and +0.806\mathbf{+0.806} at minimum clearance (dsurf=31.6,mmd_{\mathrm{surf}} = 31.6\\,\text{mm}, step 60), +0.965\mathbf{+0.965} (Δdsurf=+0.45,mm\Delta d_{\mathrm{surf}} = +0.45\\,\text{mm}, step 70) on static hazards, and +0.997\mathbf{+0.997} (Δdsurf=+1.44,mm\Delta d_{\mathrm{surf}} = +1.44\\,\text{mm}, step 80) during dynamic obstacle pass. We have added this direct alignment analysis to Section 4.3 and Appendix E.
Reviewer_CLH4OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MARS, a training-free, inference-time wrapper that steers frozen flow-matching VLA policies away from scene hazards without retraining, geometric models, or learned critics. Grounded Risk Verbalization (GRV) uses a frozen VLM to detect non-target risk objects and scores each by grounding confidence and proximity. It then expresses each risk as an affirmative counterfactual action command (e.g., "pick up the wine bottle") that stays within the VLA's instruction distribution. Trajectory-Anchored Flow Repulsion (TAFR) evaluates the VLA's velocity field from shared noise under the task instruction and under each counterfactual prompt. It accumulates clipped, severity-weighted residuals in a correction state decoupled from the nominal trajectory, which yields a hyperparameter-bounded correction to the nominal action chunk. Experiments cover three simulation safety benchmarks (VLA-Arena, LIBERO-Safety, SafeLIBERO) with four backbones (0.45B–3B), plus two real robots on four tabletop tasks. Gains are concentrated on π0.5: VLA-Arena SR rises from 39.3 to 49.2 while cumulative cost falls from 12.6 to 7.5, and SafeLIBERO collision rate drops from 79.4% to 47.6%. Effects on weaker backbones are small and sometimes negative. Real-world SSR improves from 55% to 75% and from 56.7% to 73.3%. Ablations compare against prompt injection and a non-anchored variant. A component analysis identifies the backbone's ability to distinguish nominal from counterfactual prompts as the main bottleneck.

Critical strengths
  1. Simple, training-free, backbone-agnostic mechanism. Expressing risks as affirmative counterfactual commands rather than avoidance clauses keeps the VLA's conditioning in-distribution; the prompt-injection ablation (VLA-Arena SR 39.3→15.7) shows why this matters. Because MARS wraps any flow-matching VLA without retraining, it offers a practical, deployable path to scene-adaptive safety.

  2. Bounded, anchored intervention. Evaluating all velocities on the nominal trajectory leaves the original sampling path untouched and gives a correction bound that depends only on hyperparameters. The anchored variant outperforms corrected-state steering (49.2 vs. 43.8 SR). Limiting how far a correction can push the action is essential for any deployable safety layer.

  3. Broad, candid evaluation. Results span three simulation benchmarks, four backbones, and two real robots. The paper openly reports neutral or negative effects on weaker backbones and includes a component analysis explaining them. This transparency makes the claimed π0.5 gains more credible and clarifies when the method applies.

Critical weaknesses
  1. Generality is overstated. The abstract claims MARS "improves safety across four VLA backbones," and the contributions call it "compatible with any flow-matching-based VLA." Yet meaningful gains appear only on π0.5. On StarVLA-PI, StarVLA-GR00T, and SmolVLA, SafeLIBERO CR is slightly worse in all three cases (e.g., 83.3→83.7, 53.9→54.3), and VLA-Arena CC increases for GR00T and SmolVLA. The real-world experiments use only π0.5. The headline "~40% cost reduction" should be explicitly scoped to π0.5.

  2. The SafeLIBERO collision reduction mostly reflects collision-free failures. From the reported SR/SSR/CR, collision-free but failed episodes rise from ~1.9% to ~31.1%, while SSR improves only from 18.75 to 21.25. Thus most of the ~32 pp CR drop comes from failed episodes that no longer touch obstacles, not from safer task completion. SR also drops on three of four suites (Spatial, Object, Goal); the aggregate SR gain comes entirely from Long (42.0→56.9).

  3. Missing external baselines. MARS is compared only with unguided backbones and its own variants. SafeLIBERO was introduced alongside the plug-and-play AEGIS safety layer [1], which makes AEGIS an obvious comparison. OmniGuide [2], which the authors identify as the closest training-free method, is also not evaluated. A simple geometric repulsion baseline built on the VLM's bounding boxes would further clarify the value of counterfactual velocity residuals.

  4. Limited statistical rigor. No variance or confidence intervals are reported across the five seeds, although many gains are only 1–4 pp. The defaults for b_a and w_a are chosen via ablations on the same evaluation benchmarks, where they are optimal on all three, which amounts to tuning on the test set. Real-world results rely on 20 trials per task.

References [1] Hu et al., "VLSA: Vision-Language-Action Models with Plug-and-Play Safety Constraint Layer," arXiv:2512.11891, 2025. [2] Song et al., "OmniGuide: Universal Guidance Fields for Enhancing Generalist Robot Policies," arXiv:2603.10052, 2026.

Questions for authors

See weakness

8 comments

We thank Reviewer CLH4 for the thorough and constructive review. Every point you raised — scoping generality to π0.5\pi_{0.5}, decomposing the SafeLIBERO CR reduction, adding external baselines, and strengthening statistical reporting — was valid, and the paper is substantially improved as a result. We address each point below.


1. Scoping Generality Claims Across Backbones (W1)

W1: Generality is overstated. The abstract claims MARS "improves safety across four VLA backbones," and the contributions call it "compatible with any flow-matching-based VLA." Yet meaningful gains appear only on π0.5\pi_{0.5}... The headline "~40% cost reduction" should be explicitly scoped to π0.5\pi_{0.5}.

We agree and have revised the paper accordingly:

  • The Abstract, Introduction, Contributions and Conclusion now attribute the headline gains, including the "~40% cost reduction", to π0.5\pi_{0.5} only.
  • "Compatible with any flow-matching VLA" now means architecturally compatible, not empirically effective.

As the component analysis in Section 4.3 (Figure 6(a)) shows, MARS's safety gains depend on the backbone's language generalization. On π0.5\pi_{0.5} — pretrained on diverse language-conditioned demonstrations — the backbone grounds open-vocabulary counterfactual prompts into distinguishable velocity fields (73.0% repulsion completion rate), enabling strong safety improvements. On narrowly fine-tuned backbones (StarVLA-PI 42.7%, StarVLA-GR00T 38.3%, SmolVLA 18.0%, and <30%< 30\% for the four additional policies), the residuals are negligible. To ensure MARS does not degrade these backbones, we evaluated a two-stage gating rule (MARS-Gated, Section 4.3 and Appendix H.6) that detects low-confidence residuals and sets δK←0\boldsymbol{\delta}_K \leftarrow \mathbf{0}, recovering the exact unguided trajectory:

Backbone (RCR\mathrm{RCR})VariantActive Guidance RateVLA-Arena (SR ↑\uparrow / CC ↓\downarrow)LIBERO-Safety (SSR ↑\uparrow)SafeLIBERO (SR ↑\uparrow / SSR ↑\uparrow / CR ↓\downarrow)
π0.5\pi_{0.5} (73.0%)Unguided / MARS / MARS-Gated0% / 100% / 98.4%39.3% / 49.2% / 49.2% (12.55 / 7.53 / 7.53)56.4% / 60.8% / 60.8%59.7/18.8/79.4% → 61.4/21.3/47.6% → 61.4/21.3/47.6%
StarVLA-PI (42.7%)Unguided / MARS / MARS-Gated0% / 100% / 48.2%34.8% / 36.1% / 36.3% (16.40 / 12.73 / 12.68)38.8% / 40.2% / 40.3%28.8/8.8/83.3% → 28.6/8.6/83.7% → 29.7/8.9/83.4%
StarVLA-GR00T (38.3%)Unguided / MARS / MARS-Gated0% / 100% / 55.0%25.0% / 26.9% / 26.5% (16.93 / 17.26 / 16.54)36.1% / 37.0% / 37.0%24.4/8.6/82.1% → 24.6/8.8/82.4% → 24.7/8.7/81.7%
SmolVLA (18.0%)Unguided / MARS / MARS-Gated0% / 100% / 15.0%18.0% / 21.7% / 21.4% (8.43 / 8.60 / 8.38)29.2% / 28.8% / 29.3%7.1/2.1/53.9% → 6.6/2.1/54.3% → 7.1/2.1/53.9%

Gating preserves 98.4% of active steering on π0.5\pi_{0.5} without any performance loss, while automatically rolling back low-confidence interventions on StarVLA-PI, StarVLA-GR00T, and SmolVLA—eliminating negative transfer on VLA-Arena CC (17.26 → 16.54 on StarVLA-GR00T, 8.60 → 8.38 on SmolVLA) and SafeLIBERO (SmolVLA returns to 7.1/2.1/53.9%, StarVLA-GR00T CR drops 82.4% → 81.7%, and StarVLA-PI SR rises to 29.7%, +0.9 pp over baseline, with CR 83.4% within 0.1 pp of baseline).

Out-of-Sample Validation on Four Additional VLA Architectures: To further validate that MARS's safety benefits are not confined to π0.5\pi_{0.5}, we evaluated MARS on four additional open-source VLA/world-model policies beyond the four backbones in Table 1—Cosmos3-Edge, VLA-JEPA, Cosmos-Policy, and LingBot-VLA—using their publicly released LIBERO-finetuned checkpoints on SafeLIBERO (the only benchmark sharing scenes and objects with LIBERO).

Backbone (SafeLIBERO)Unguided (SR / SSR / CR)MARS (SR / SSR / CR)Δ\Delta SR / Δ\Delta SSR / Δ\Delta CR
Cosmos3-Edge39.06 / 9.06 / 88.1240.81 / 22.03 / 53.28+1.75 / +12.97 / −34.84
VLA-JEPA30.00 / 7.50 / 86.5629.06 / 16.56 / 50.31−0.94 / +9.06 / −36.25
Cosmos-Policy34.06 / 7.19 / 87.8130.19 / 14.22 / 50.31−3.87 / +7.03 / −37.50
LingBot-VLA25.62 / 7.50 / 86.2517.50 / 7.66 / 83.44−8.12 / +0.16 / −2.81

Even on these narrowly fine-tuned backbones, MARS reduces CR across all four (−2.81 to −37.50 pp) and raises SSR on three of four (+7 to +13 pp). Cosmos3-Edge also gains SR (+1.75 pp). The CR reduction trades off SR on Cosmos-Policy (−3.87 pp) and LingBot-VLA (−8.12 pp), where the frozen policy lacks collision-free alternative paths. On VLA-JEPA the SR cost is negligible (−0.94 pp), while the SSR-to-SR ratio roughly doubles on Cosmos3-Edge (9.06%→22.03%) and VLA-JEPA (7.50%→16.56%). Where the trade-off is unfavorable, MARS-Gated rolls back to the exact unguided trajectory, ensuring MARS never degrades a weaker backbone. Full per-backbone comparisons with all external baselines are provided in Appendix H.6.


2. Decomposition of SafeLIBERO Collision Reduction & Per-Suite Analysis (W2)

W2: The SafeLIBERO collision reduction mostly reflects collision-free failures. From the reported SR/SSR/CR, collision-free but failed episodes rise from ~1.9% to ~31.1%, while SSR improves only from 18.75 to 21.25... SR also drops on three of four suites (Spatial, Object, Goal); the aggregate SR gain comes entirely from Long (42.0->56.9).

Your decomposition is exactly right, and it exposed a framing problem in our original manuscript. We now state it plainly in Sec. 4.1 and Appendix F.2:

π0.5\pi_{0.5}, SafeLIBEROUnguided+ MARSΔ\Delta
Safe success18.75%21.25%+2.50
Unsafe success40.94%40.19%−0.75
Collision-free failure1.88%31.13%+29.25
Colliding failure38.44%7.44%−31.00
SR / CR59.69 / 79.3861.44 / 47.63+1.75 / −31.75

The dominant effect is clear: most of the CR reduction comes from converting colliding failures into non-contact failures, not from increasing safe completions. The SSR gain (+2.5 pp) and SR gain (+1.75 pp) are not statistically significant. We acknowledge this honestly in the revised manuscript and no longer frame the CR reduction as evidence of safer task completion.

That said, we believe this conversion still represents a meaningful safety improvement in practice. When the frozen policy's only known grasp corridor is blocked by an obstacle, unguided π0.5\pi_{0.5} collides with the obstacle and fails (38.44% of episodes). MARS repels the arm away, and the episode terminates without contact—failed, but without disturbing the environment. In deployment scenarios where collisions carry real costs (e.g., knocking over fragile objects, triggering emergency stops), failing safely is preferable to failing destructively, even when the task itself is not completed.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Final Summary

Dear Area Chair, Senior Area Chair, and Reviewers,

We thank all three reviewers for their feedback and thank the reviewer who raised their score to 3. Below we summarize (1) our core contribution and findings, and (2) how the initial concerns have been resolved.


1. Core Idea and Key Findings

Existing inference-time safety layers for VLA policies either inject negative prompts or impose 3D geometric repulsion requiring RGB-D, depth, and kinematics. We ask: can a frozen flow-matching VLA steer itself away from hazards using its own velocity field and risks from a frozen VLM with 256×256256\times256 RGB?

MARS connects two ideas:

  1. GRV: VLAs are trained on affirmative demos, not negations. GRV uses a frozen VLM to identify hazards and express them as affirmative counterfactual commands, keeping conditioning in-distribution and turning the VLA's own instruction-following competence into a steering signal.
  2. TAFR: Standard negative CFG subtracts counterfactual velocity inside a single evolving state, causing compounding drift. TAFR evaluates both velocities along the unperturbed nominal trajectory and accumulates clipped residuals in a decoupled correction δk\boldsymbol{\delta}_k.

Three key findings on π0.5\pi_{0.5}:

  • Affirmative steering vs. prompt injection: Appending "avoid [obstacles]" drops VLA-Arena SR from 39.339.3\\% to 15.715.7\\%; MARS raises SR to 49.249.2\\% (p<10−10p \lt 10^{-10}) and reduces CC by 4040\\% (12.55→7.5312.55\to7.53).
  • Trajectory anchoring matters: Under identical settings, vanilla negative CFG nearly doubles CC (23.5023.50) with CR unchanged (80.0080.00\\%); Action-Endpoint Repulsion barely helps (CC 18.0618.06, CR 78.7578.75\\%). CFG-style corrected-state recovers partial gains (CC 9.849.84, CR 58.4458.44\\%); anchoring on the nominal path (TAFR) unlocks full improvement (CC 7.537.53, CR 47.6347.63\\%) with a Lipschitz-free bound ∥Pcorr−Pnom∥∞≤D\Vert \mathbf{P}^{\mathrm{corr}}-\mathbf{P}^{\mathrm{nom}}\Vert _\infty\le D.
  • RGB-only vs. 3D methods: MARS achieves the lowest VLA-Arena CC (7.537.53) and highest LIBERO-Safety SSR (60.860.8\\%). OmniGuide achieves higher SR (52.552.5\\%) but at much higher CC (19.8719.87); Workspace Repulsion and scene-cropped AEGIS achieve higher SafeLIBERO SSR but with large drops on other benchmarks. MARS transfers zero-shot to two real robots (120120 pairs: safe SR 55.855.8\\%\to74.2\\%, p=2.7×10−5p=2.7\times10^{-5}).

2. How Review Concerns Were Resolved

  1. External & Equal-Budget Baselines (CLH4 W3, Ei1G Q3, wqBc W3/Q2): We compare three NFE-matched RGB ablations (Vanilla Negative CFG, CFG-Style Corrected-State, Action-Endpoint Repulsion) and three external 3D methods (Workspace Repulsion, AEGIS/VLSA, OmniGuide) across all three benchmarks. We candidly discuss where RGB-D methods hold advantages.

  2. Backbone Generality & Gating (CLH4 W1, Ei1G W5/Q6, wqBc W2/Q3): Headline gains are on π0.5\pi_{0.5}. A training-free gate (MARS-Gated) retains steering on 98.498.4\\% of π0.5\pi_{0.5} steps while returning weaker backbones to unguided performance (~1 pp). Four additional backbones (Cosmos3-Edge, VLA-JEPA, Cosmos-Policy, LingBot-VLA) all show CR reductions (34.834.8–37.537.5 pp) and roughly doubled SSR on the first three.

  3. Theoretical Bound (Ei1G W1/Q1, wqBc W1/Q1): In TAFR, ∥Pcorr−Pnom∥∞=∥δK∥∞≤D\Vert \mathbf{P}^{\mathrm{corr}}-\mathbf{P}^{\mathrm{nom}}\Vert _\infty=\Vert \boldsymbol{\delta}_K\Vert _\infty\le D holds without Lipschitz assumptions, vs. (eΛ−1)D(e^\Lambda-1)D drift in the corrected-state variant. Eq. (14) is an action-locality guarantee. The counterfactual endpoint is closer to the 3D hazard in 73.073.0\\% of risk instances.

  4. SafeLIBERO Framing & Scope (CLH4 W2, Ei1G W2/Q2/Q4/Q5, wqBc Q4): The 31.7531.75 pp CR reduction (p<10−10p \lt 10^{-10}) primarily shifts colliding failures to non-contact failures; overall SR/SSR changes are not significant (p=0.31/0.08p=0.31/0.08). GRV operates at object level; part-level affordances remain future work. We added 5-seed Mean±SD, Wilson 95% CIs, and corrected typos.

These revisions resolve the raised concerns while clarifying: frozen flow-matching VLAs can be steered at inference time using RGB and affirmative counterfactual language while keeping the nominal path unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.