- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies the gap between local action safety and long-term task feasibility in VLA policies. The authors argue that an action may be locally safe and preferred by the policy, but still lead to states from which safe task completion is unlikely. They formalize this idea through feasible-future mass and derive conditional guarantees for finite-candidate decoding. Since computing the exact quantity is impractical, they propose VICS-G, a training-free decoder that selectively reranks candidate actions using policy likelihood, local robustness, and history-based corrections.
Experiments on Safety-CHORES show reductions in cumulative safety cost across six settings while maintaining success rates close to those of the original policy.
Critical strengths
- Well-motivated problem formulation.
I find the distinction between local safety and future feasibility meaningful, particularly for robotic tasks where an initially safe action can make later progress difficult. The feasibility–likelihood gap provides a useful perspective on the limitations of existing action selection strategies. Rather than treating safety as a property of individual actions, the paper emphasizes whether a decision preserves the possibility of eventual safe task completion.
- Principled and relatively thorough theoretical analysis.
The feasible-future formulation accounts for both future policy behavior and environment dynamics, including execution uncertainty. The distinction between feasibility support and magnitude is interesting, as it separates whether safe completion remains possible from how much probability mass is assigned to those continuations. I also appreciate the analysis of different approximation errors, including candidate coverage, support rejection, and ranking errors, as well as the authors' acknowledgment of the conditional nature of their guarantees.
- Practical and computationally efficient approach.
VICS-G does not require retraining the underlying VLA policy or performing online trajectory rollouts. The selective intervention mechanism is also appealing, as it preserves the original policy's action when no intervention is triggered rather than modifying every decision. This makes the method relatively lightweight and potentially useful in settings where retraining or extensive inference-time computation is undesirable.
- Promising empirical results and careful evaluation of safety–progress trade-offs.
Across six Safety-CHORES settings, VICS-G reduces cumulative safety cost by 1.9%–57.5%, while keeping task success within 2.5 percentage points of policy sampling. I appreciate that the authors evaluate safety cost together with task success and episode length, since lower safety cost alone can be misleading when a policy terminates prematurely. The additional ablations, execution-feedback experiments, and rollout-based study also provide useful insight into the behavior and limitations of the proposed approach.
Critical weaknesses
- The connection between the theoretical formulation and the practical decoder is not sufficiently established.
The central theoretical quantity is feasible-future mass, which depends on the safe task-completing continuations available after taking an action. However, the continuation correction in VICS-G (Eq. 46) is based on fixed action preferences, recent repetitions, and failed or collision-related retries. While the authors acknowledge that this is a heuristic rather than a calibrated estimate, it remains unclear whether it actually captures future feasibility. Furthermore, the theoretical recovery guarantees rely on score accuracy and support separation conditions that are not verified for the deployed decoder. This leaves a gap between the theoretical motivation and the practical method.
- The incremental benefit of the continuation correction remains unclear.
While the full VICS-G system achieves encouraging results, the zero-weight ablation in Appendix G provides less consistent evidence for the contribution of the continuation correction itself. At the nominal weight, all confidence intervals for changes in safety cost and violations include zero. On ObjectNav-base, adding the correction even decreases success while increasing cost and violations. The rollout-based experiment in Appendix H is interesting, but its improvement in safe success is not statistically significant and does not directly validate the heuristic used in VICS-G. These results make it difficult to determine how much of the improvement is attributable to the proposed future-feasibility correction.
- The baseline comparisons do not fully isolate the source of improvement.
RCD and VICS-G differ in several components beyond the continuation score, including the intervention gate, trust region, and replacement authorization. Comparing the complete systems is reasonable, but these differences make it difficult to attribute the observed improvements specifically to future-aware decoding. Although the zero-weight ablation helps address this concern, the Fetch ablation uses a different candidate configuration from the main evaluation. A simple retry-avoidance baseline under the same intervention mechanism would help clarify whether the proposed correction offers benefits beyond conventional history-based heuristics.
- Lack of real-world validation and uncertain practical applicability.
The experiments are conducted entirely in simulation, without real-robot evaluation. This is a concern given the paper's focus on safe robotic action execution. VICS-G relies on monitoring signals such as simulator-provided safety costs, collision feedback, object visibility, and target distance, which may be difficult to obtain reliably on physical robots. Although the authors investigate execution perturbations in Appendix I, these experiments use relatively simple action-magnitude perturbations and do not address uncertainty in the monitoring signals themselves. Without real-world validation or a more realistic robustness analysis, it remains unclear whether the reported safety improvements would transfer to physical robotic systems.
Questions for authors
-
Could the authors provide a more direct validation of the continuation correction ? In particular, how well does its ranking of candidate actions agree with their actual safe-completion probabilities under the frozen policy? Appendix H provides a rollout-based comparison, but it does not directly evaluate whether the heuristic used in VICS-G captures the theoretical feasible-future mass.
-
Appendix G provides a useful zero-weight ablation, but the contribution of the continuation correction remains unclear. Could the authors further explain why adding leads to worse performance on ObjectNav-base? Also, would the conclusions for Fetch remain the same under the top-12 candidate configuration used in the main experiments, rather than the all-20-action configuration used in the ablation?
-
Appendix F describes the monitoring signals available to the decoder, including simulator-provided safety costs and target information. How would these signals be obtained or approximated in real-world deployment, and how sensitive is the method to their availability?
5 comments
W3: “The baseline comparisons do not fully isolate the source of improvement [...] a simple retry-avoidance baseline under the same intervention mechanism would help.”
We appreciate this suggestion and have now added the requested retry-only control (Table R1 in our shared response), complementing the existing zero-weight ablation in Appendix G.
The controls address different questions. Setting removes the additional correction but retains the gate, local-risk scoring, filtering, and replacement authorization. Retry-only more directly examines whether avoiding previously ineffective actions explains the observed results.
Retry-only matches nominal VICS-G at reported episode-level precision. We therefore do not claim that non-retry preferences independently improve performance in this Fetch setting.
VICS-R provides complementary evidence through explicit continuation evaluation, although a statistically established direct VICS-R–VICS-G effect would require its own paired comparison.
We additionally evaluate SafeDec-style kinematic hard-constrained decoding (HCD) (Table R2 in our shared response). Relative to HCD, nominal VICS-G achieves 47.2% versus 25.6% task success, 21.7% versus 11.1% joint safe success, and lower cost (2.23 versus 5.49).
W4 + Q3: “Lack of real-world validation and uncertain practical applicability [...] the experiments [...] do not address uncertainty in the monitoring signals themselves.”
We agree that monitoring requirements deserve direct examination. Beyond the imperfect-execution study in Appendix I, we now evaluate simulator-cost masking (Table R3 in our shared response).
With numerical simulator-cost feedback withheld, nominal VICS-G retains 47.2% task success and 21.7% joint safe success, while achieving 13.7% lower mean cost than policy sampling. It preserves approximately 77% of its full-monitor cost advantage.
This demonstrates that favorable observed behavior does not depend entirely on numerical simulator-cost feedback. Other signals—including target monitoring and recent execution information—remain available. We do not equate this experiment with robustness to arbitrary monitoring errors or physical-robot validation.
On the overall rating
The revised experiments directly address the requests for retry-only controls, expanded explicit lookahead, a hard-constrained comparator, and restricted-monitoring evaluation. Together with the conditional approximation theory, they provide a clearer account of what the framework establishes and what remains dependent on the practical heuristic.
We respectfully ask the reviewer to reconsider the weak rejection in light of this more targeted evidence.
Follow-up on Our Responses and Revisions
Dear Reviewer hWgR,
Our detailed responses and additional experiments address the theory–implementation connection, continuation correction, baseline attribution, and monitoring limitations. The supporting evidence is presented in our shared experimental comments and revised manuscript (highlighted in purple).
We believe these additions substantially strengthen the paper. We would particularly value your thoughts on whether they help resolve your key concerns and welcome further discussion of any points that remain unclear.
We thank the reviewer for recognizing the theoretical analysis and practical efficiency of VICS. We particularly appreciate the emphasis on distinguishing the complete decoder's performance from the incremental contribution of its continuation approximation.
The key question is whether the practical correction captures future feasibility rather than merely encoding recovery heuristics.
This is precisely the approximation problem our theoretical analysis addresses. Theorem 1 identifies policy-relative feasible-future mass in the exact action marginal; Theorem 2 specifies sufficient conditions for recovering viable candidates despite coverage, support, and ranking errors.
VICS-G deliberately uses lightweight history-based signals instead of repeatedly simulating continuations. We do not assume that zG is a calibrated estimate of feasible-future mass.
We further strengthen the theory–implementation connection through VICS-R, introduced in Appendix H, which explicitly evaluates simulated continuations. Its expanded evaluation on 180 previously unseen Fetch Safe houses (Table R1 in our shared response) shows 51.7% versus 47.2% task success, 24.4% versus 21.7% joint safe success, and lower mean cost (1.85 versus 2.23) relative to VICS-G at nominal weight.
These results provide stronger episode-level evidence for future-aware decoding beyond history-based recovery. VICS-R more directly operationalizes the feasible-future principle through prospective safe-completion estimates. Since its continuations follow VICS-G, we do not claim exact estimation of the frozen-policy Zfeas or direct validation of zG.
We agree that the continuation correction must be evaluated separately from the complete decoder. Appendix G already provides this controlled comparison, holding the gate, candidate selection, local-risk scoring, and fallback fixed while varying only ηv.
ObjectNav-base is the clearest unfavorable case. At ηv=0.10, task success decreases by 3.0 percentage points relative to zero weight (paired bootstrap interval [−6.0, −0.5]), while mean cost and violations increase by 0.91 and 1.09, respectively, with intervals spanning zero. We report this result rather than suggesting uniform improvement.
However, the incremental effect of zG should be distinguished from the performance of the complete decoder. In the main ObjectNav-base evaluation (Table 1), nominal VICS-G reduces mean safety cost by 57.5% relative to policy sampling (7.965 → 3.385), while task success remains within 2.5 percentage points (78.0% → 75.5%) and joint safe success slightly increases (60.0% → 60.5%).
Importantly, we use a single shared nominal configuration across all six settings, rather than selecting the best continuation weight for each task. The ObjectNav-base result thus reveals a genuine limitation of the lightweight correction, not task-specific optimization.
The new held-out Fetch Safe evaluation provides direct paired nominal-versus-zero comparisons under the matched top-12 configuration (Table R5 in Part 2/2). Within VICS-G, the correction yields favorable changes (+0.56 pp task success, −0.21 cost, −0.12 violations).