acceptodds
Under review as a conference paper at ICLR 2027

R2G-VLA: Risk- and Role-Guided Grounding Correction for Robust VLA Policies

Abstract

Vision-language-action (VLA) policies have advanced language-conditioned manipulation, yet their closed-loop robustness remains limited by visual grounding shifts. In real scenes, task semantics may stay unchanged while target positions, receiver locations, object relations, or viewpoints change, inducing visual-token-level distribution shifts in frozen policies. Such shifts can mislead action prediction through distractors or perturbed relational cues, making inference-time grounding correction important for robust VLA execution during deployment. We propose R2G-VLA, a risk- and role-guided grounding correction framework for robust VLA policies under physical scene perturbations. R2G-VLA uses action-token uncertainty as a self-diagnostic signal and propagates it to visual tokens to estimate token-level grounding risk. Instead of intervening on scattered high-response tokens, it aggregates risky tokens into spatially coherent regions that better match object-level perturbations in manipulation scenes. To avoid role-agnostic suppression, R2G-VLA further performs semantic role-aware token correction over target, receiver, support, background, and distractor regions, preserving task-relevant evidence while reducing misleading visual cues. The corrected visual representation is re-injected into the same policy for a second forward pass, and a safety gate accepts the correction only when uncertainty and action-shift constraints are satisfied. Experiments on LIBERO-Spatial show that R2G-VLA improves robustness across five physical scene perturbations, highlighting the effectiveness of risk- and role-guided internal grounding correction.

Primary area:
applications to robotics, autonomy, planning
Keywords:
VLA, robot manipulation, visual grounding, robustness, test-time adaptation, uncertainty estimation, token correction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_NyRmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes R2G-VLA, an inference-time grounding correction framework designed to improve the robustness of frozen Vision-Language-Action (VLA) policies under physical scene perturbations. The method estimates visual-token grounding risk by propagating action prediction uncertainty to intermediate visual representations and aggregates high-risk tokens into spatially connected regions. It then applies semantic role-dependent corrections to different visual regions, including targets, receivers, support surfaces, backgrounds, and distractors. The modified representations are processed through a second forward pass, and a safety gate determines whether to accept the corrected action.

Critical strengths
  1. The paper addresses the robustness of frozen VLA policies under physical scene perturbations. The proposed framework performs inference-time correction without modifying model parameters or input observations, providing an alternative to robustness methods that require additional training or policy adaptation.
  2. The paper evaluates multiple perturbation types and examines the contributions of individual components. The experiments cover target displacement, receiver displacement, relational changes, viewpoint shifts, and distractors. The ablation studies examine risk localization, connected-region aggregation, role-aware correction, and safety-gated inference.
Critical weaknesses
  1. The proposed method mainly combines gradient-based attribution, connected-region selection, role-dependent feature scaling, and uncertainty-based gating. These techniques are closely related to existing interpretability and token intervention methods. The paper should clarify its methodological novelty beyond integrating established components into VLA inference.
  2. All experiments are conducted using a single VLA architecture, OpenVLA, on the LIBERO-Spatial benchmark. It remains unclear whether the proposed approach generalizes to other VLA architectures, robotic tasks, or real-world environments. Broader evaluations are needed to support its applicability to general VLA policies.
  3. The paper primarily compares against frozen OpenVLA and a RobustVLA-style training baseline, rather than existing inference-time approaches such as BYOVLA and DTP. Since these methods also address robustness through visual interventions, direct comparisons under identical perturbation settings would better establish the advantages of R2G-VLA.
  4. The role-guided correction mechanism relies on explicit masks obtained from task metadata or object annotations. Such information may not be available or sufficiently accurate in real-world deployments. The paper does not adequately evaluate the sensitivity of its performance to imperfect role assignments or the feasibility of obtaining these masks automatically.
  5. The method assumes that visual tokens with large gradients of action uncertainty correspond to misleading grounding evidence. However, high sensitivity does not necessarily indicate incorrect or harmful visual information. The paper lacks quantitative validation of the risk maps against ground-truth perturbation regions or alternative attribution methods, leaving the reliability of this central assumption unclear.
Questions for authors

see the weakness.

Reviewer_aByHOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes R2G-VLA, which uses gradients of action-token entropy to identify high-risk visual tokens, applies connected-region aggregation and role-guided feature scaling, and uses a safety gate to select the final action. Experiments with OpenVLA on LIBERO-Spatial show improved task success rates under physical scene perturbations.

Critical strengths

None

Critical weaknesses
  1. The experimental evaluation is insufficient. The method is evaluated only with OpenVLA on LIBERO-Spatial, without experiments on other mainstream VLA models, additional benchmarks, or real robots. These results provide limited evidence of the method's general effectiveness and practical applicability. The current grounding-risk measure also relies on discrete action-token distributions, and its applicability to flow-matching-based VLAs such as π0\pi_0 remains unclear.

  2. The authors define grounding risk using the gradient norm of action-token entropy, but this measure only reflects local sensitivity. The paper does not sufficiently establish the relationship between high sensitivity and incorrect grounding, or explain why feature scaling can correct such errors. The experiments also lack direct evidence supporting this mechanism.

  3. Role-guided correction relies on simulator object metadata and counterfactual rendering to construct role masks, limiting its applicability to real-world settings. The existing ablations do not sufficiently distinguish the contribution of semantic role information from that of feature scaling strength.

Questions for authors
  1. Could the authors provide experiments on additional simulation benchmarks and real robots to evaluate the method's generalization and practical effectiveness? Please also clarify its applicability to flow-matching-based VLAs such as π0\pi_0 and narrow the relevant claims if necessary.

  2. Could the authors provide direct evidence that the proposed grounding-risk measure identifies incorrect grounding and explain why feature scaling can effectively correct such errors?

  3. How can role-guided correction be implemented without privileged simulator information? Could the authors further validate the independent contribution of semantic role information?

Reviewer_p5yuOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes R2G-VLA, an inference-time grounding correction framework designed to improve the robustness of frozen vision-language-action (VLA) policies under task-preserving physical scene perturbations. The method first estimates token-level grounding risk using gradients of action uncertainty and aggregates high-risk tokens into spatially coherent regions. It then applies semantic role-guided token correction, preserving task-relevant visual evidence while attenuating potentially misleading background and distractor features. A conservative safety gate determines whether to accept the corrected action based on uncertainty and action-shift constraints. Experiments on LIBERO-Spatial demonstrate improvements over frozen OpenVLA and a RobustVLA-style baseline, with the average success rate across five perturbation types increasing from 46.8% to 56.4% compared with OpenVLA. Ablation studies further support the contributions of the proposed components.

Critical strengths
  1. The paper addresses visual grounding drift in frozen VLA policies under task-preserving physical scene perturbations. Improving robustness without additional policy training is a meaningful and practically relevant research direction.

  2. The combination of uncertainty-based risk localization and role-aware token correction is intuitively reasonable. Distinguishing task-relevant objects from distracting evidence provides a useful perspective beyond uniform token suppression.

  3. The framework integrates action-uncertainty attribution, connected-region aggregation, role-guided correction, and safety-gated reinference into a coherent pipeline. Each component has a clear motivation, and the overall design is relatively interpretable.

  4. R2G-VLA improves the perturbed average success rate from 46.8% to 56.4% over frozen OpenVLA on LIBERO-Spatial. The component-level ablations provide supporting evidence for the effectiveness of the proposed design.

Critical weaknesses
  1. Role-guided correction relies on explicit semantic role masks, which can be obtained from task metadata and object-level annotations in simulation. This introduces additional structured information that may not be readily available in real-world deployment and potentially complicates fair comparisons with baselines that do not use such information.

  2. The proposed risk map uses the gradient magnitude of action uncertainty, but high sensitivity does not necessarily indicate incorrect visual grounding. Moreover, role-guided correction primarily attenuates visual token representations. Although the ablations demonstrate its empirical utility, whether this mechanism actually identifies and corrects grounding errors remains insufficiently established.

  3. The framework combines established techniques, including gradient-based attribution, spatial region aggregation, feature reweighting, and conservative action selection. While their integration is meaningful, the methodological novelty beyond existing attribution-based and token-intervention approaches could be better substantiated.

  4. Evaluation is restricted to OpenVLA and 10 LIBERO-Spatial tasks, with no direct comparisons against closely related inference-time approaches such as BYOVLA and DTP. Furthermore, each condition contains only 50 episodes, and the main paper does not report confidence intervals or statistical significance analyses, making the reliability of smaller improvements difficult to assess.

Questions for authors
  1. How dependent is R2G-VLA on privileged semantic information? Could the authors evaluate robustness to imperfect role assignments and clarify whether the baselines have access to equivalent semantic information?

  2. Does the proposed correction mechanism genuinely improve visual grounding? Can the authors quantitatively validate the relationship between uncertainty-gradient risk scores and actual grounding errors?

  3. What is the distinctive advantage over existing inference-time interventions?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.