acceptodds
Withdrawn from ICLR 2027

Perception over Action? Rethinking Adversarial Robustness in Vision-Language-Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Visual adversarial patches can disrupt perception in vision–language–action (VLA) models and prevent manipulation tasks from being completed. We systematically compare robustness adaptation strategies to examine how their training objectives and adaptation scope relate to unseen-attack robustness and clean-task preservation. Specifically, we compare action-supervised adaptation of the entire policy, termed VLA Tune, with Visual Encoder Tune, which aligns visual features with a clean reference while retaining the original downstream policy. Across four LIBERO suites, Visual Encoder Tune achieves 66.6% mean clean success, compared with 55.0% for VLA Tune and 63.5% for the original policy. On LIBERO-Spatial, its mean success under three unseen attacks reaches 60.0%, versus 47.2% for VLA Tune and 0.0% for a visual-augmentation baseline. Representation diagnostics show closer recovery of clean-reference features under attack and less drift on clean observations. The advantage in task success also appears under transient and persistent patch exposure, although attacks optimized against the final defender substantially reduce performance. These findings support visual-encoder adaptation with clean-reference alignment as an effective strategy for improving transfer robustness while preserving acquired manipulation skills.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Robust VLA, Security, Safety Alignment

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_WxVeOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes to investigate the influence of robustness adaptation strategies, which is an important research topic. The experiments demonstrate that the visual-encoder adaptation with clean-reference alignment provides an effective solution.

Critical strengths
  1. Evaluating different robustness strategies is important for the reliability of VLA models.
  2. The authors conduct experiments to investigate the effectiveness of different robustness strategies, providing useful insights into their relative strengths and limitations.
Critical weaknesses
  1. The abstract does not clearly articulate the key challenges addressed by this work. Furthermore, the connection between these challenges and the claimed contributions is not sufficiently established. The authors should better highlight the limitations of existing approaches and explain how their work addresses them.
  2. In the introduction, the authors state that “Bringing corrupted visual features closer to their original clean representations may therefore provide a response that remains useful beyond a particular attack objective.” However, this idea is not particularly novel, as reducing discrepancies between clean and adversarial representations is a common objective in adversarial robustness research, including adversarial training. The authors should clarify how their motivation differs from existing approaches and what new insights their work provides.
  3. The manuscript contains several unclear or logically disconnected statements. For example, the authors write, “Whether this yields better task execution remains an empirical question: a small feature distance alone does not guarantee a successful manipulation trajectory.” However, the latter part of the sentence is a statement rather than a question, and the subsequent discussion does not directly address the empirical question raised. The authors should improve the logical flow and ensure that the questions or challenges introduced are properly addressed in the following discussion.
  4. For a paper primarily focused on evaluation, a substantial portion of the manuscript is devoted to describing existing adversarial attacks and adaptation strategies. The authors should streamline the descriptions of existing methods and devote more attention to the experimental insights and conclusions drawn from the evaluation.
  5. The main novelty and contributions of this work are not sufficiently articulated. The paper primarily presents comparisons among existing methods, without clearly identifying the critical findings or novel insights that emerge from these comparisons. A strong evaluation study should go beyond reporting comparative results to reveal meaningful patterns, explain the underlying factors influencing performance, and provide actionable insights for future research.
Questions for authors

Please refer to Critical Weaknesses.

Reviewer_fDjZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • The paper compares three ways to defend vision–language–action (VLA) policies against visual adversarial patches. VAT fine-tunes the whole policy with visual augmentations, VT uses adversarial training with action supervision, and VET updates only the visual encoder to match clean reference features, keeping the rest of the policy frozen.

  • Most experiments use OpenVLA across four LIBERO suites, with a smaller evaluation on SpatialVLA. VET achieves the best reported results, with 66.6% mean clean success compared with 55.0% for VT and 63.5% for the original policy. On LIBERO-Spatial, it achieves 60.0% success under unseen attacks, versus 47.2% for VT. The authors also examine feature changes, action predictions, and when patches appear during an episode.

  • Overall, VET works better in these experiments, but both defenses struggle against attacks tailored to them: success falls to 0% for VT and 9% for VET. The difference is also small on SpatialVLA, leaving uncertainty about whether VET offers the same benefit on other architectures.

Critical strengths

1. The evaluation covers unseen and adaptive attacks. The authors test unseen TMA targets, UADA, and UPA, along with an adaptive attack against each defended model. Reporting these separately helps distinguish robustness to transferred attacks from robustness to attacks designed for the defense. The adaptive evaluation is limited, though, as discussed in the Weaknesses.

2. VET maintains clean performance on OpenVLA. VET achieves 66.6% mean clean success across four LIBERO suites, compared with 63.5% for the original policy and 55.0% for VT. Preserving ordinary task performance is a useful practical benefit. Still, I give this finding less weight because VET closely follows earlier feature-alignment defenses (Schlarmann et al., 2024; Xu et al., 2025), and the supporting experiments have gaps discussed in the Weaknesses.

Critical weaknesses

1. The comparison does not isolate what makes VET better. VET and VT differ in four ways: which parameters are trained, the loss, whether the training patch is refreshed, and the learning rate (1e-5 versus 2e-5). Section 4 acknowledges this, but there are no ablations to separate their effects. VET may handle unseen attacks better simply because it sees more varied patches during training. Testing VT with patch updates and encoder-only training with action loss would help clarify this. VET also closely follows existing feature-matching defenses (Schlarmann et al., 2024; Xu et al., 2025), which limits the methodological novelty.

2. The evidence leaves me uncertain about the reported gains. All results use one random seed without error bars, and Table 1 does not specify the number of trials. The starting models perform below published OpenVLA results, and VT's clean success on LIBERO-Spatial changes from 62.3% in Table 1 to 72.0% in Table 5 without explanation. Tables 3 to 5 also leave the evaluation suite unspecified. The adaptive evaluation covers only one setting, omits attacks aimed directly at encoder features, and lacks a comparison with Xu et al. (2025). On SpatialVLA, the advantage over VT nearly disappears. Finally, the conclusion claims VET avoids the cost of full-policy retraining, but the paper reports no training cost, time, or memory numbers. These gaps make it difficult to judge how reliable or general VET's advantage is.

Questions for authors

Clarification questions

  1. Why does VT’s clean success on LIBERO-Spatial change from 62.3% in Table 1 to 72.0% in Table 5? Were different trials, seeds, or checkpoints used?
  2. How many rollouts does each Table 1 entry represent? Values like 62.3%, 59.4%, and 26.1% suggest a total other than 100.
  3. Which LIBERO suite is used in Tables 3, 4, and 5?
  4. How often is VET’s patch refreshed, and how many optimization steps are used each time? Do the gradients pass through the whole policy or just the encoder?
  5. What patch sizes, opacity values, and placement ranges are used for training and evaluation?
  6. What explains the gap between the clean OpenVLA baselines and the published results? Did you follow the official training recipe?

Experiments that could change my assessment

  1. I would like to see VT with patch refresh, encoder-only training with action loss, and both methods trained at the same learning rate. These comparisons would help establish where VET’s gains come from.
  2. Please report results over multiple seeds with error bars, at least for LIBERO-Spatial.
  3. A comparison with Xu et al. (2025) and an adaptive attack targeting the encoder features directly would strengthen the evaluation.
  4. Please report GPU hours, wall-clock training time, and peak memory for all three methods to support the efficiency claim.

Future work that does not affect my score

  1. Testing other VLA architectures, such as OpenVLA-OFT or π0, and evaluating physical patches or real robots would help show how well these findings carry over to other settings.
Reviewer_rbCbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper compares robustness adaptation strategies for VLA models. VLA Tune updates the full policy using action supervision, while Visual Encoder Tune aligns visual features with a frozen clean reference and preserves the downstream policy. Experiments mainly on OpenVLA show better clean-task preservation and transfer robustness with Visual Encoder Tune. Additional evaluations examine attack timing and representation changes.

Critical strengths
  1. The research question is well motivated.

  2. The comparison diagram in Figure 1 is clear and easy to follow.

Critical weaknesses
  1. The geometry optimization in VLA Tune is underspecified. Section 3.2 introduces an optimized patch geometry (\phi_t) at training stage t, but Equation (4) only defines the policy-training loss. Please specify the objective and constraints for optimizing (\phi_t), and how these updates alternate with policy training.

  2. The comparison does not establish what drives the advantage of Visual Encoder Tune. The ablation studies in the experiment section could be improved to better demonstrate the contribution of each proposed component.

  3. Qualitative results are limited to Figure 2. Comparisons across more examples, including successful and failed cases, would make the differences between methods clearer.

Questions for authors

Please address the weaknesses above.

Reviewer_XeA9OpenReview ↗
Rating
1
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

I read this paper as a study of how to improve VLA robustness while preserving existing manipulation skills. VET aligns visual features with a clean reference while keeping the downstream policy fixed. On OpenVLA, its clean success rate is 66.6%, compared with 55.0% for VT and 63.5% for the original policy. This suggests that avoiding performance degradation during adaptation is an important part of its value. However, the different patch-training procedures make it difficult to isolate the benefit of encoder-only adaptation. The smaller gains on SpatialVLA and vulnerability to adaptive attacks also leave the generality of the findings uncertain.

Critical strengths
  1. The improvements on OpenVLA are large and fairly consistent. In Table 1, VET outperforms VT under all clean and attacked conditions across the four task suites. The experimental results show that the evaluated training scheme has practical application value.

  2. The authors also report cases where the method does not perform well. VET drops from 56.0% under transfer attacks to 9.0% under adaptive attacks (Table 5), and its advantage on SpatialVLA is also much smaller. These results make it easier for readers to judge where the method is useful and where it still fails.

Critical weaknesses
  1. Although the authors explicitly frame the experiments as a comparison of complete training recipes, the current design still makes it difficult to explain where VET's advantages come from. In particular, VT fixes the patch texture, while VET refreshes it, so the gains may partly come from the diversity of training attacks. Considering that the paper's main contribution is the comparison of different adaptation strategies, I think key controlled comparisons under the same patch-refresh and geometry settings are still needed.

  2. The coverage and representativeness of the baselines are insufficient. The current defense baselines mainly include a visual augmentation scheme drawing on RobustVLA and an action-supervised fine-tuning scheme constructed in this paper. The paper has not sufficiently explained whether these two settings can represent the actual performance of existing defense methods.

  3. The key attack and adaptation training procedures cannot be fully reproduced from the paper. It does not provide the patch sizes, opacity, and geometric transformation ranges actually used in the experiments, nor does it fully explain the attack optimization step sizes, iteration and restart settings, VT's geometry optimization procedure, or VET's patch-refresh configuration. These parameters directly affect attack strength and training effectiveness.

Questions for authors
  1. How much of the gap between VET and VT remains when the patch-refresh and geometry-handling arrangements are the same? How well does action-supervised training work when only the encoder is updated, and can it achieve similar advantages?

  2. Why does the advantage become much smaller on SpatialVLA? Do the authors think this is mainly related to the visual encoder, the action interface, or the adaptation training settings?

  3. Could the authors provide a complete attack configuration table, including the specific algorithms and budgets used for VT’s geometry optimization and VET’s patch refreshing?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.