- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the robustness of Vision-Language-Action (VLA) models against "one-step observation perturbations," where visual or proprioceptive inputs are severely corrupted at a single inference step within an episode. Across 8 perturbation types and 5 VLA models, the authors find that single-step corruptions at critical moments severely degrade task success rates, and longer action chunk execution lengths amplify this vulnerability. To mitigate this, they propose CARE, a training-free method that dynamically shortens the execution length when the newly predicted action chunk diverges from the unexecuted portion of the previous chunk. Experiments on LIBERO, Meta-World, and a bimanual robot setup validate the approach under synthetic single-step corruptions.
Critical strengths
- Clear mechanistic insight into action chunking under anomalies
- Good writing
- Comprehensive evaluation across backbones
Critical weaknesses
-
Assuming severe observation corruption at strictly a single isolated inference step is detached from real-world physical control. In practice, hardware glitches, illumination shifts (e.g., auto-exposure recovery), motion blur, or communication drops rarely occur as an isolated single-step impulse in the middle of a trajectory and immediately vanish. As shown in Appendix D.4, when perturbations realistically persist for just 5–10 steps, CARE's effectiveness degrades substantially.
-
Even under the idealized one-step threat model, CARE sets the minimum execution length to , meaning the robot still executes the first corrupted action before replanning. In real-world control, a single erroneous action (e.g., premature gripper opening under blur in Section 6.4) is sufficient to cause irreversible task failure.
-
Accepting the corrupted chunk pollutes the reference buffer for step , causing a cascading discrepancy even after clean observations are restored.
Questions for authors
-
Given that real-world sensor corruptions (e.g., camera overexposure, packet loss bursts) typically span multiple consecutive frames rather than a single step, how can CARE be adapted to handle realistic, temporally correlated disturbances where replanning at still encounters corrupted inputs?
-
CARE causes a notable 11.6% drop in clean success rate on -FAST (Table 2). Does discrete autoregressive token sampling inherently induce higher inter-chunk variance that falsely triggers short execution horizons under clean observations?
-
When a severe discrepancy is detected at step (), why does CARE execute the corrupted action () instead of falling back to the unexecuted clean actions from the previous chunk?
7 comments
Q2: Sensitivity of -FAST to Execution Length
Thank you for raising this interesting question. We suspect that the clean-performance drop primarily reflects -FAST’s sensitivity to execution length, rather than autoregressive sampling falsely triggering short horizons. Figure 4 shows that its performance drops substantially even with fixed execution lengths outside a favorable range. This suggests that varying the execution length is itself challenging for this model, which may explain the degradation under CARE. Autoregressive token sampling could contribute to this sensitivity, but our experiments do not isolate its role.
We thank Reviewer UeVo again. We also appreciate their recognition of the mechanistic insight into action chunking and the comprehensive evaluation. We hope our responses clarify the paper’s contributions to robustness evaluation. We believe the finding that even a single corrupted observation can cause lasting failure offers a useful perspective on VLA robustness.
Gentle Reminder
Dear Reviewer UeVo,
Thank you again for your valuable comments and feedback on our submission. With two days remaining in the discussion period, we would like to kindly follow up and ask whether our rebuttal has sufficiently addressed your concerns. Should any questions or concerns remain, we would be grateful for the opportunity to provide further clarification.
Thank you very much for your time and consideration.
We thank Reviewer UeVo for the thoughtful and constructive feedback. Your comments have helped us sharpen the scope of our robustness claims and clarify CARE’s limitations and the interpretation of our results.
W1-1: A single, isolated, severe observation corruption
Thank you for your suggestion. We agree that real-world observation corruptions can persist across multiple steps. However, their duration in steps depends on the disturbance and the sensing and control frequencies, making it difficult to specify a single representative duration. We therefore use one-step corruption, which allows us to focus on the timing of corruption without varying its duration, and to evaluate vulnerability at the temporal lower bound of attack duration.
This allows us to examine two important aspects of robustness. First, it enables temporal analysis, revealing how the impact of corruption depends on when it occurs, as shown in Figure 3. Second, it shows that even the shortest attack can cause serious harm. A single corrupted input can lead to task failure despite subsequent observations being clean, demonstrating that sustained attacks are not necessary to expose this vulnerability.
Our protocol is therefore designed to isolate key aspects of robustness, rather than to fully reproduce real-world corruptions. We will clarify this motivation in the revision.
W1-2 & W2; Q1 & Q3: Limitations of CARE
We thank the reviewer for these helpful comments on CARE’s limitations.
W1-2 & Q1: Robustness to multi-step corruption. We agree that CARE has limited effectiveness against multi-step corruption. Research on VLA robustness to one-step observation perturbations is still at an early stage, and our work provides a controlled analysis of this vulnerability and a simple mitigation method. Although CARE’s effectiveness decreases under 5- and 10-step perturbations, it still provides greater robustness than Vanilla in most evaluated settings (Appendix D.4).
W2 & Q3: Irreversible failures caused by the first action. We agree that CARE cannot always prevent irreversible failures caused by the first action. We view this limitation as a trade-off for CARE’s simple design, which requires no additional training or inference for execution length selection. By relying solely on discrepancies between existing action chunks, CARE can limit prolonged execution of potentially erroneous actions, but cannot determine which prediction is correct or whether the first action is safe. Setting Ht=1 allows replanning at the next step, but does not guarantee that the action executed before replanning is safe.
The reviewer’s suggestion to fall back to the unexecuted portion A^ of the previous chunk when r0>τ is a useful extension that could address some of these failures. However, a large discrepancy between chunks can also arise when an actual change in the environment requires an updated action plan. In such cases, falling back to the previous prediction could prevent a necessary update. Because discrepancy alone cannot distinguish these situations, always falling back to the previous prediction could improve robustness to corruption at the cost of clean task performance.
A possible extension is to combine CARE with an observation-reliability mechanism that determines when to retain a previous plan and when to accept a newly predicted chunk. CARE requires only minor changes to the execution procedures of action-chunking VLA models, making integration with such complementary mechanisms a promising direction for future work.
W3: Transient reference contamination and its empirical impact
Thank you for pointing this out. We agree that the corrupted chunk At becomes the reference for the next prediction, so the discrepancy can remain elevated even after clean observations are restored. We observed that CARE selects an execution length of 1 at the corrupted step and again at the following prediction, after which normal execution lengths resume. Although the reference contamination temporarily affects execution-length selection, it does not prevent CARE from preserving task performance in our evaluations as shown in Table 2.