- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates the robustness of VLA models to joint-level physical faults, including joint locks, range restrictions, and increased friction, which introduce mismatches between commanded and executed actions. The authors systematically evaluate two VLA backbones on LIBERO and analyze the impact of different fault conditions. They further examine language-based feedback and kinematic correction, finding limited benefits, and propose an embodiment-side action calibration method that partially recovers task performance while largely preserving fault-free behavior. Real-world experiments further validate the potential of the proposed calibration approach.
Critical strengths
- The paper investigates an underexplored dimension of VLA robustness: physical execution faults at the robot-joint level, complementing existing studies on visual, task-level, and action-space perturbations. Overall, it is a novel and practically relevant problem formulation.
- The authors evaluate two VLA backbones across four LIBERO suites, considering multiple fault types, severities, and affected joints. The analysis reveals substantial joint-dependent performance degradation and demonstrates that increased friction can impair task success even without restricting joint-position limits.
- The authors also explore language-based fault feedback, constrained inverse kinematics, and embodiment-side action calibration. These experiments provide useful insights into the limitations of directly communicating execution mismatches or applying local kinematic corrections, motivating alternative approaches to embodiment-aware adaptation.
- The proposed calibration method partially recovers task performance while keeping the VLA backbone frozen and largely preserving fault-free performance. The authors also validate the approach on a real-world robot, providing preliminary evidence of its practical applicability.
Critical weaknesses
- I am not entirely clear about how the physical perturbations are implemented in the experimental setup. Since a large portion of the experiments is conducted in simulation, could the authors clarify how the proposed joint-level physical perturbations are modeled and deployed in the simulated environment? In particular, how is increased friction implemented in the simulator, and what does its effect look like in practice? I encourage the authors to provide additional visualization examples.
- I am interested in understanding how these embodiment-site faults affect task execution over longer-horizon tasks. Do tasks fail from the very beginning due to the perturbations, or do errors gradually accumulate over time, eventually leading to task failure?
- The evaluation primarily focuses on individual joint-level faults, leaving the robustness of VLAs under simultaneous faults in multiple joints unclear. In real-world scenarios, mechanical degradation or actuator failures may affect multiple joints at the same time, and the resulting execution errors may interact. It would be valuable to evaluate the effectiveness of the proposed calibration method under multi-joint faults.
Questions for authors
See the weaknesses above.
5 comments
Response to D3LH: Fault Accumulation Test
We test whether the VLA can recover after the fault is removed. We vary how long the fault remains active, then remove it and let the VLA continue with the robot in a fault-free condition.
Task success generally declines with longer fault exposure, even after the fault is removed. Success remains high after brief exposure and tends to fall as the robot spends more time operating under the fault.
Limited recovery from accumulated execution errors motivates reference-guided calibration. At fault removal, the VLA must continue from the state the robot reached during faulty execution. The decline in success suggests that the VLA struggles to recover from those accumulated deviations on its own. Reference-guided calibration learns to correct the deviations as they arise.
Section 4.2 of the main paper presents the plot and describes the experiment in detail.
Response to D3LH: Multi-Joint Fault Evaluation
Thank you for suggestion of multi-joint fault evaluation. We extended the evaluation to simultaneous faults in multiple joints using the existing embodiment-side calibrators. We tested both multiple joint locks and a joint lock combined with increased friction in another joint.
Simultaneous joint locks. Embodiment-side calibration achieves higher success rates in all 12 conditions, with mean gains of 7.48, 9.20, and 5.82 percentage points for the three configurations.
Each entry reports Base → embodiment-side calibration success rate (%).
| Locked joints | Spatial | Object | Goal | LIBERO-10 | Mean |
|---|---|---|---|---|---|
| , | 36.7 → 58.2 | 21.3 → 22.4 | 66.7 → 69.0 | 2.0 → 7.0 | 31.67 → 39.15 |
| , | 38.7 → 52.2 | 22.0 → 31.4 | 48.7 → 52.0 | 4.7 → 15.2 | 28.50 → 37.70 |
| ,, | 75.3 → 84.4 | 78.0 → 83.4 | 60.0 → 60.4 | 24.0 → 32.4 | 59.33 → 65.15 |
Simultaneous faults of different types. Embodiment-side calibration improves success in seven of the eight mixed-fault conditions and matches Base in the remaining condition. In each configuration, we locked one joint and increased friction in a different joint by a factor of 150.
| Fault combination | Spatial | Object | Goal | LIBERO-10 | Mean |
|---|---|---|---|---|---|
| Locked + increased friction in | 14.4 → 39.0 | 76.0 → 81.0 | 77.2 → 77.2 | 45.0 → 50.0 | 53.15 → 61.80 |
| Locked + increased friction in | 30.8 → 50.4 | 18.6 → 28.2 | 48.8 → 54.4 | 5.4 → 13.4 | 25.90 → 36.60 |
Recovery varies across configurations, and several LIBERO-10 conditions remain challenging. We added these evaluations to Appendix F and report the results in Tables 5 and 6.
Response to D3LH: Implementation Detail about Physical Perturbation and Visualization on Friction Fault
Thank you for requesting clarification.
We evaluate a 7-DoF Franka Panda arm in LIBERO, which uses robosuite and the MuJoCo physics engine. VLA-predicted end-effector commands are executed through an operational-space pose controller (
OSC_POSE) at 20 Hz, with replanning every eight steps.Each fault is applied to one arm joint after the environment is reset to the selected benchmark initial state and remains active throughout the episode. We evaluate each arm joint separately. The fault is implemented by changing the selected joint’s simulator parameters.
Joint locks and range restrictions narrow the joint’s position limits around its initial position. Let qi0 denote the initial position of joint i, and let qi,min and qi,max denote its original limits. The imposed interval is
[max(qi,min,qi0−hi),min(qi,max,qi0+hi)].
For a joint lock, we set hi=10−4rad. This small nonzero interval effectively immobilizes the joint while avoiding a zero-width constraint.
For a range restriction, we set
\qquad r \in \{0.05, 0.03\}.$$ The ratio $r$ specifies the retained fraction of the original joint span before clipping to the original limits. **We model increased friction by scaling the selected joint’s MuJoCo `frictionloss` parameter.** $$f_i^{\mathrm{fault}} = \gamma f_i^{\mathrm{nominal}}, \qquad \gamma \in \{150, 200, 300\}.$$ The nominal value is 0.1, giving perturbed values of 15, 20, and 30, respectively. This parameter governs dry friction at the joint. Joint damping, actuator force limits, joint position limits, and contact friction between objects retain their nominal settings. Increased resistance can reduce joint displacement, cause trajectory deviations, or stall motion despite continued end-effector commands. The resulting changes in the robot’s motion affect subsequent observations and VLA predictions. **The supplementary visualization shows how increased friction affects different motions depending on the joint.** The examples exhibit patterns similar to those observed under joint locks. In the illustrated task, friction at $j_2$ and $j_3$ impedes lateral and vertical motion, respectively, while friction at $j_4$ and $j_5$ interferes with gripper alignment and grasping. Figure 6 in the appendix presents these results, and we added a detailed explanation of the experiment to Appendix E.