- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a model-update-free method for recovering from potential failures in the OpenVLA baseline. ReAlign-VLA monitors the model’s internal states at runtime and applies deviation-conditioned interventions when abnormal behavior is detected, improving OpenVLA’s task success rate on the LIBERO benchmark.
Critical strengths
-
Failure reasoning and recovery for VLA models is a timely and important research problem.
-
The deviation-gated, tiered recovery strategy is relatively novel and could inspire the development of more comprehensive failure-recovery pipelines.
Critical weaknesses
-
The authors are encouraged to improve the paper’s presentation and writing to communicate the main arguments more clearly. For example, Figure 1 appears to be AI-generated and is not presented as a vector graphic, which reduces its visual quality and professionalism.
-
Figure 3 shows substantial overlap between successful and failed cases when the deviation score is below 0.8. It is unclear how ReAlign-VLA handles failures that fall within this low-deviation region. More importantly, does this overlap suggest that the deviation score itself may not be sufficiently reliable for identifying potential failures?
-
The evaluation is limited to a single benchmark (LIBERO) and a single base VLA model (OpenVLA). This limited experimental scope is insufficient to support strong conclusions about the general effectiveness and generalizability of ReAlign-VLA.
Questions for authors
-
Does the substantial overlap in Figure 3 suggest that the deviation score may not be sufficiently reliable for identifying potential failures? In particular, how does ReAlign-VLA handle failure cases that fall within the low-deviation region?
-
Does the proposed pipeline risk overfitting to a narrow combination of tasks, models, and environments? Can ReAlign-VLA generalize across different benchmarks and VLA backbones? If the VLA is deployed in an environment that differs from the one used during profile construction, does the effectiveness of the pipeline degrade?
ReAlign-VLA: Response to Reviewer vzWi
Thank you for highlighting the presentation and generalization issues. We replaced the mechanism figure with an audited implementation diagram and verified its rendering in the ICLR-formatted PDF. The revised method text identifies simulator geometry and distinguishes within-attempt safeguards from the fixed retry schedule.
1. Score overlap and handling low-deviation failures. The score quantifies deviation from the successful-execution profile and is used as one signal in the recovery controller, rather than as a calibrated failure probability. We now report both sides of the overlap. In the Spatial shadow audit, 39/48 failed episodes crossed at least once, while 21/152 successful episodes also crossed; nine failures remained below threshold. In the separate LIBERO-10 validation audit, crossings occurred in 46/47 failures and 44/53 successes (83.0% false-positive rate). The protocols are reported separately because they use different populations and exposure. For a low-deviation failure, the runner continues the current attempt until success, abort, or horizon; after unsuccessful completion it resets and follows the next fixed retry. The later retry is an execution opportunity, not retrospective evidence that the monitor detected the earlier failure.
2. Generalization across tasks, backbones, and deployment conditions. The revision adds two complementary pieces of simulator evidence while keeping their scope distinct. First, the new LIBERO-10 configuration study evaluates 200 randomized initial configurations in the same ten tasks: enabled and disabled joint safeguards achieved 148/200 and 145/200 successes (exact McNemar p=0.6072). This broadens initial-state coverage within LIBERO; its 60/140 source-version split is documented, and the paired estimate is reported with uncertainty. Second, a separate ManiSkill PutCarrotOnPlateInScene-v1 study uses a privileged-geometry behavior-cloning controller and contact-informed release logic. On 200 paired states from previously used layout families, task success and continuous stable release each increased from 191/200 to 200/200 (9 rescues, 0 damages; exact McNemar p=0.00390625), with mean policy steps of 58.705 versus 65.445. This is evidence for that task-specific simulator controller, not a ReAlign-on-OpenVLA transfer result or unseen-family test. Across the VLA experiments, four suites are all within LIBERO, repeated seeds reuse state pools, and Octo baseline results are kept separate from ReAlign transfer. The paper therefore states the supported scope and identifies distribution-shift, cross-backbone, unseen-family, and physical deployment evaluations as remaining directions.