acceptodds
Under review as a conference paper at ICLR 2027

ReAlign-VLA: Deviation-Guided Recovery for Off-the-Shelf Vision-Language-Action Models

Abstract

Vision-Language-Action models have shown strong potential for language-conditioned robotic manipulation, but reliable deployment requires the ability to recover from execution failures, not merely to generate actions. In practical manipulation, routine visual uncertainty, contact errors, and action drift can turn small deviations into task failure, while recovery strategies based on VLA fine-tuning or auxiliary models limit scalability in open-ended and changing environments. We propose ReAlign-VLA, a model-update-free recovery framework that couples internal-state runtime monitoring with deviation-conditioned intervention. Rather than training a separate failure detector or invoking an external VLM planner, ReAlign-VLA constructs a progress-conditioned profile of successful executions from the frozen VLA's hidden states. During deployment, it extracts hidden states through a lightweight forward hook and computes a sliding-window normalized deviation score to identify when the current execution departs from successful behavior. The deviation score then gates a tiered recovery mechanism, escalating from action-level correction for physical execution errors to deterministic sub-goal refinement for semantic or planning ambiguity. Progress-aware locking and early abort make the intervention conservative, preserving near-successful trajectories while discarding attempts that remain far from the success profile. Experiments on four LIBERO benchmark suites show that ReAlign-VLA consistently improves an unmodified OpenVLA baseline in both task success and execution efficiency.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action models, robotic manipulation, failure detection, failure recovery, runtime monitoring, representation learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vzWiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a model-update-free method for recovering from potential failures in the OpenVLA baseline. ReAlign-VLA monitors the model’s internal states at runtime and applies deviation-conditioned interventions when abnormal behavior is detected, improving OpenVLA’s task success rate on the LIBERO benchmark.

Critical strengths
  1. Failure reasoning and recovery for VLA models is a timely and important research problem.

  2. The deviation-gated, tiered recovery strategy is relatively novel and could inspire the development of more comprehensive failure-recovery pipelines.

Critical weaknesses
  1. The authors are encouraged to improve the paper’s presentation and writing to communicate the main arguments more clearly. For example, Figure 1 appears to be AI-generated and is not presented as a vector graphic, which reduces its visual quality and professionalism.

  2. Figure 3 shows substantial overlap between successful and failed cases when the deviation score is below 0.8. It is unclear how ReAlign-VLA handles failures that fall within this low-deviation region. More importantly, does this overlap suggest that the deviation score itself may not be sufficiently reliable for identifying potential failures?

  3. The evaluation is limited to a single benchmark (LIBERO) and a single base VLA model (OpenVLA). This limited experimental scope is insufficient to support strong conclusions about the general effectiveness and generalizability of ReAlign-VLA.

Questions for authors
  1. Does the substantial overlap in Figure 3 suggest that the deviation score may not be sufficiently reliable for identifying potential failures? In particular, how does ReAlign-VLA handle failure cases that fall within the low-deviation region?

  2. Does the proposed pipeline risk overfitting to a narrow combination of tasks, models, and environments? Can ReAlign-VLA generalize across different benchmarks and VLA backbones? If the VLA is deployed in an environment that differs from the one used during profile construction, does the effectiveness of the pipeline degrade?

1 comment
AuthorsOpenReview ↗

ReAlign-VLA: Response to Reviewer vzWi

Thank you for highlighting the presentation and generalization issues. We replaced the mechanism figure with an audited implementation diagram and verified its rendering in the ICLR-formatted PDF. The revised method text identifies simulator geometry and distinguishes within-attempt safeguards from the fixed retry schedule.

1. Score overlap and handling low-deviation failures. The score quantifies deviation from the successful-execution profile and is used as one signal in the recovery controller, rather than as a calibrated failure probability. We now report both sides of the overlap. In the Spatial shadow audit, 39/48 failed episodes crossed at least once, while 21/152 successful episodes also crossed; nine failures remained below threshold. In the separate LIBERO-10 validation audit, crossings occurred in 46/47 failures and 44/53 successes (83.0% false-positive rate). The protocols are reported separately because they use different populations and exposure. For a low-deviation failure, the runner continues the current attempt until success, abort, or horizon; after unsuccessful completion it resets and follows the next fixed retry. The later retry is an execution opportunity, not retrospective evidence that the monitor detected the earlier failure.

2. Generalization across tasks, backbones, and deployment conditions. The revision adds two complementary pieces of simulator evidence while keeping their scope distinct. First, the new LIBERO-10 configuration study evaluates 200 randomized initial configurations in the same ten tasks: enabled and disabled joint safeguards achieved 148/200 and 145/200 successes (exact McNemar p=0.6072p=0.6072). This broadens initial-state coverage within LIBERO; its 60/140 source-version split is documented, and the paired estimate is reported with uncertainty. Second, a separate ManiSkill PutCarrotOnPlateInScene-v1 study uses a privileged-geometry behavior-cloning controller and contact-informed release logic. On 200 paired states from previously used layout families, task success and continuous stable release each increased from 191/200 to 200/200 (9 rescues, 0 damages; exact McNemar p=0.00390625p=0.00390625), with mean policy steps of 58.705 versus 65.445. This is evidence for that task-specific simulator controller, not a ReAlign-on-OpenVLA transfer result or unseen-family test. Across the VLA experiments, four suites are all within LIBERO, repeated seeds reuse state pools, and Octo baseline results are kept separate from ReAlign transfer. The paper therefore states the supported scope and identifies distribution-shift, cross-backbone, unseen-family, and physical deployment evaluations as remaining directions.

Reviewer_6YZ5OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ReAlign-VLA, which extracts hidden-state and action statistical features from successful trajectories of a frozen OpenVLA to establish successful reference distributions according to execution progress. During deployment, the method triggers graded recovery through normalized deviation scores within a sliding window, including original instruction retry, action perturbation, and subgoal decomposition based on predefined templates, while using progress locking and early termination to constrain interventions. The authors evaluate task success rates, execution steps, deviation signals, and the role of recovery components on four LIBERO suites.

Critical strengths

This paper focuses on runtime failure perception and recovery, which is of practical significance. Utilizing the model's existing internal representations to build statistical references, avoiding additional training of detectors or introducing external inference models, is a concise and worthwhile direction to explore. The method connects anomaly monitoring with actual intervention and attempts to reduce interference with normal execution through progress-dependent constraints. Relative to the same OpenVLA baseline, success rate improvements are reported across all four suites, providing preliminary support for the effectiveness of the overall recovery framework.

Critical weaknesses

The primary issue is that the contribution of the core monitoring mechanism has not been fully isolated. In Table 3, random retries yield higher improvements than the full method on Spatial and Goal, while currently lacking complete comparisons matching action and retry budgets, as well as ablations such as hidden-only, action-only, and no progress bucketing, making it difficult to confirm whether the gains come from internal state deviation monitoring.

Secondly, the estimation method for online execution progress, the information source for target distance, and the specific switching and termination processes for each level of recovery are inadequately explained. The detection analysis mainly shows statistical differences between successful and failed distributions, but the step-wise detection rate under the selected threshold is only 12.5%, and trajectory-level detection and intervention results are still needed to illustrate its practical role.

Finally, the reduction in total execution steps may be affected by early termination of failed attempts, and cannot yet directly support the conclusion of improved execution efficiency for successful tasks.

Questions for authors

See above.

3 comments
AuthorsOpenReview ↗

ReAlign-VLA: Response to Reviewer 6YZ5

Thank you for identifying that the earlier experiments did not isolate the monitor's contribution. We now distinguish the tested retry bundle, score safeguards, and shadow detection analyses.

1. Budget matching and the monitor's contribution. On the reused LIBERO-10 state pool, staged retry and ordinary retry used the same maximum action budget and obtained 147/200 and 127/200 successes, respectively (exact McNemar p=0.00453p=0.00453). This comparison estimates the full retry-bundle contrast. To assess the joint score safeguards, the fixed schedule without lock/abort effects obtained 152/200, compared with 147/200 with safeguards (9 versus 14 discordant outcomes; p=0.405p=0.405). In a new-configuration study, enabled and disabled safeguards obtained 148/200 and 145/200 successes (9 versus 6 discordant successes; p=0.6072p=0.6072; Wilson intervals [67.5,79.6]% and [65.9,78.2]%). The 1.5-point marginal difference is small relative to its uncertainty; we therefore report the estimate without attributing the retry-bundle gain to the safeguards. The new study preserves and separately documents 60 original and 140 amended pairs.

The five-arm feature analysis is a post-hoc comparison on the earlier state pool. “Hidden36” removes geometry from the score while retaining the shared geometric progress guard; “action/geometry7” combines action and geometry inputs. We also report a retrospective action-summary control using XYZ action magnitude and gripper command on 100 previously viewed validation episodes. Its AUROC was 0.3525, compared with 0.4179 for Full43; the paired Full43-minus-action-summary difference was 0.0654 (descriptive within-task bootstrap 95% interval [-0.0533, 0.1891]). This adds a geometry-free action-summary reference point, while remaining distinct from a raw-vector online ablation. The analyses are presented as signal characterization rather than a causal ranking of feature groups.

2. Progress signal, geometry, and strategy switching. The profile uses task-wise statistics from successful trajectories, with at most the first 200 feature vectors per trajectory; the implementation uses no progress bins. The revision now describes the actual 43-D input: 36 hidden-state summaries plus seven action, progress, and simulator-geometry features. Nearest-object distance and alignment are simulator-provided. Within an attempt, score and geometric progress regulate action-noise locking and early abort. After failure or abort, the environment resets and the runner follows a fixed S0/S1/S2 order. This distinction clarifies that the threshold regulates safeguards rather than selecting the next strategy.

3. Trajectory-level detection and execution efficiency. The 12.5% figure is the fraction of failed-episode timesteps crossing the threshold, not episode recall. We add episode-level shadow analyses: at threshold 0.8, Spatial crossings occurred in 39/48 failed and 21/152 successful episodes; a separate LIBERO-10 validation audit crossed in 46/47 failed and 44/53 successful episodes. We report the two populations separately and include missed failures and false positives; threshold crossings are not physical failure-onset labels. We also added a matched-image component timing audit using 144 triples (432 calls): median policy, hook, and monitored-chain times were 286.18, 286.91, and 287.39 ms, with a 1.538 ms median paired monitored-minus-policy difference. The measured chain excludes recovery selection/execution, reset, and robot communication, so it is a component-cost estimate rather than end-to-end latency. Action counts are likewise reported as simulator steps, not wall-clock completion time.

Thank you for the additional experiments. However, my core concern about the independent benefit of the monitoring mechanism remains unresolved. I will therefore keep my original score.

Thank you for emphasizing that the monitor's incremental contribution must be separated from the effect of retrying.

  1. Under the same 1,560-action maximum on the previously evaluated LIBERO-10 state pool, profile-staged retry obtained 147/200 successes versus 127/200 for random retry (33 versus 13 discordant pairs; exact two-sided McNemar p=0.00453p=0.00453). This supports the tested retry bundle on that pool, not a causal benefit from its monitor. The no-safeguard schedule obtained 152/200 versus 147/200 with safeguards (9 versus 14 discordances; p=0.405p=0.405). On 200 new initial configurations of the same ten tasks, enabled and disabled safeguards obtained 148/200 and 145/200 (9 versus 6 discordances; p=0.6072p=0.6072). The new-configuration point estimate is 1.5 percentage points in favor of safeguards, but the paired evidence is inconclusive.

  2. Results are task-dependent: on Spatial, staged retry obtained 179/200 versus 183/200 for random retry (p=0.344p=0.344). The five-arm feature-group analysis reused the earlier state pool: full43 obtained 147/200, hidden36 142/200, action/geometry7 149/200, and no score safeguards 152/200. Hidden36 removes geometry from the score but retains the shared geometric progress guard; action/geometry7 is not a pure action-only system. These post-hoc comparisons do not establish a causal ranking of feature groups.

  3. The profile is task-wise and uses no progress bins. Its 43-D input combines 36 hidden-state summaries with seven action, progress, and simulator-geometry features. Score/progress checks regulate within-attempt noise locking and early abort. After failure or abort, the runner resets and advances through a fixed S0/S1/S2 order; the score does not select the next strategy.

  4. At threshold 0.8, the Spatial shadow audit crossed in 39/48 failed episodes and 21/152 successful episodes. A separate LIBERO-10 validation audit crossed in 46/47 failures and 44/53 successes. These are distinct episode-level shadow analyses, not calibrated detection or failure-onset localization. Mean total simulator actions on LIBERO-10 were 764.8 for staged retry and 859.9 for random retry, but these counts do not establish wall-clock speed or lower cost among successful episodes. The matched-image component benchmark measured median policy, hook, and monitored-chain latency of 286.18, 286.91, and 287.39 ms, respectively; the monitored chain excludes retry execution, reset, and robot communication.

Reviewer_9VoPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ReAlign-VLA, a runtime recovery framework that does not update VLA parameters nor relies on additional VLM or learning-based failure detectors. The method first utilizes the hidden state and action features of the frozen OpenVLA in the successful trajectory to construct a success profile based on the execution progress; during deployment, the deviation score of the current state relative to the success profile is calculated using a sliding window, and hierarchical recovery is triggered when the deviation exceeds the threshold: re-executing the original action, adding small action perturbations, or rewriting the instructions using a deterministic sub-goal template. Experiments were conducted on four LIBERO suites, acquiring large improvement compared to the OpenVLA baseline, while the execution step cost was less than 1.0×.

Critical strengths
  1. Success profiling, sliding window deviation detection, tiered recovery, and progress-aware safeguards form a relatively complete runtime loop. Particularly, the upgrade from action-level retry to semantic-level refinement reflects the design principle of "as much conservative intervention as possible".
  2. The method only requires offline collection of success trajectories and establishment of statistical profiles. It reads the hidden state through forward hooks without the need for re-training VLA or deploying additional inference models. No intervention at every step is required, thereby reducing the frequency of intervention.
  3. On LIBERO suites, ReAlign-VLA has improved compared to the same OpenVLA-7B baseline. The improvement is very obvious. Compared to simply citing the results from existing papers, the matched baseline comparison is more convincing.
Critical weaknesses
  1. Semantic recovery relies on keyword matching and predefined manipulation templates, and is mainly applicable to limited task types such as pick-and-place. For complex combined instructions, objects with the same name, multi-step condition dependencies, or undefined skills, this decomposition is likely to fail. Therefore, the paper's applicability to open-ended and changing environments currently lacks experimental support.

  2. In Table 3, Random retry shows a higher improvement on Spatial compared to the full method; No early abort also shows a higher improvement. This indicates that many benefits may come from repeated execution or a longer trial budget, rather than necessarily from deviation-guided tiered recovery. Moreover, the ablation experiments show that different improvements may be too task-type-specific.

  3. The experiments are insufficient to support the generalization of the method. This paper only uses one base method and it is uncertain whether it can be generalized to other methods. It also cannot prove that the significant improvement is due to the poor performance of the base method itself.

  4. There seems to be a problem with the citation format of the paper.

Questions for authors

please refer to Weakness.

3 comments
AuthorsOpenReview ↗

ReAlign-VLA: Response to Reviewer 9VoP

Thank you for the detailed review. We revised the method description and added controlled analyses to clarify what the evaluated retry procedure contributes, how S2 operates, and which conclusions are supported by the available comparisons.

1. Complex instructions and semantic recovery. In the evaluated implementation, S2 prepends a fixed instruction prefix to the original command. The revision describes this operation precisely as a language-level prompt intervention and distinguishes it from task-specific instruction parsing or plan generation (main paper, Method; supplement, “Fixed S2 Instruction Prefix”). The current evaluation focuses on the LIBERO tasks and does not test open-ended compositional instructions; we have made this scope explicit.

2. Retry budget and task dependence. We added equal-maximum-budget comparisons to distinguish the staged retry bundle from ordinary retry. On the previously evaluated LIBERO-10 state pool, both methods used a 520-action per-attempt horizon and up to three attempts; staged retry obtained 147/200 successes versus 127/200 for ordinary retry (33 versus 13 discordant successes; exact two-sided McNemar p=0.00453p=0.00453). This is a paired follow-up on a reused state pool. On the separate Spatial audit, staged and ordinary retry obtained 179/200 and 183/200 successes, respectively (3 versus 7 discordances; p=0.344p=0.344), with mean total actions of 204.8 and 199.5. Together, these results show task-dependent performance and motivate separating retry opportunity from monitor effects. On the reused LIBERO-10 pool, the fixed schedule without joint score-dependent lock/abort safeguards obtained 152/200 versus 147/200 with safeguards (9 versus 14 discordances; p=0.405p=0.405); the paired estimate is inconclusive about the safeguards' incremental effect.

We then completed 200 paired new initial configurations from the same ten LIBERO-10 tasks. Enabled safeguards yielded 148/200 successes (74.0%; Wilson 95% CI [67.5,79.6]) versus 145/200 (72.5%; [65.9,78.2]) when disabled; the paired discordances were 9 versus 6 (exact p=0.6072p=0.6072). The estimated difference is 1.5 percentage points, with paired evidence that remains uncertain. The study includes 60 pairs from the original runner and 140 after a fixture-transform reset correction. We document the two source hashes and task-aligned strata separately so readers can assess the engineering amendment and the result transparently.

3. Generalization and baseline strength. We added a separate ManiSkill PutCarrotOnPlateInScene-v1 simulation study to provide an additional test of recovery behavior in a contact-rich task. A privileged-geometry behavior-cloning controller with a contact-informed release mechanism was evaluated on 200 paired new states from previously used layout families. Task success and continuous stable release were each 191/200 for the baseline and 200/200 for the candidate (9 rescues, 0 damages; exact McNemar p=0.00390625p=0.00390625). Mean policy steps were 58.705 and 65.445, respectively, within a 180-step horizon. This result supports the task-specific simulator controller under the tested setup; because it uses a separate controller and previously used layout families, we present it separately from ReAlign-on-OpenVLA and do not interpret it as unseen-family transfer. The OpenVLA feature-group analyses reuse state pools, and repeated execution seeds are not new configurations. Octo baseline and adapter results are also reported separately from ReAlign transfer. The revised scope statement identifies cross-backbone, unseen-task-family, and physical deployment as directions not evaluated here.

4. Citations. We reformatted the revision using the ICLR style and verified that the in-text citations resolve against the bibliography.

Thank you for highlighting the need to distinguish the retry bundle from the monitoring component, and to clarify the scope of the semantic intervention and generalization evidence.

  1. The evaluated S2 intervention prepends one fixed instruction prefix to the original command. It does not parse objects, resolve same-name references, generate conditional plans, or add skills. Our experiments do not test open-ended compositional instructions, so we interpret S2 only as a fixed prompt intervention on the evaluated LIBERO tasks.

  2. Under a shared 520-action attempt horizon and three-attempt maximum on the previously evaluated LIBERO-10 state pool, profile-staged retry obtained 147/200 successes versus 127/200 for random retry (33 versus 13 discordant pairs; exact two-sided McNemar p=0.00453p=0.00453). At the first 520-action budget, the counts were 98 versus 97, so the cumulative difference emerged over subsequent attempts. On Spatial, staged retry obtained 179/200 versus 183/200 for random retry (p=0.344p=0.344). On LIBERO-10, disabling score-dependent lock/abort effects within the same schedule yielded 152/200 versus 147/200 with safeguards (9 versus 14 discordances; p=0.405p=0.405). The retry-bundle result therefore does not establish an independent monitoring benefit.

  3. On 200 new initial configurations of the same ten LIBERO-10 tasks, enabled and disabled joint safeguards obtained 148/200 and 145/200 successes (9 versus 6 discordances; exact p=0.6072p=0.6072). This adds initial-state variation within LIBERO but does not test unseen tasks. The separate ManiSkill controller study uses a privileged-geometry behavior-cloning controller, not ReAlign transfer. Published CoT-VLA/ThinkAct values are contextual comparisons, not matched re-evaluations. We do not claim cross-backbone, unseen-task, or physical-robot generalization.

  4. The four-suite archived aggregates report higher success than the corresponding one-shot OpenVLA baselines, but execution seeds reuse suite state pools. We therefore present them as archived outcomes rather than independent new-state replications. The matched retry comparisons above provide the more controlled evidence about the retry bundle.

  5. We acknowledge the citation-format concern and will check the final submission PDF against the ICLR style requirements.

Reviewer_9VoPOpenReview ↗

Response to reviewer

Thank the authors for the comprehensive rebuttal, which resolves several of my initial questions. Nonetheless, my primary concerns regarding real-world deployment feasibility and the novelty of the proposed method remain. Taking everything into account, I will maintain my original rating, and I lean slightly negative without strongly opposing acceptance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.