acceptodds
Under review as a conference paper at ICLR 2027

EM-VLA: Closing the Loop Between Execution Monitoring and Action Generation for Long-Horizon Manipulation

Abstract

Vision-language-action (VLA) models have demonstrated strong capabilities in robotic manipulation, yet reliable long-horizon execution remains challenging. Existing approaches introduce high-level reasoning or execution monitoring, but execution feedback is often used primarily to trigger intervention rather than continuously inform both reasoning and control. We propose EM-VLA, a hierarchical VLA that coordinates high-level reasoning and low-level control through a persistent state maintained by a lightweight execution monitor. The monitor integrates observed transitions, visual-language context, and internal action-model responses to update this state across interactions. The state guides subtask transitions and replanning while adapting action generation to the current execution stage. Internal responses from action generation then inform subsequent state updates, closing the loop between monitoring and control. This design enables the VLA to revise its plan and behavior as execution conditions change without requiring high-level reasoning at every action query. To support training, we derive temporally aligned subtask plans and supervision for reasoning and monitoring from successful demonstrations. Evaluations on LIBERO, CALVIN, RoboTwin 2.0, and real-world robotic tasks show improved long-horizon performance over the compared baselines and stronger recovery from execution-time perturbations.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Robot Learning, Long-Horizon Manipulation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Moo6OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses the difficulty of inferring execution state from a single observation in long-horizon manipulation, and proposes EM-VLA, a hierarchical VLA built on the π0/π0.5 architecture. A lightweight Transformer execution monitor maintains a fixed-size recurrent memory with a dedicated state token h_t, and decodes three signals (DONE, MATCH, PROGRESS) that drive subtask transitions and replanning, so that high-level reasoning need not be invoked at every action query. In addition, h_t controls bottleneck adapters in each action-expert layer via top-2 sparse routing, enabling stage-dependent action generation. Routing-weight statistics and weighted residual norms from the action expert are fed back to the monitor as a feedback vector f_t, forming a “monitoring–control” loop. Supervision comes from a two-pass VLM annotation of successful demonstrations (subtask list plus keyframe alignment) and counterfactual negatives for the MATCH head. Experiments cover LIBERO (97.6% avg.), CALVIN ABC→D (avg. length 4.35), RoboTwin 2.0 (91.6% Clean / 90.9% Randomized), and a real-world Piper robot (80% on standard tasks, 62.5% under execution perturbation).

Critical strengths
  1. Clear motivation and a worthwhile problem. Elevating execution feedback from a trigger or warning to a persistent conditioning signal that serves both high-level planning and low-level control is a fresh perspective that matches a real pain point in long-horizon manipulation.
  2. Complete system with low overhead. The monitor costs 2.1 ms and the adapters 0.56 ms, negligible against VLM prefix computation (37.4 ms) and action decoding (23.8 ms). High-level reasoning (~389 ms) is invoked only on demand, which is pragmatic.
  3. Broad evaluation. Three simulation benchmarks plus real-robot experiments, including a dedicated execution-perturbation protocol (mid-task interference and undoing completed steps) that probes recovery more directly than standard success rates.
  4. Reusable supervision pipeline. Subtask segments, progress labels, and counterfactual match samples are constructed automatically from successful demonstrations without failure data. The appendix provides full VLM prompts and training details, which aids transparency.
Critical weaknesses
  1. The core “closed-loop feedback” claim lacks sufficient evidence (most critical) The feedback vector f_t consists of routing-weight statistics and weighted residual norms. However, the routing weights g_t = softmax(R(h_t)) are a deterministic function of h_t and are directly supervised by the progress label (Eqs. 9, 11). The routing part of f_t therefore carries little information beyond h_t and is largely an echo of a progress estimate; only the residual norms (which depend on x_t) bring new information. The paper neither analyzes this redundancy nor ablates the components of f_t. The “decoupled execution monitoring” variant is underspecified. Does it retain adapters and routing, with only f_t removed? If so, the 4.24→4.35 gap can be attributed to feedback. If routing was also removed, the contributions of state-conditioned action generation and action-to-monitor feedback cannot be separated. There is no comparison with simpler alternatives, e.g., feeding h_t or the predicted progress p̂_t directly to the action expert as a token or FiLM conditioning, without MoE-style adapter routing. Without this, the necessity of the routing–feedback machinery is not established.

  2. The adapter routing analysis holds by construction Figure 3 shows routing shifting from Adapters 0/1 to 2/3 with progress, but this is exactly the target of L_stage and does not show that such stage specialization improves action quality. Causal evidence is needed: e.g., shuffling or fixing the routing, disabling routing, or varying K and top-k.

  3. Insufficient evidence for replanning and recovery mechanisms The paper stresses that high-level reasoning is not needed at every step and that replanning is triggered by persistently low MATCH, yet it does not report how often replanning is actually triggered in each benchmark or real-world experiment, whether triggers are correct, or the effect of false/missed triggers. The window length defining “persistently” low/high, and the values and sensitivity of the thresholds τ_m, τ_p, τ_d, are not given in the main text. Appendix Table 6 also omits λ_m, λ_p, and W. There is no baseline that invokes high-level reasoning at a fixed frequency without a monitor (Hi Robot / OneTwoVLA style), so the implicit claim that monitor-driven on-demand reasoning beats periodic reasoning is never directly tested. The execution-perturbation result (a 30-point lead) is a headline claim but is supported by a single qualitative case (Fig. 6), in which a completed step is undone after task completion. This corresponds to DONE dropping and triggering a new plan, a relatively easy case. No qualitative or quantitative analysis is provided for mid-task interference (e.g., monitor signals over time). Moreover, the baselines (π0, StarVLA, OpenVLA) have neither subtask plans nor completion estimation, so the comparison is not like-for-like.

Questions for authors
  1. The routing statistics in f_t are determined by h_t and supervised by progress labels. What information beyond h_t do they give the monitor? Can you ablate the components of f_t (routing statistics vs. residual norms)?
  2. What exactly does the “decoupled execution monitoring” variant remove? Does it still include adapters and routing?
  3. Compared with directly feeding h_t or p̂_t as conditioning, how much does adapter routing add?
  4. Across benchmarks and real-world experiments, how many times is replanning triggered, for what reason (low MATCH vs. low DONE), and with what accuracy? How are the thresholds and the “persistence” window set, and how sensitive are results?
7 comments

Thank you for the constructive comments. Our responses are as follows.

W1.1 & Q1: What additional information does the action expert provide through the feedback vector?

The feedback vector ftf_t contains two types of information. The routing statistics summarize the adapter selection and weighting determined by the monitor state hth_t, and thus explicitly expose the modulation applied to action generation. The weighted residual norms, in contrast, depend on the action expert's hidden features xtx_t and quantify the magnitude of the adapters' modifications. Therefore, the latter provides action-dependent information beyond the monitor state.

To examine their individual contributions, we fix the four-adapter, top-2 configuration and ablate the feedback components on CALVIN.

Feedback to monitorMean lengthFive-task success
None4.31174.50%
Routing statistics only4.3275.80%
Residual norms only4.34275.90%
Both4.34775.50%

Compared with no feedback, routing statistics and residual norms individually improve mean sequence length by 0.009 and 0.031, respectively. Combining both yields the highest mean length (4.347), although residual norms alone achieve the highest five-task success (75.9%). These results suggest that action-dependent residual feedback provides a more substantial contribution, while routing statistics offer complementary information about the modulation applied to the action expert.

W1.2 & Q2: What does the decoupled execution-monitoring variant remove?

The decoupled variant in Table 5 retains the execution monitor for subtask tracking and replanning but removes the state-conditioned action adapters, their router, and the feedback vector ftf_t. Thus, it does not isolate the contribution of action-to-monitor feedback.

To distinguish the effects of monitor-to-action modulation and action-to-monitor feedback, we introduce an intermediate variant, action routing, which retains the same adapters and monitor-conditioned routing as the full method but disables action-to-monitor feedback.

CALVIN variantMean lengthFive-task success
Decoupled monitor4.2471.70%
Action routing4.3174.50%
EM-VLA4.3575.50%

Introducing action routing improves the average sequence length by 0.07 tasks and five-task success by 2.8 percentage points, although this comparison also includes additional adapter capacity. With the same routing and adapter configuration, EM-VLA further improves the average sequence length by 0.04 tasks and five-task success by 1.0 percentage point over action routing. These results demonstrate the contributions of both coupling directions.

W2 & Q3: What does adapter routing add compared with direct conditioning?

We compare direct FiLM conditioning on hth_t with adapter routing under action-loss-only training, without stage or balance supervision. We further evaluate five adapter configurations with stage and balance losses enabled, while keeping action-to-monitor feedback disabled.

Action conditioningStage supervisionAdapters KKTop-kkMean lengthFive-task success
Direct hth_t (FiLM)Off——4.0766.90%
Routed adaptersOff424.2472.30%
Routed adaptersOn224.2973.40%
Routed adaptersOn414.2772.90%
Routed adaptersOn424.3174.50%
Routed adaptersOn444.2973.40%
Routed adaptersOn824.3173.80%

Under action-loss-only training, adapter routing improves mean sequence length by 0.17 tasks and five-task success by 5.4 percentage points over direct FiLM conditioning. This comparison suggests that routing provides benefits beyond directly injecting the monitor state into the action expert, although the additional adapter capacity is also a factor. Among the tested configurations, four adapters with top-2 routing achieve the highest five-task success. At top-2, increasing the adapter count from four to eight produces negligible changes in mean sequence length (less than 0.01), suggesting that simply increasing adapter capacity does not consistently improve performance.

W3.1 & Q4.1: How frequently is replanning triggered?

Replanning frequency and trigger accuracy. We conduct 500 additional standard LIBERO-Long rollouts to quantify the frequency and causes of replanning, followed by manual inspection of all detected triggers. The results are reported in the following table.

Rollout groupRolloutsReplansTask success
w/ replanning31 (6.2%)5525/31 (80.6%)
w/o replanning469 (93.8%)0456/469 (97.2%)
All50055481/500 (96.2%)

A total of 55 replanning events occur across 31 rollouts. Of the 55 replans, 50 are triggered by low MATCH scores and five by low DONE scores. Manual video inspection indicates that 42 of the 55 triggers are appropriate. Additionally, among the 13 unsuccessful rollouts without replanning, we identify three cases in which a MATCH trigger is missed. The remaining failures typically stem from imprecise manipulation while the monitor tracks the correct task stage.

Detection under execution disturbances. To evaluate the monitor's sensitivity to infrequent environmental changes, we introduce four types of disturbances, each tested in 100 trials. The disturbance settings and detection criteria are as follows:

  • Pre-grasp target shift: The target object is moved to a different position before the robot grasps it. Detection requires a decrease of at least 0.20 in PROGRESS relative to its pre-intervention value.
  • Mid-carry object return: The object is moved back to its original position while the robot is transporting it. Detection requires three consecutive MATCH scores below 0.3.
  • First placement undone: The object is moved away from its target location after the first placement is completed. Detection requires either the same MATCH criterion or a DONE score below 0.8 after plan exhaustion.
  • Post-task undone: A placed object is removed from its target location after the task is completed. Detection requires a DONE score below 0.8.

We evaluate whether each disturbance is detected within five monitor updates following its occurrence. The pre-grasp target shift does not trigger replanning, as the original grasp instruction may remain valid, whereas the other three disturbances trigger replanning when their respective detection criteria are met. The results are presented in the following table.

DisturbanceDetected
Pre-grasp target shift76/100
Mid-carry object return96/100
First placement undone85/100
Post-task undone100/100
Total357/400
Reviewer_3tM1OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EM-VLA, which coordinates high-level planning and low-level action generation through an execution monitor with persistent memory. The monitor updates the execution state based on post-execution observations, action history, vision-language context, and internal feedback from the action model, predicting task completion, subtask matching, and subtask progress to control phase switching and on-demand replanning. Meanwhile, the execution state modulates action generation through sparse adapter routing, and the routing statistics and residual magnitudes are fed back to subsequent state updates. The authors construct subtask and monitoring supervision from successful demonstrations and evaluate on LIBERO, CALVIN, RoboTwin 2.0, and four real-robot tasks.

Critical strengths

This paper treats execution monitoring as a state representation that continuously participates in control, rather than merely being used to trigger one-time failure interventions, which aligns well with the state tracking needs in long-horizon tasks. The method connects high-level subtask management, low-level action modulation, and execution feedback, resulting in a clear overall structure. Experiments cover multiple simulation benchmarks and real robots, and directly test recovery capabilities through perturbations such as interrupting the current step and undoing completed steps. Existing ablations also show that continuous monitoring and its coupling with action generation can further improve performance over the base policy, providing preliminary support for the method.

Critical weaknesses

The contribution of the core bidirectional coupling still requires more detailed experimental attribution. Existing ablations mainly compare an independent monitor with the full method, and have not yet separately verified the roles of modulation from monitor to action, feedback from action to monitor, and additional adapter capacity; the routing analysis is also affected by explicit progress supervision, so it cannot solely prove that closed-loop feedback brings about phase specialization.

Secondly, part of the monitor's supervision comes from successful demonstrations and counterfactual pairing, and it is necessary to further test its ability to identify real failures such as grasping emptiness, slipping, and stagnation. In addition, the pseudocode does not clearly specify the plan update rules after completed steps are undone, leaving an explanatory gap with the real-machine recovery cases. For real machines, there are only 10 tests per task per condition, and some simulation gains are also small. It is recommended to supplement repeated experiments or confidence intervals, and clarify these limitations in the discussion.

Questions for authors

See above.

4 comments

We appreciate your feedback and questions. For your concerns, our responses are as follows.

W1.1: Ablation Study of Bidirectional Coupling and Adapter Capacity

We conduct additional ablation experiments on five-task sequences in CALVIN.

Bidirectional Coupling. The original CALVIN ablation includes both the decoupled monitor and EM-VLA. The decoupled monitor uses the execution monitor only for subtask switching and replanning, with no action adapters or feedback from the action expert.​ ​To distinguish the effects of monitor-to-action modulation and action-to-monitor feedback, we introduced an intermediate variant, ​action routing​, which retains the same adapters and monitor-conditioned routing as the full method but disables action-to-monitor feedback.

CALVIN variantAvg. Len.Five-task success
Decoupled monitor4.2471.70%
Action routing4.3174.50%
EM-VLA4.3575.50%

Introducing action routing improves the average sequence length by 0.07 tasks and five-task success by 2.8 percentage points, although this comparison also includes additional adapter capacity. With the same routing and adapter configuration, EM-VLA further improves the average sequence length by 0.04 tasks and five-task success by 1.0 percentage point over action routing. These results demonstrate the contributions of both coupling directions.

Adapter Capacity. To further examine the influence of adapter capacity and routing sparsity, we evaluate five configurations by varying the total number of adapters per layer (KK) and the number of activated adapters (top-kk).

Adapters per layer (KK)Active per layer (top-kk)Avg. Len.Five-task success
224.2973.40%
414.2772.90%
424.3174.50%
444.2973.40%
824.3173.80%

The configuration with four adapters and top-2 routing achieves the highest five-task success (74.5%). At top-2, increasing the adapter count from four to eight produces negligible changes in mean sequence length (less than 0.01), suggesting that simply increasing adapter capacity does not consistently improve performance.

W1.2: Role of explicit progress supervision in routing specialization.

We agree that Figure 3 alone cannot establish that closed-loop feedback induces phase specialization, as the router receives explicit subtask-progress supervision. To isolate the effect of this supervision, we compare action routing with and without the stage loss under the same adapter configuration.

Avg. Len.Five-task success
w/o stage loss4.2472.30%
w/ stage loss4.3174.50%

Introducing stage supervision improves the average sequence length by 0.07 tasks and five-task success by 2.2 percentage points, demonstrating its contribution to stage-dependent routing.

W2.1: Detection of environmental disturbances and recovery of undone steps.

To evaluate the monitor's sensitivity to infrequent environmental changes, we introduced four types of disturbances, each tested in 100 trials. The disturbance settings and detection criteria are as follows:

  • Pre-grasp target shift: The target object is moved to a different position before the robot grasps it. Detection requires a decrease of at least 0.20 in PROGRESS relative to its pre-intervention value.
  • Mid-carry object return: The object is moved back to its original position while the robot is transporting it. Detection requires three consecutive MATCH scores below 0.3.
  • First placement undone: The object is moved away from its target location after the first placement is completed. Detection requires either the same MATCH criterion or a DONE score below 0.8 after plan exhaustion.
  • Post-task undone: A placed object is removed from its target location after the task is completed. Detection requires a DONE score below 0.8.

We evaluated whether each disturbance was detected within five monitor updates following its occurrence. The pre-grasp target shift does not trigger replanning, as the original grasp instruction may remain valid, whereas the other three disturbances trigger replanning when their respective detection criteria are met. The results are presented in the following table.

DisturbanceDetected
Pre-grasp target shift76/100
Mid-carry object return96/100
First placement undone85/100
Post-task undone100/100
Overall357/400

Regarding recovery, MATCH evaluates consistency with the active subtask rather than explicitly verifying all previously completed subtasks. Undoing an earlier placement may therefore make the current transition inconsistent with the active subtask. If MATCH does not detect the change, DONE can identify the unmet overall goal after plan exhaustion. Upon replanning, the planner uses the current observation, original instruction, and completed prefix to generate a revised suffix. The completed prefix records execution history but does not prevent previously completed actions from being scheduled again when necessary. Thus, the revised suffix can include an earlier grasp or placement to recover from an undone step. We have clarified this update rule in Algorithm 1 and provide a real-robot disturbance rollout with corresponding monitor signals in Appendix F.

W2.2: Real robot evaluation scale and limitations.

We run 10 real-robot trials per task and condition, in line with prior work [1, 2]. In addition, we tested each task in the standard setting and under three types of perturbation (object, background, and execution), for a total of 40 trials per task for each policy. Nevertheless, we acknowledge that more trials would improve statistical reliability, but real-robot experiments are time-consuming. We plan to expand our evaluation and update the additional data and results in our forthcoming code repository to benefit the community. In particular, short tasks provide fewer subtask transitions at which monitoring and replanning can contribute. Thus, the high baseline success rates in simulation evaluations leave limited room for improvement. We have discussed these limitations in the revised manuscript.

Reference

[1] Moo Jin Kim et al. OpenVLA: An Open-Source Vision-Language-Action Model. 8th Conference on Robot Learning, 2024.

[2] Kevin Black et al. π₀.₅: a Vision-Language-Action Model with Open-World Generalization. 9th Conference on Robot Learning, 2025.

Thanks for you response. I think most of my concerns have been addressed. Since I gave a positive score, I'll keep my score as 3. Good luck!

Thank you for your positive score. We sincerely appreciate the time and effort you have devoted to reviewing our work. Your constructive review has helped us further improve the quality of our manuscript. We would be happy to discuss any questions or concerns you may have at any time.

Reviewer_FVywOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EM-VLA, a hierarchical VLA framework with an execution monitor for long-horizon manipulation tasks. During inference, the VLM first decomposes the instruction into subtasks and stores them. It updates them only when the monitor decides that replanning is necessary. The action expert predicts actions based on the current subtask. After each action chunk is executed, a lightweight monitor updates the execution state based on the executed actions and the new observation. It outputs three scores indicating whether the entire task is complete, whether the current execution matches the subtask, and the progress of the current subtask. These scores determine when to advance to the next subtask and when to replan. The same execution state also determines which adapters in the VLA’s MoE-like architecture are used by the action expert, and the adapter usage state is fed back to the monitor. All supervision is automatically annotated by the VLM from successful demonstrations. The experiments compare EM-VLA with existing VLAs on LIBERO, CALVIN, RoboTwin 2.0, and real robots. The real-robot experiments also test object variations, background variations, and human disturbances during execution.

Critical strengths
  1. Using the same pi0.5 backbone, the method achieves consistent improvements across three simulation benchmarks and the real-robot experiments, especially on LIBERO-Long and the randomized settings of RoboTwin, while introducing little additional inference overhead.
  2. The method is well designed. The authors use a lightweight monitor to continuously maintain the execution state and allow it to interact with the VLA. The monitor updates the state based on visual changes after each execution and information from the VLA. This state determines when to advance to the next subtask and when to replan. The same state also determines which adapters the action expert uses, and the adapters’ responses are fed back to the monitor. This avoids invoking high-level reasoning at every step while allowing the execution state to continuously influence action generation.
Critical weaknesses

The monitor is trained only on successful demonstrations and has never seen an actual failure. Can it still make correct decisions when it encounters OOD failure cases? The authors should also report how many times replanning was triggered during the experiments and whether these triggers actually corresponded to failures in task execution. The only disturbance experiment is conducted on real robots, but it does not include comparisons with pi0.5 or other feedback-based methods, nor does it include a version without the monitor. It is therefore impossible to determine whether the difference comes from the monitor or the backbone.

Questions for authors
  1. Since the simulation experiments do not include any failures or disturbances, it is currently unclear whether the monitor will trigger replanning when an actual failure occurs. If failures are injected into simulation, such as a failed grasp or an object being moved away, when will replanning be triggered, and how many times will it be triggered? One disturbance in the real-robot experiments reverses an already completed step. Which mechanism detects this situation? Based on my understanding, MATCH appears to check only the current subtask.
  2. The real-robot disturbance experiments currently compare only against OpenVLA, pi0, and StarVLA, while EM-VLA itself is built on the stronger pi0.5 backbone. This makes it difficult to determine how much of the improvement under disturbances comes from the monitor. Could the authors add results for base pi0.5 and feedback-based methods such as “pi0.5 with a failure detector that triggers replanning”?
2 comments

We appreciate your feedback and questions. They helped us explain our method more clearly.

Q1.1: Unclear Replanning Trigger Under Actual Failures

We conduct 500 additional standard LIBERO-Long rollouts to quantify the frequency and causes of replanning, followed by manual inspection of all detected triggers. The results are reported in the following table.

Rollout groupRolloutsReplansTask success
w/ replanning31 (6.2%)5525/31 (80.6%)
w/o replanning469 (93.8%)0456/469 (97.2%)
All50055481/500 (96.2%)

A total of 55 replanning events occur across 31 rollouts. Manual video inspection indicates that 42 of the 55 triggers are appropriate. Additionally, among the 13 unsuccessful rollouts without replanning, we identify three cases in which a MATCH trigger is missed. The remaining failures typically stem from imprecise manipulation while the monitor tracks the correct task stage.

Q1.2: When and how often is replanning triggered by simulated failures?, such as a failed grasp or an object being moved away.

To evaluate the monitor's sensitivity to infrequent environmental changes, we introduce four types of disturbances, each tested in 100 trials. The disturbance settings and detection criteria are as follows:

  • Pre-grasp target shift: The target object is moved to a different position before the robot grasps it. Detection requires a decrease of at least 0.20 in PROGRESS relative to its pre-intervention value.
  • Mid-carry object return: The object is moved back to its original position while the robot is transporting it. Detection requires three consecutive MATCH scores below 0.3.
  • First placement undone: The object is moved away from its target location after the first placement is completed. Detection requires either the same MATCH criterion or a DONE score below 0.8 after plan exhaustion.
  • Post-task undone: A placed object is removed from its target location after the task is completed. Detection requires a DONE score below 0.8.

We evaluate whether each disturbance is detected within five monitor updates following its occurrence. The pre-grasp target shift does not trigger replanning, as the original grasp instruction may remain valid, whereas the other three disturbances trigger replanning when their respective detection criteria are met. The results are presented in the following table.

DisturbanceDetected
Pre-grasp target shift76/100
Mid-carry object return96/100
First placement undone85/100
Post-task undone100/100
Total357/400

Q1.3: How to detect the situation which reverses an already completed step?

MATCH evaluates the consistency between the current observation and the active subtask. Although it does not explicitly verify previously completed subtasks, undoing an earlier placement may alter the environment state in a way that becomes inconsistent with the current subtask, thereby triggering replanning. If the current transition remains consistent with the active subtask, DONE can detect the unmet overall goal once the plan is exhausted. Upon replanning, the planner takes the new observation, original instruction, and completed task prefix as input to generate a revised plan suffix. Since the completed prefix preserves the execution history, the newly generated suffix can include previously completed actions, such as repeating an earlier grasp or placement, when necessary. We have clarified this rule in Algorithm 1. The updated case study in Appendix F now presents a single real-robot rollout with additional disturbances and the corresponding monitor signals.

Q2: Comparison with base pi0.5 and feedback-based method (pi0.5 with a failure detector).

We have added three π0.5\pi_{0.5}-based baselines to the real-robot evaluation:

  • Base π0.5\pi_{0.5}: Generates actions conditioned on the instruction and current observation. This baseline is included to isolate the contribution of the monitoring mechanism.
  • π0.5\pi_{0.5} + Periodic Replanning: Tracks completed subtasks and invokes the planner every four action chunks.
  • π0.5\pi_{0.5} + FAIL-Detect Replanning: Tracks completed subtasks and invokes the same planner when FAIL-Detect [1] raises an alarm.

The additional results are reported in the following table. Under execution perturbations, EM-VLA achieves a 25% improvement over base π0.5\pi_{0.5} and a 15% improvement over the strongest baseline, π0.5\pi_{0.5} with FAIL-Detect-based replanning. These results suggest that the improvement is not solely attributable to the stronger π0.5\pi_{0.5} backbone but also to EM-VLA's monitoring and replanning mechanism. In our real-robot experiments, FAIL-Detect occasionally triggers unnecessary replanning, whereas periodic replanning may delay necessary plan updates when disturbances occur between scheduled calls. We have included these additional comparisons in Table 4 of the revised manuscript.

MethodStandard successStandard lengthExecution-perturbation successExecution-perturbation length
Base π0.5\pi_{0.5}29/40 (72.5%)0.82515/40 (37.5%)0.627
π0.5\pi_{0.5} + FAIL-Detect replanning27/40 (67.5%)0.79419/40 (47.5%)0.694
π0.5\pi_{0.5} + periodic replanning29/40 (72.5%)0.83818/40 (45.0%)0.669
EM-VLA32/40 (80.0%)0.87525/40 (62.5%)0.806

Reference

[1] Chen Xu et al. Can We Detect Failures Without Failure Data? Uncertainty-Aware Runtime Failure Detection for Imitation Learning Policies. Robotics: Science and Systems XXI, 2025.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.