acceptodds
Under review as a conference paper at ICLR 2027

Learning to Act under Visual Interruptions with Vision-Language-Action Models

Abstract

Vision-language-action (VLA) models have demonstrated strong capabilities in robotic manipulation, but they are typically developed and evaluated with all camera streams available throughout task execution. When a camera stops delivering frames during task execution, the policy must continue acting without access to subsequent observations from the missing view. Despite its practical importance, how such interruptions affect closed-loop manipulation remains insufficiently understood. To investigate this problem, we introduce MAIL-Bench, a benchmark that evaluates visual interruptions with VLA models. By interrupting different cameras at multiple stages of each policy's successful reference trajectory, MAIL-Bench measures how well policies retain their capabilities when visual inputs become unavailable. Building on this benchmark, we propose MINT, which trains VLA policies to handle missing visual inputs and temporarily supplies predicted images when cameras fail. It uses optical-flow extrapolation for missing wrist views or all-vision loss, and an action-conditioned world model for missing third-person views, withdrawing the predictions when they become unreliable. Experiments on and GR00T N1.5 show that MINT improves task success under camera loss over the original models. Experiments on the AgiBot G2 further demonstrate real-robot closed-loop deployment under camera loss.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robot learning, embodied AI, vision-language-action models, benchmark, robustness, world models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_odvBOpenReview ↗
Rating
4(+2)
Confidence
3
Informativeness
3(+1)
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes MAIL-Bench, a benchmark for evaluating VLA models under visual interruption. It creates scenarios where different camera inputs are interrupted at multiple stages of successful reference trajectories.

The paper also proposes MINT, which trains VLA policies to handle missing visual inputs and temporarily provides predicted images when cameras fail. It uses optical-flow extrapolation when the wrist camera is unavailable and an action-conditioned world model when the third-person camera is unavailable.

Experiments on π0.5 and GR00T N1.5 show that MINT improves task success under camera interruption compared to the original models. The paper also conducts real-robot experiments to demonstrate the effectiveness of the proposed approach.

Critical strengths
  1. The paper identifies an important yet relatively underexplored scenario in robotics, where camera observations may become unavailable during task execution, and addresses this problem in a novel way.
  2. By providing both a benchmark and methodological baselines that can serve as foundations for future research, the paper has the potential to have a meaningful impact on the research community.
  3. The proposed method consistently improves performance over the baseline models across different settings, suggesting that the methodology is robust and stable.
Critical weaknesses
  1. The evaluation is limited to wrist and third-person views, and there is insufficient validation for settings involving other types of camera views.

  2. The benchmark only considers interruption points at 30%, 45%, and 60% of the task execution time. It would be valuable to investigate how the results change at other interruption points, particularly at extreme cases such as very early or very late interruptions.

  3. There is a lack of comparison between MINT and prior work on learning with missing modalities. Currently, the main evidence for MINT appears to be that it improves the performance of the baseline models, but this alone does not sufficiently establish the necessity of MINT. In particular, I would like to see what happens when existing missing-modality recovery techniques are used to replace the missing camera inputs. If such approaches achieve comparable performance, the motivation and necessity of MINT would need to be reconsidered. Therefore, I consider this a critical omission.

I believe the authors should demonstrate the necessity of MINT through direct comparisons with relevant existing methods. If these comparisons show that MINT provides clear improvements over existing approaches, I would be willing to increase my score.

Questions for authors
  1. What happens if the action-conditioned world model is used when the wrist camera is missing, and optical-flow extrapolation is used when the third-person camera is missing? I am curious why the authors chose different models for the two types of camera views.

  2. I question whether generating the missing visual input is necessary in the first place. In particular, it is unclear how much computational overhead is introduced by the visual generation process and whether the resulting computational cost is practical for real-time robotic task execution. The authors should report the computational cost and latency of visual prediction and discuss whether it is feasible for real-time deployment.

Reviewer_kogEOpenReview ↗
Rating
3
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The authors propose MAIL-Bench, a benchmark that evaluates visual interruptions with VLA models. This benchmark removes different cameras at multiple stages of each policy's rollout, thus measuring how well policies retain their capabilities when visual inputs become unavailable. Building on this benchmark, they propose MINT, a framework which trains VLA policies to handle missing visual inputs and temporarily supplies predicted images when cameras fail, via action-conditioned video/world modeling and optical flow prediction.

Critical strengths

Clean benchmark: the MAIL-Bench benchmark provided by the authors appears easy to use and provides a simple test-bed to compare visual robustness of visuomotor policies when a camera view becomes unavailable. This is important because as visuomotor policies become more adept, and real-world ready, having evaluation metrics that can assess something realistic (like a camera going blank mid rollout), will be important.

Critical weaknesses

Limited insight from the experiments: the evaluated baselines (pi0.5, GR00T 1.5) were likely not trained with camera dropout, so their failure under mid-episode camera loss is expected. MINT is then trained on curriculum that samples uniformly from [0.3, 0.7] of the trajectory, which closely matches the benchmark's (MAIL-Bench) own test conditions (30/45/60%), so its not really testing generalization.

Limited benefit from world modeling: the ablation in Figure 4 suggests the gains come mainly from the masked-training recipe (missing-view FT) rather than the proposed method (MINT) which comes with substantial test-time cost to predict the next frames.

Questions for authors

Does MINT need to be told a camera died? The manuscript says the benchmark sets the availability flag to 0, but real-world failures would just show up as a stale or black frame.

Evaluations with stale frame: I would be curious to know how the baseline models perform with, instead of zeroing a frame / view mid trajectory, we just stop changing the frame. For example, if at time=t the third person camera goes out, we can instead keep the third person camera the same for t'>t, and the wrist-camera still the same. This would help us determine if the real problem is visual consistency, or lack of modality altogether. It seems like from the appendix this is possible, but I couldn't find results for it in the main manuscript.

Reviewer_JDV8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies VLA robustness when a camera stream is persistently lost during execution. It has two main contributions: (i) MAIL-Bench, a paired closed-loop benchmark that removes vision by role at 30/45/60% of the policy's own completion time, scored conditioned on healthy success; (ii) MINT, which trains with missing views and, at inference, selectively completes frames via optical flow or a world model, gated by a reliability check.

Critical strengths

This paper is well-motivated and deployment-relevant:

Mid-run camera dropout is a common failure mode in real-world deployment that has rarely been evaluated systematically. The authors target a genuine gap here.

Reliability gating + fallback is a pragmatic safety design:

Predictions are used only when they pass route-specific reliability checks; once rejected, they are irreversibly withdrawn for the rest of the episode. This gating strategy is important for closed-loop robustness, as it prevents unreliable generation from steering the policy astray.

Reasonably complete experiments:

The study covers two mainstream backbones (π₀.₅, GR00T N1.5), three task categories, and nine interruption settings, complemented by targeted ablations (Clean FT + prediction, Missing-view FT). The experimental scope is adequate for supporting the main claims.

Critical weaknesses

The evaluation only runs on already-confirmed-successful trajectories, which is unrealistic. All robustness numbers are conditioned on healthy success: the episode is first rolled out with full cameras, only successful scenes enter the interruption evaluation, and the onset T_f is derived from that rollout's completion time T_healthy. A real system can neither foresee whether a rollout will succeed nor use a "healthy reference" to set the 30/45/60% onsets. This introduces selection bias, since only policy-friendly scenes are evaluated. I'd like to see results that run from scratch on arbitrary scenes without a confirmed-successful reference, which would better reflect actual deployment conditions.

The world model's runtime reliability is not quantified, especially its discard rate. Only reconstruction PSNR and out-of-domain fraction are reported, which feels insufficient for a core component of the method. The authors should add: the fraction of world-model-generated frames ultimately discarded by the gate or never passing validation, and the typical timing of withdrawal. These numbers would directly tell whether frame completion is worth its computational cost.

The contribution is somewhat limited and combinatorial. Missing-view/masked training is an existing line, completion reuses off-the-shelf Farnebäck flow and PersistWorld, and what is genuinely new is mainly the reliability gating plus the combination, along with the benchmark itself. MAIL-Bench also covers only a single embodiment and 18 RoboCasa365 tasks, leaving generalization under-validated.

Writing needs improvement. There are clear textual errors, e.g., the repeated "When when" at Ln 251. The manuscript could benefit from another proofreading pass.

Questions for authors

I'm not fully convinced about the optical-flow branch's benefit, and I think a key ablation is missing here. It essentially warps the last frame in 2D and cannot synthesize robot-object interaction; the paper itself admits warping cannot reliably capture robot-object configuration changes. With such a tight gate (motion budget 0.15 m / 0.70 rad), it is likely withdrawn once the arm actually moves. Could the authors add an ablation that removes the optical-flow branch entirely to quantify its actual gain?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.