acceptodds
Withdrawn from ICLR 2027

When the Safety Layer Is Not Safe: In-depth Analysis of VLA Failure Detection under Distribution Shift

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-language-action (VLA) policies achieve impressive success rates on manipulation benchmarks, yet recent robustness analyses show this competence is brittle and collapses under modest open-world variation. Runtime failure detectors are increasingly used alongside VLA policies. Like the policies they monitor, these detectors perform well under ideal conditions, raising an alarm in time for the robot to stop, back off, or ask for help. Existing work, however, trains, calibrates, and evaluates these detectors only on clean rollouts and rarely tests them under the distribution shifts (e.g., a changed camera viewpoint or robot initial state) that cause policies to fail in the first place. We therefore collect a 5,501-rollout corpus of successes and failures of two VLA policies under seven controlled perturbation dimensions and introduce a three-distribution protocol that separates a detector's ranking ability from the validity of its calibrated threshold. We find that thresholds break before rankings: clean-calibrated detectors retain most of their discrimination under shift but read unfamiliar success as failure, raising false alarms by 16.5 percentage points on the shifts to which policies are most sensitive. The direction of the break is set by how the score accumulates signed per-step evidence, not by how far features drift: on the same features, kNN detectors over-alarm while a Mahalanobis detector falls silent and misses failures. Task novelty, by contrast, mainly erodes ranking. No evaluated repair restores nominal calibration without sacrificing detection, and label-free recalibration is circular because the unlabeled deployment pool contains the failures the monitor is meant to catch. Neither a high AUROC nor a low false-alarm rate therefore certifies a monitor. A safety layer built for the moment a policy cannot be trusted must be evaluated and calibrated under the shifts that create that moment, with scores that do not mistake novelty for failure. Code and data will be released upon publication.

Primary area:
interpretability and explainable AI
Keywords:
Failure Detection, Robustness, Robot manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_5XM8OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors analyze the robustness of runtime failure detectors for VLA policies under distribution shifts. They evaluate failure detectors on two VLA policies, OpenVLA-OFT and π0, under seven types of environmental and instruction perturbations, with the main evaluation being a protocol that compares clean successes, perturbed successes, and perturbed failures to differentiate failure discrimination, threshold validity, and sensitivity to novelty. Results indicate that some detectors retain strong ranking performance but generate too many false positives, while others become less sensitive and miss failures. The authors further analyze temporal score accumulation, generalization to unseen tasks, and recalibration.

Critical strengths
  1. The paper asks that if robot policy becomes unreliable across those same changes, do failure detectors also become unreliable? Which I believe is an important and well throughout question Testing the monitor itself across distribution shifts is a relevant concern for deploying this system.
  2. Helpful difference of evaluation dimensions i.e comparing clean successes, shifted successes and shifted failures makes it possible to separate the sensitivity to the unfamiliar from the sensitivity to the failure.
Critical weaknesses
  1. The main insight that ranking performance could stay solid while a fixed threshold would become unstable under distribution shift isn't necessarily a big leap conceptually. Furthermore, the paper could better communicate what insights it contributes more generally through its three-distribution protocol, rather than just through its systematic empirical investigation of existing detectors.
  2. These experiments use independently calibrated timestep-wise quantiles rather than the form of the simultaneous functional calibration inspired by SAFE. These pointwise thresholds hint that we do not get control over false-alarms at the entire trajectory level, and some setups seem miscalibrated even in the absence of interference. Using measures of degradation relative to clean performance may help to identify further degradation, but does not demonstrate that the original calibration scheme would likely perform the same way.
  3. The bulk of evidence is from applying each policy to a simulated benchmark family with multiple severity cells but sparse outcome support. Using alarms offline without a stopping or recovery controller calls into question whether the simulated results can settle the question of whether the alarm behavior is likely to raise or lower physical risk. Additionally, repairs are selected ex-post based on coverage error, making the experiment's achievements diagnostic rather than representative of a deployment procedure that does not use outcome labels. 4.The related work could better cover studies on robot failure diagnosis and robustness under environmental changes. For example, The Colosseum [1] evaluates manipulation policies under a range of environmental perturbations, RoboMD [2] investigates conditions that make robot policies fail, and Colosseum V2 [3] examines VLA generalization across tasks and environments. Although these works focus on policy robustness rather than runtime detector calibration, discussing them would help place the paper’s findings in context and clarify its contribution.

References

  1. Pumacay, Wilbert, et al. "The colosseum: A benchmark for evaluating generalization for robotic manipulation." arXiv preprint arXiv:2402.08191 (2024).
  2. Sagar, Som, et al. "Robomd: Uncovering robot vulnerabilities through semantic potential fields." International Conference on Learning Representations. Vol. 2026. 2026.
  3. Morgan, Jeremy, et al. "Colosseum v2: Benchmarking generalization for vision language action models." arXiv preprint arXiv:2605.27759 (2026).
Questions for authors
  1. How would the outcomes differ if the original functional calibration described in SAFE were used rather than timestep-wise quantiles? Would the finding that thresholds become unreliable while detectors maintain strong ranking performance still hold?
  2. If the repair hyperparameters were chosen using a separate validation set rather than by minimizing coverage error on the evaluation set, would the observed trade-off between calibration and failure detection still hold?
  3. The primary analysis centers on OpenVLA-OFT and π0. Would the supplementary policy rollouts already collected be sufficient to demonstrate whether the same behavior of excessive false alarms and missed failures is exhibited by other policies?
Reviewer_78c6OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies how runtime failure detectors for vision-language-action (VLA) policies behave under distribution shift when they are trained and calibrated only on clean rollouts, i.e. rollouts without any perturbations of the environment. The authors collect successful and failed rollouts of OpenVLA-OFT and π0 on the ten LIBERO-10 tasks under seven LIBERO-plus perturbation dimensions. The primary analyses use 2,981 rollouts; the 5,501 reported in the abstract includes auxiliary rollouts from other policies and VLA-author-released rollouts.

They propose a three-distribution protocol comparing what the authors call clean successes, perturbed successes, and perturbed failures, with three metrics:

  • the change in false-alarm rate on successes (ΔFPR);
  • the AUROC between failures and successes under shift (SpecAUC);
  • the AUROC between shifted and clean successes (ShiftResp).

They apply the protocol to existing detectors: SAFE MLP/LSTM probes, plus Mahalanobis, kNN, and LogpZO scores on the same hidden features. They report four findings:

  • Calibrated thresholds degrade while ranking is largely preserved.
  • Whether a detector over-alarms or goes silent depends on how its score accumulates per-step evidence.
  • Task novelty mainly harms ranking, while condition shift mainly moves the operating point.
  • None of several adapted recalibration and augmentation methods restores nominal calibration without losing detection.

The paper proposes no new detector or remedy, and all experiments are in simulation.

Critical strengths

A relevant, underexplored problem with a sensible protocol. Separating threshold validity from ranking is a useful way to evaluate detectors under distribution shift. The main empirical finding is practically relevant: thresholds degrade while ranking largely survives. For example, at robot-init L5 the OFT MLP keeps an AUROC of 0.93 while only 20% of successful rollouts are correctly labeled as a success. This shows that a single AUROC can hide a detector that is unusable at its calibrated operating point.

A useful dataset. The corpus contains both successes and failures for two policies across seven controlled shift dimensions. If released as promised, it would be a valuable resource for evaluating failure detectors under shift.

Critical weaknesses
  1. The contribution rests entirely on the empirical analysis, and that analysis is too narrow to support its general claims.

The authors state that they do not propose a new detector (Sec. 2), and the repair study evaluates adaptations of existing methods (Sec. 7). The paper's value must therefore come from the evaluation protocol and the breadth of the analysis. Both are limited.

  • The metrics are standard quantities under new names. ΔFPR is a difference in false-positive rates. SpecAUC is AUROC computed on perturbed data. ShiftResp is the AUROC for separating clean from perturbed successes, i.e., a standard out-of-distribution detection AUROC. The separation they enable is useful but conceptually modest.
  • The setting is narrow, and the policies are dated.
    • All experiments are in simulation on one benchmark family (LIBERO-10: ten tasks, one embodiment), with two primary policies and no real-robot validation.
    • Both policies are older-generation VLAs: π0 (2024) and OpenVLA-OFT (early 2025).
    • A newer open-weight VLA with an official LIBERO checkpoint, π0.5, was available. The authors used it to collect auxiliary rollouts (Table 2, App. H) but not in the main analysis.
    • The detectors read the policy's hidden states, and newer VLAs are trained differently (e.g., co-trained on web and multi-source data). It is therefore unclear whether the findings carry over to the policies practitioners would deploy today.
  • The detector families are narrow. The claims are about clean-calibrated detectors in general, but the evidence covers only detectors that read policy hidden states. Other families may respond very differently to visual distribution shifts such as a changed camera viewpoint, and the paper cites [1] in its related work but does not evaluate it. Missing families include:
    • action-space consistency methods, such as STAC [1] and the action-chunk consistency detector of VLA-FAIL [2];
    • progress monitors based on VLMs or reward models, such as Robometer [3].
  • The training data is small. OFT has only 38 failures among 500 clean rollouts, so each 250-rollout training split contains roughly 19 failures. Several of the perturbation types (camera viewpoint, lighting, background, sensor noise) are exactly what domain randomization during probe training would target. It is therefore unclear whether the observed degradation reflects a fundamental limitation or an under-trained detector.
  1. No remedy is proposed, and the repair study (Sec. 7) is not rigorous enough to support its conclusion that no repair restores calibration under distribution shift without sacrificing detection.

Since this section is the basis for the suggestions in Table 1, it matters that this repair study is rigorous.

  • The repair pool does not contain the perturbations that matter. All methods that use deployment data draw on one pool of 155 rollouts, from language-instruction and object-layout perturbations at the mildest severity (L1) only. One of these barely moves the OFT detector (ΔFPR = −0.7 pp, Table 5). Camera-viewpoint and robot-initial-state perturbations, where calibration breaks most, never appear in the pool. The paper's "labeled shifted rollouts" setting is therefore never tested with labels from the perturbation it is meant to repair, which may explain why TrainAug/CalAug do not help OFT.
  • Configurations are selected on evaluation data, using coverage error alone. For each method, the reported configuration is the hyperparameter setting (e.g., inflation level, ACI step size, retention fraction) with the smallest coverage error on the same evaluation rollouts that results are reported on (App. G). This requires success labels on the test rollouts, which a deployment would not have, so the numbers are best-case readouts rather than the output of a usable procedure.
  • Different methods are evaluated on different split subsets. Therefore you cannot really compare them.
  • There is no reference point. Without an oracle threshold (e.g., a band calibrated on labeled successes under the same perturbation) or a simple score-recalibration baseline, Fig. 6b is hard to interpret. Since every repair shifts a decision boundary on fixed scores, the frontier may largely trace the underlying ROC curve. -The methods lose their guarantees. They are adaptations of the cited methods without their guarantees, as the authors acknowledge.
  1. The presentation substantially reduces the paper's value.

Because the contribution lies in the protocol and analysis, I expect exceptional clarity but the paper lacks it.

  • Style. The paper is written in a rhetorical, narrative voice rather than standard technical language, and findings are often stated as slogans before the quantities behind them are defined. Examples:
    • section titles: "Thresholds break before rankings," "Repair is bounded by information";
    • anthropomorphic descriptions of detectors: "the monitor still knows which rollouts look worse," "falls silent," "reads unfamiliar success as failure";
    • informal phrasing: "it is sobering," recalibration being "poisoned."
  • "Shift" is never defined, and it is used for several different things.
    • The title refers to distribution shift, but the paper never states which distribution changes relative to which reference (presumably clean LIBERO-10 rollouts). Nor does it say whether the perturbations amount to covariate shift, a change in failure prevalence, or both. Defining this clearly matters because the repairs it adapts, such as conformal prediction under covariate shift, depend on exactly that assumption.
    • "Shift" also denotes a single perturbation type ("camera shift," "the two strong shifts").
    • It denotes perturbations of seen tasks as opposed to unseen tasks ("condition shift" vs. "task novelty"), although unseen tasks are themselves a distribution shift.
    • It denotes the displacement of detector scores ("score shift," the ShiftResp metric).
    • It denotes synthetic translations of features ("mean shifts").
    • "Shifted" and "perturbed" are used interchangeably, and "drift" overlaps with all of these meanings.
  • Bespoke terminology. Examples include ShiftResp, SpecAUC, "calibration circularity," "drift-estimation bias," "oracle-relative bias," "c-robust," "information budget," "safety gap," "strong shifts," and "deployment-proxy cell D." Most are not used in related work, and several rename standard quantities.
  • Terms used without definition or before definition.
    • LogpZO is never explained.
    • The First/Last/First&Last/Mean feature aggregations are used in Sec. 3.2 but never defined.
    • FCP is never expanded.
    • "Coverage error" first appears in Sec. 7.
    • The intro cites "drift-estimation bias of 48% to 317%" before the term is defined.
    • The distinction between feature steps and environment steps is only explained in App. E.
  • Standard names for non-standard methods. "Mahalanobis (Lee et al., 2018)" and "kNN (Sun et al., 2022)" are implemented as supervised, signed contrasts accumulated over time (Eq. 4), not the usual distance-to-training-data scores. The mechanism finding of Sec. 5 rests on these constructions, so the naming misrepresents what the conclusions apply to.
  • Careless errors.
    • The abstract and introduction headline a "5,501-rollout corpus," but only 2,981 rollouts (OpenVLA-OFT and π0) enter any analysis. The remaining 2,520 are auxiliary or legacy rollouts that contribute to no reported result (Table 2).
    • The Fig. 8 caption refers to panels c and d, but the figure has panels a and b.
    • Fig. 5b plots I against C − A, two quantities that share a term, so their correlation is partly by construction (the authors note this).
    • The bibliography entries for π0 and π0.5 are broken.

[1] Agia, C., Sinha, R., Yang, J., Cao, Z., Antonova, R., Pavone, M. and Bohg, J., 2024. Unpacking failure modes of generative policies: Runtime monitoring of consistency and progress. CoRL 2024.

[2] Seligmann, F., Gospodinov, E., Dincer, E.U. and Neumann, G., 2026. VLA-FAIL: Efficient Task Failure Detection for Finetuned Vision-Language-Action Models. CoRL 2026.

[3] Liang, A., Korkmaz, Y., Zhang, J., Hwang, M., Anwar, A., Kaushik, S., Shah, A., et al., 2026. Robometer: Scaling general-purpose robotic reward models via trajectory comparisons. RSS 2026.

Questions for authors
  1. Both primary policies are older-generation VLAs. Table 2 and App. H show that you collected π0.5 rollouts with its official LIBERO checkpoint but used them only as auxiliary data. Why was π0 chosen over π0.5 as a primary policy, and do the main findings hold for π0.5 or other recent VLAs?
  2. Do you expect the results for hidden-state detectors to transfer to action-space consistency and progress-based detectors? Would those detectors fail in the same way under distribution shift, and why?
  3. Do ΔFPR and SpecAUC change if the probes get more clean training data (OFT has roughly 19 failures per training split) or are trained with augmentation or domain randomization over the perturbed factors?
  4. Why does the deployment pool contain only language-instruction and object-layout rollouts at severity L1? How do TrainAug/CalAug and re-anchoring perform when the pool includes camera-viewpoint or robot-initial-state rollouts?
  5. Please define formally what distribution shift means in the paper: which distribution changes, relative to which reference, and whether the LIBERO-plus perturbations satisfy the covariate-shift assumption that the adapted repairs (e.g., weighted conformal prediction) rely on.
Reviewer_1XD6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This is an analysis paper. It asks how runtime failure detectors for VLA policies, mainly the SAFE MLP and LSTM probes on policy hidden states, behave when the environment shifts.

Corpus. Successes and failures of OpenVLA-OFT and π0 on LIBERO-10 under seven LIBERO-plus perturbation dimensions and five severities. Collection is adaptive, targeting ≥30 of each outcome per policy and dimension. The primary analysis uses 2,981 rollouts (1,000 clean + 1,981 perturbed) out of a 5,501-rollout inventory.

Protocol. Three distributions — clean successes, perturbed successes and perturbed failures — give three metrics:

  • ΔFPR: the change in false-alarm rate from clean to perturbed successes.
  • SpecAUC: AUROC within the shifted condition.
  • ShiftResp: whether perturbed successes outscore clean successes.

Findings.

  1. Thresholds break before rankings. On camera and robot-initialization shifts, SAFE probes keep SpecAUC around 0.79–0.87, yet their false alarms rise by 16.5 pp on average (up to 31.3).
  2. The direction of the break depends on how the score accumulates signed per-step evidence. kNN detectors over-alarm by 44–50 pp, while Mahalanobis falls silent.
  3. In a 120-split × 5-seed task/condition factorial, task novelty mainly erodes ranking. Unseen-task AUROC ranges from 0.20 to 0.92 across splits.
  4. No evaluated repair restores calibration without giving up detection: quantile inflation, ACI-style feedback, re-anchoring, density-ratio weighting, CORAL and labeled augmentation. Label-free recalibration is circular because the unlabeled pool contains failures (drift-estimation bias of 48–317%).
Critical strengths
  1. Point out the under-studied question with a reusable protocol. Separating ranking (SpecAUC), novelty response (ShiftResp) and threshold validity (ΔFPR), all measured relative to the clean FPR, is a real methodological contribution. The headline finding — monitors keep their ranking but lose their operating point exactly on the shifts that make policies fail — is actionable for anyone deploying conformal-style monitors. The observation that a falling FPR can mean the monitor has gone silent is a useful warning.
  2. Breadth and care of the analysis. The exhaustive 120-split factorial shows that conclusions from a few held-out-task splits can mostly reflect which tasks were chosen (B-cell AUROC 0.20–0.92; the mean over 3 splits has a 95% range of 0.43–0.76). This is a valuable warning for this sub-field's evaluation practice. The calibration-circularity argument is stated precisely and quantified, and limitations are disclosed candidly.
Critical weaknesses
  1. Metrics don't capture timeliness. All three metrics are computed on the peak score m = max_t s_t, so a failure that is flagged only after it has become irreversible counts the same as one flagged early. For a safety layer, timeliness is the point. Could you report a timing-aware metric under shift, e.g., detection lead time relative to failure onset, or the fraction of failures flagged before a fixed deadline? Does "rankings survive" still hold once timeliness is required (e.g., SpecAUC on the score at a fixed step, or a time-to-detection curve)? Even onset annotations on a small subset would help.
  2. Lack of real-world experiments to verify the transferability of the protocol. The protocol could serve as a reusable evaluation suite in simulation. Before that, however, it needs evidence that the metrics and conclusions obtained in simulation correlate positively with real-world behaviour. All evidence comes from one simulated benchmark family and two policies. Even a small real-robot study (one policy, two shift types such as camera and lighting, a few dozen rollouts per condition) showing that the simulated ΔFPR/SpecAUC ordering across shifts is preserved on hardware would substantially strengthen the paper.
Questions for authors
  1. The abstract and introduction present a 5,501-rollout corpus as a contribution, but all analyses use only the OpenVLA-OFT and π0 subset (1,000 clean + 1,981 perturbed = 2,981; Table 2). The remaining 2,520 rollouts (π0.5 and OpenVLA, 1,520; legacy π0/π0-FAST clean runs, 1,000) are never analysed, and the legacy runs serve only as regression checks. Why were π0.5 and OpenVLA, whose perturbed rollouts are already collected, left out? Please either report at least the headline ΔFPR/SpecAUC results for them, which would also strengthen generality beyond two policies, or state the analysed corpus size (2,981) in the abstract and contributions.
  2. How do the main results (ΔFPR, SpecAUC, ShiftResp on camera and robot-init) change with (i) SAFE's original functional conformal band, and (ii) scoring the full episode instead of truncating to the per-task minimum length?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.