acceptodds
Under review as a conference paper at ICLR 2027

When Does an Observable Lose Failure Evidence? Exact Ranking Losses for Runtime Detection in Vision–Language–Action Policies

Abstract

No runtime failure detector for a vision–language–action (VLA) policy can beat the Bayes-optimal ranking of the observation it reads. This ceiling is standard; what a practitioner needs to know is what separates it from the score actually deployed. We decompose that gap exactly into a representation loss from compressing the observation (for example, into sparse-autoencoder atoms), a projection bias from fitting a misspecified likelihood family, and an estimation term, and we characterize each. A compression is lossless if and only if it preserves the success posterior; otherwise its AUROC cost has an exact form and is bounded by the missing conditional information with a sharp constant . Ignoring the direction of time costs at most AUROC, where is the path-reversal divergence of outcome class ; the constant is sharp, yet irreversibility does not identify this value, and fully reversible paths can separate the outcomes perfectly. Low-order Markov likelihood projections can erase all evidence, and score consistency yields AUROC consistency exactly when the limit ties only states with equal likelihood ratio, a condition that a consistent maximum-likelihood fit can violate. For evaluation, early termination adds an exact bonus to prefix AUROC, and without structural assumptions no finite-sample procedure can certify a low ceiling. On seven LIBERO simulation interfaces spanning OpenVLA, OpenVLA-OFT, and 0.5 (7,000 report rollouts; no physical robot), episode length alone reaches AUROC 0.9996–1.0000 as the termination identity predicts, other atom-level families exceed the pre-specified Grammar-LLR witness by up to 0.42 on the same rollouts, and the ordering of atom-level and hidden-state witnesses changes across interfaces.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.