DoOver: Detect Failure from Demos, Do It Over On Demand

Current robot policies often continue acting obliviously after execution goes wrong. We introduce DoOver, which couples lightweight, policy-native failure detection with on-demand recovery mechanism. With a $0.526$M-parameter head that learns jointly with the policy from timestamps in expert demonstrations and reuses policy's visual features, DoOver tracks changes in predicted progress to detect stalls and regressions. It requires no failure data, pretrained foundation models, separate detector training, or runtime action resampling. Across $40$ simulated tasks with flow-matching and diffusion backbones and $13$ detection baselines per backbone, DoOver leads the eligible baselines in aggregate average precision across task-calibrated detection, alarm-threshold sweeps, and threshold generalization, with $6,600\times$ lower failure-detection compute than the sampling-based detectors STAC and ACE. Beyond detection, we establish a systematic benchmark of failure-triggered recovery for frozen policies, comparing seven operators spanning five inference-time intervention families. Among them, failure-triggered feedback world model improves recovery rates over plain replanning by $6.8$/$5.1$ pp and over normal execution by $16.8$/$16.3$ pp for flow-matching/diffusion policies, while avoiding the high cost of guided sampling during normal execution.