- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CASH, a framework for detecting and recovering from failures of frozen action-chunking VLA policies. The authors distinguish two types of failure: execution failures, in which the executed actions do not produce the expected state transition, and policy failures, in which the proposed action plan itself is wrong. Following this motivation, the authors propose two modules, the Execution Anomaly Detector (EAD) and the Policy Anomaly Detector (PAD). EAD checks whether the observed latent transition is consistent with the change that the executed actions should have produced. PAD is a value model that judges, before execution, whether a proposed action chunk is plausible under the current state and instruction. EAD is built on a latent forward–inverse dynamics model trained with cycle consistency, and both detectors are trained only on successful trajectories. Through experiments, the authors show that the proposed detection and recovery improve success rates, and that the detection modules detect failures better than existing baselines.
Critical strengths
- The paper tackles an important yet challenging problem: recovering from failures without failure data or recovery demonstrations.
- The motivation for decomposing failure handling into two separate modules is clear.
- The experiments support several of the main claims. The detection module raises alarms earlier than the baselines and achieves higher failure recall, and its advantage remains after the length bias is removed (AUC_H4 in Table 2). Using both modules together yields the highest success rate in Table 3. Figure 3 also shows why the inverse head is needed, as it increases the action sensitivity of the forward model.
Critical weaknesses
-
Heuristic recovery strategy. The failure detection method itself is reasonable, and decomposing it into two modules is a good design. However, the recovery procedures, in particular the PAD path (foreshortening to N_v steps and a window reset triggered by a fixed event count), rely on hand-designed rules. The paper would benefit from further explanation of these design choices.
-
Readability of Figure 1. Figure 1 is difficult to read. Text elements overlap and the overall layout is complex, so the figure needs improvement.
-
Link between action sensitivity and performance. Figure 3 shows that the inverse head increases the action sensitivity of the forward model. However, the paper does not show whether this improvement translates into better detection or recovery, for example by reporting the detection AUC or success rate of the forward-only variant.
-
Contribution of the privileged reset. In Table 3, the PAD configuration combines foreshortening and the window reset. The reset is privileged: it writes the initial joint state directly to the simulator, and its steps are not counted against the episode budget. It is therefore unclear how much of the +9.5 pp gain from PAD comes from the reset, and no ablation separates the two.
-
Lack of qualitative results. The paper provides no videos or snapshots of the recovery process, which makes it difficult to see how the method operates in practice.
Questions for authors
-
The recovery strategy, especially the PAD path, appears heuristic overall. Could the authors explain the reasoning behind these design choices?
-
I suggest improving the readability of Figure 1.
-
Could the authors provide an ablation that separates foreshortening and the window reset in the PAD path, so that the contribution of the privileged reset is clear?
-
Could the authors provide qualitative results, such as videos or snapshot sequences of the recovery process?
-
The title refers to “embodied agents,” but the method and experiments focus on runtime monitoring and recovery for action-chunking VLA policies in simulated tabletop manipulation. I suggest revising the title, or clarifying in the paper, so that the stated scope matches the evaluated setting.