- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Echo, a framework for training VLAs to understand the current status of task progress by jointly predicting task progress signals and actions. It represents task status using a stop signal (to detect whether execution should terminate) and a failure signal (to detect whether execution is in a failure state). To learn these signals, Echo trains VLAs with success, failure, pre-intervention, and recovery demonstrations. Echo also uses this signal during action generation by asynchronously denoising action and status, where it directly predict status signals, while denoising actions conditioned on the predicted signals.
Critical strengths
- This paper is well-motivated and addresses a practically important problem.
- The proposed method is simple yet effective.
- Real-world experiments and component-wise analysis demonstrate the effectiveness of the proposed method clearly.
Critical weaknesses
- The proposed method needs manually annotated subtask boundaries, intervention points, and categorized success/failure/recovery trajectories. It should include discussions of the practical cost of obtaining these annotations and whether the annotations can be obtained automatically.
- Echo compresses execution status into only two scalar signals, stop and failure. While this is sufficient for the evaluated tasks, more complex long-horizon manipulation may involve qualitatively different failure modes that require different recovery behaviors. It is unclear whether a single failure signal provides enough information to distinguish such cases and guide appropriate corrective actions.
Questions for authors
- It would be practical if the paper included analysis of the sensitivity to annotation quality, as manual annotation is required, yet the proposed method avoids dense frame-level progress annotations.
2 comments
Thank you again for your thoughtful review. We have provided detailed responses to your concerns, including additional experiments and analyses. As the discussion period is approaching its end, we would greatly appreciate it if you could take a look at our responses. We would also be happy to address any remaining questions or concerns.
Response to Reviewer 2QKt
Thank you for your constructive comments and questions. We address each concern below.
W1&Q1Annotation Cost, Automation, and Boundary Noise
Our data collection pipeline uses successful demonstrations, autonomous failure trajectories, and intervention-based recovery demonstrations. Subtask boundaries in successful multi-subtask demonstrations are manually annotated, requiring approximately 1–2 seconds per boundary, while single-subtask demonstrations require no manual segmentation. For recovery demonstrations, intervention timestamps are recorded automatically during teleoperation. Terminal failure labels are assigned at the episode level, without frame-level failure annotations.
Automatic subtask segmentation methods, such as NVIDIA FLORA, could potentially be incorporated to further reduce annotation effort. However, developing an automatic segmentation method is beyond the scope of this work.
We further evaluate robustness to boundary annotation noise on LIBERO-Recovery by uniformly perturbing successful subtask boundaries within ±0.2 s. The models are retrained and evaluated under the same protocol. As shown in Table 8, the perturbations have a limited effect on status prediction and task success across both backbones, indicating that Echo is reasonably robust to moderate boundary inaccuracies. The experimental details are included in the revised manuscript's Appendix, “LIBERO-Recovery Simulation and Additional Ablations.”
Table 8. Sensitivity to ±0.2 s boundary perturbations .
[W2] Expressiveness of Two Status Signals
The two task status signals are designed to represent whether execution is ongoing, successfully completed, or failed, rather than to encode the specific type of failure or the corrective action required.
Echo generates actions conditioned jointly on visual observations, language instructions, robot state, and predicted task status signals. Consequently, similar status signals can lead to different recovery actions depending on the current observations and task context. The ability to execute these corrective behaviors is learned from recovery demonstrations, while the predicted signals provide additional conditioning information during action generation. Our experiments in Section 4.4.4, “Effect of Failure Data Quantity,” further demonstrate the benefits of incorporating failure and recovery data into policy training.
Thank you again for your valuable feedback. We would be happy to discuss any further questions or concerns.