- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper introduces Task-State Feedback (TSF), a framework for closed-loop object-centric manipulation. TSF estimates statistical features from successful demonstrations, capturing how interaction relations and task progress correlate with the physical scene. At deployment, it jointly infers relation and progress beliefs, computes an absolute compatibility score, and uses this estimate to regulate execution: hold, advance, or realign the task reference; repair a persistent relation mismatch using demonstrated LINK/UNLINK events; or search for a legal re-entry state. The method is designed for both single-arm and bimanual manipulation and does not require failure trajectories.
Contributions
Task-state alignment: Formulates execution as alignment to demonstrated task states, distinguishing valid timing variation from invalid transitions.
Structured state inference: Jointly infers interaction relations and task progress from action responses and scene evidence.
Unified closed-loop execution: Integrates relation repair and verified re-entry, claiming improved perturbed-task success without failure data.
Validation
Evaluated on six single-arm RLBench tasks and four bimanual RLBench2 tasks, using a frozen DynaMAC policy, five successful demonstrations per task, and low-dimensional simulator states, with 200 paired episodes per task and condition. Additional experiments cover monitoring, recovery stages, unseen failure families, severity and two-event compositions, longer horizons, native-system comparison with RVT and RACER, and ablations. Results report improved perturbed success and aggregate nominal non-inferiority, with gains mainly from relation repair and legal re-entry.
Critical strengths
The paper is well organized, and the figures and tables are generally clear and readable. The problem setting—multi-stage manipulation with execution monitoring and recovery—is relevant, and the overall system is described in sufficient detail to understand the pipeline.
Critical weaknesses
-
The paper claims to address deployment disturbances but only evaluates in simulation using exact, temporally aligned, low-dimensional states with known correspondences. It explicitly excludes perception error, has no mechanism for ambiguous correspondences, and lacks real-robot validation. The claim is not supported by the evidence.
-
The core machinery consists of HMM-style Bayesian updates, a single-Gaussian Mahalanobis distance, and a weighted geometric mean. In this idealized setting, closed-form template matching is sufficient, so the experiments do not demonstrate algorithmic necessity. Moreover, the paper positions “recovery without failure data” as a central contribution, but this is not novel.
Questions for authors
-
The paper claims deployment robustness, but the evaluation assumes exact, temporally aligned states with known correspondences and explicitly excludes perception error, with no real-robot validation. How do the authors reconcile this claim with the evidence?
-
Eq. 7 uses a single Gaussian per scene factor, while Appendix A.1 acknowledges alternative motion modes. For multi-modal demonstrations (e.g., different grasp orientations), how does TSF avoid misspecification? Is it restricted to unimodal tasks, and if so, where is that stated?
-
Given exact states and known correspondences, could a simple nearest-neighbor template matcher with threshold-based relation repair perform similarly? The nearest-state ablation still retains the full recovery interface. Can the authors provide a controlled comparison to justify the full TSF machinery?
-
The paper claims “recovery without failure data” and generalization to unseen failures, but FAIL-Detect and Sentinel already monitor without failure data, recovery is within demonstrated support, and Figure 4(b) mainly tests statistical deviation. What is the unique contribution, and how can TSF handle physically recoverable states not in the demonstrations?