- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the problem of inconsistent subtask completion boundary annotations in hierarchical robot policy learning and proposes a Consistency-Aware Subtask Decomposition (CASD) method. Specifically, the method first generates candidate subtask boundaries using robot kinematic information and constructs segment representations by integrating pretrained vision-language-action (VLA) features, robot states, and action information. It then learns skill identity and completeness from a small set of human-annotated reference demonstrations, categorizing candidate segments as full, partial, or mixed skill executions. Finally, a structured loss and dynamic programming are employed to generate subtask labels and temporal boundaries that cover entire demonstration trajectories. Extensive experiments across multiple datasets validate the effectiveness of the proposed method, demonstrating competitive performance.
Critical strengths
- The paper presents a relatively clear research motivation by addressing the inconsistency in subtask completion criteria across demonstrations. It proposes a specific method to establish a unified annotation convention, which can lead to significant improvements in subsequent policy training.
- The proposed method is well-motivated and relatively comprehensive. It enables the model not only to recognize the skill performed within a segment but also to determine whether the segment represents a complete execution of that skill.
Critical weaknesses
The paper presents a relatively clear motivation, but its technical novelty is somewhat limited. Although the paper addresses the practically meaningful problem of inconsistent subtask completion criteria, the main components of CASD, including kinematics-based candidate boundary generation, MLP-based skill classification, structured loss, and dynamic programming, are largely based on established techniques. The overall framework appears to be more of a combination of existing techniques rather than a method with a clearly distinguished technical innovation. Furthermore, given that the paper primarily addresses a data annotation consistency problem, it is worth questioning whether a simpler and more efficient approach could achieve the same objective without relying on such a complex combination of technical components. 2. Lack of comparison with simpler rule-based segmentation methods. To address inconsistent subtask completion boundaries, could unified skill completion rules be directly defined based on observable physical events, such as gripper closure, object release, or end-effector retraction, and applied across the entire dataset? Such an approach might be simpler and more efficient, without requiring an additional completeness scoring model. The authors are encouraged to include comparisons with such rule-based methods to justify the necessity and practical advantages of the proposed complex learning framework.
Questions for authors
- The proposed method relies on a small set of human-annotated reference demonstrations to learn a unified subtask completion criterion. However, it remains unclear whether these reference annotations are sufficiently consistent. The paper mentions that two annotators contributed to the reference dataset. If different annotators disagree on the completion boundaries of the same skill, how does CASD ensure that the learned completion criteria are reliable and consistent?
- Does complete skill execution necessarily imply successful skill completion? If a skill execution ultimately fails, how does CASD distinguish between the completeness of the action execution and the actual success of the skill?
- Is a unified subtask completion criterion always appropriate? Inconsistent completion criteria across different demonstrations do not necessarily imply that all demonstrations should follow the same completion convention. The appropriate completion timing may depend on the specific task context and subsequent actions. How does CASD account for such context-dependent variations?
Reminder: Please Review Authors’ Rebuttal and Provide Feedback
Dear Reviewers,
Thank you for your reviews. If you have not already done so, please read the authors' rebuttal and share your feedback in the discussion thread at your earliest convenience. Your timely responses would be greatly appreciated.
Best,
AC