- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper focus on an interesting and challenging task. The main contributions include a risk-aware diagnostic benchmark and a benchmark-derived adaptation framework. The benchmark demonstrate that the gentle retrieval failure mainly comes from three interaction-state variables. And SRAH framework employs the interaction states on the frozen policy for residual correction, where the failed trajectory is used for instructing the policy what "danger looks like". The paper provided relatively comprehensive validation, but may contain some overclaims and presentation issue.
Critical strengths
- This paper aim to address the gentle retrieval through yielding occlusions, which is an interesting, practical and challenging task in robot manipulation. Therefore, such benchmark and further adaption framework is timely and meaningful at current stage. The benchmark, experiments, and adaption methods are overall reasonable and relatively comprehensive.
- The author's design of the risk metric in the benchmark was reasonable and verified through manual annotation. Even though it contained pre-defined weights, the scanning of weights also reduced my concerns about sensitivity.
Critical weaknesses
- The benchmark itself appears novel in its specific setting, but the paper does not sufficiently position its contributions against recent deformation-aware and contact-rich manipulation benchmarks (e.g. SoftVTBench, MoDeSuite, Tabero). Adding a discussion on such works would help reduce some overclaiming and focus on the core contributions.
Questions for authors
- Did you use a separate development set to derive the GRAM findings that motivated SRAH? The paper clearly separates train/val/test for model fitting, but it is unclear whether the SRAH design was informed by the final evaluation set
- Does hazard gating actually prevent budget violations? Fig.4 shows that the gate decreases near risky events, but this does not mean that those events are avoided. A controlled comparison with and without hazard gating would help clarify
- Is there a plan to expand the coverage of the benchmark? Currently, only two types of gentle retrieval scenarios are being discussed
- Expressions are highly repetitive and overly defensive. The introduction repeatedly summarizes contributions, which makes the paper unnecessarily difficult to read
2 comments
Thank you for acknowledging our clarifications and experiments, as well as identifying ways we could strengthen the research. This is a lot of work and we're really grateful for it!
Thanks for authors' rebuttal, especially for the additional experiments and contribution clarification.
However, considering the relatively narrow coverage of the benchmark -- only two complementary retrieval scenarios. I cannot give a higher rating than current evaluation, therefore will maintain my original rating.
Overall, I would encourage authurs enrich this research with more comprehensive evaluation in more diversified "Gentle Retrieval through Yielding Occlusions" scenarios.