- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper studies reward learning for contact-rich manipulation, where correct insertion and misalignment may look nearly identical in images. It proposes a Bayesian-privileged reward learning framework that trains a dual-view visual reward model with privileged pose and contact labels. A risk estimator is applied to detect high-risk visual positives. The experiments show that the framework suppresses most negative-to-positive errors while retaining correct positives. Adding the learned reward to PPO also raises aggregate success.
Critical strengths
- The paper addresses a concrete failure mode of image-based rewards in contact-rich tasks. Utilizing privileged physical informationis is well motivated and convincing.
- The proposed history-conditioned risk estimator is a practical complement to the visual model. It uses executed action history, which is available at deployment, to selectively downgrade unreliable visual positives.
- The evaluation connects reward quality to downstream deployment. It reports false-positive suppression and positive retention on Factory tasks, a matched PPO comparison, and student-teacher agreement on real Franka Peg Insert trajectories.
Critical weaknesses
- The paper lacks substantial comparison with other methods. Although ablation studies are included, there is no comparison against other visual reward models or against methods that also use privileged physical information. As a result, the advantage of the Bayesian-privileged reward learning framework is not sufficiently demonstrated.
- The gain of downstream policy is small, and absolute performance remains low. With no reported performance of other reward models under the same setting, it remains unclear how much the proposed framework could help downstream policy learning.
Questions for authors
- Could the authors compare the framework with representative visual reward models and with methods that also train on privileged physical information, under the same Factory tasks and the same PPO setup?
- The matched PPO success increases from 9.7% to 13.5%. Is this difference consistent across random seeds? If most negative-to-positive errors are already suppressed, why does aggregate success stay near 13%?