- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies local, precise peg-in-hole insertion under grasp uncertainty using proprioception alone. The policy uses a temporal convolutional encoder to process a 0.6-second history of joint states, gripper states computed through forward kinematics, and previous actions, and outputs joint-position increments. Training uses PPO with an asymmetric privileged critic and auxiliary contact-state supervision. At deployment, the policy receives no images, external force/torque or tactile measurements, hole pose, grasp pose, or part model. The authors also introduce mjfit, a procedural generator of convex collision geometry that preserves assembly clearances in MuJoCo.
The evaluation includes in-distribution performance, generalization to unseen geometries, analysis of the history representation, and real-world deployment of a frozen policy. In simulation, the proprioceptive and visual policies achieve success rates of 99.0% and 99.5%, with median insertion times of 1.05 s and 0.36 s, respectively. Across five real-world parts, the deployed policy succeeds in 112/151 trials, approximately 74.2%.
The task's scope is important: trials begin with the peg already touching the fixture near the hole. Furthermore, the modality-comparison policies and the deployed policy use different training randomization ranges. The paper therefore demonstrates a local insertion skill under restricted sensing, rather than global hole localization from arbitrary initial configurations.
Critical strengths
- The real-world demonstration under restricted sensing is a substantive contribution. The paper provides concrete evidence that a policy can use its own motion feedback to search for and insert into an opening with millimeter-scale clearance, without directly observing the hole pose, grasp pose, or external contact measurements. The real-world evaluation covers four stadium-shaped parts of different sizes and proportions, an unseen circular cross-section, and uncertainty in grasp position and tilt. The 112 successes over 151 trials make this more convincing than isolated demonstrations with a fixed part and grasp.
- mjfit is an independently valuable simulation contribution supported by targeted validation. It addresses the problem that collision-geometry approximation errors can change the outcome of tight assembly. It constructs convex geometry that preserves the intended clearance rather than relying on generic approximate decomposition. Section 2.2 and Appendix B evaluate it using passive probes with analytic answers and report CPU/GPU speed–accuracy trade-offs. The benchmark holds solver settings fixed, accounts for the method's own curve-tessellation error, and explains how solver compliance limits the resolution of some probes.
Critical weaknesses
- The modality comparison does not sufficiently control training-time information, limiting the attribution of the generalization advantage. Section 2.3 states that the proprioceptive policy uses a privileged critic with access to relative peg–hole pose and true contact labels, together with auxiliary contact supervision for its history representation. In contrast, the visual policy's critic observes only images, and the manuscript does not clearly establish whether the visual representation receives equivalent auxiliary supervision. Thus, although the task, resets, randomization, rewards, and actions are matched, the training-time information is not. The experiments support the narrower conclusion that the proposed proprioceptive policy outperforms the implemented visual baseline on the tested geometry shifts; they do not isolate sensing modality as the cause. Similar in-distribution success does not rule out effects of the training setup on out-of-distribution generalization.
- The task-level benefits of the key learning components are insufficiently established. The representation analysis is useful: the history latent achieves 70% balanced accuracy on seven insertion modes, compared with 57% from the current observation, and the latent-swap experiment shows that the representation affects actions. However, these analyses do not directly quantify how history and contact supervision improve task success, learning efficiency, or transfer.
Questions for authors
- Why does the proprioceptive policy use a privileged critic while the visual policy uses an image-only critic? Does the visual representation receive equivalent contact-state supervision? With training-time information matched and training budgets specified, does the generalization gap on extreme sizes and random convex cross-sections remain?
- Appendix A specifies a 5 s simulation episode limit, whereas Figure 1 includes a successful real-world trajectory with a final frame at 5.5 s. What are the real-world timing origin, timeout threshold, and operational definition of "seated"?
- Section 2.3 states that armature, damping, and friction were identified from real-world measurements, while Section 3.3 states "no real-world data or fine-tuning." Were real-world data used only for dynamics identification, or also for checkpoint selection, controller adjustment, or other decisions?
2 comments
I acknowledge the author's re matching of visual strategy training information and the addition of task level ablation with historical input and contact supervision. I maintain a score of 3 (weak acceptance) and request the author to clarify in the final draft the stopping rules for evaluation failure due to lag, as well as the training budget and statistical criteria for the new experiments.
We agree that the “image-only” critic is a potential confounder, and have re-trained the camera-based policy, giving the critic access to the exact same privileged information as the proprioceptive policy (now, the two policies differ only in the actor’s observations) and updated Figs. 7 and 9, as well as Tab. A4, accordingly. The throughput (Fig. 7) and robustness (Fig. 9) trends both remains similar with the privileged critic. We have also clarified in the manuscript that the camera-based policy is indeed trained with the same contact-supervision loss as the proprioception policy.
To address this concern, we run two sets of ablation experiments (1) removing information from the observation with the same architecture, and (2) removing components from the architecture (contact supervision, temporal encoder) with the same observation.
We see this task as an indefinite horizon RL problem — the 5s timeout during training is there to speed up convergence. During evaluation, we allow the policy to run until success or termination failure (dropping the peg, or traveling further than a threshold distance away from the initialization) both in simulation and the real world. In the real world, a peg is seated if the entire bottom face of the peg is flush against the entire bottom face of the hole (this is easy to detect visually). Any rollout where the peg becomes jammed and remains only partially inserted is considered a failure.
We have revised the language in Section 3.3 to say “without fine-tuning the policy on any real-world data”. Dynamics identification is the component of our system that requires real-world data (position response of the robot arm moving in free space). For checkpoint selection, we deployed the best checkpoint (measured by training time success rate in sim).