- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes Center of Pressure (CoP) as an intermediate tactile representation for sim to real RL. CoP consists of a normal force vector and a contact location per fingertip, chosen to sit between coarse binary contact and raw taxel arrays. A parametric stress distribution model gives a differentiable bidirectional mapping between taxel forces and CoP, with the CoP force recovered in closed form by regularized least squares. Unknown taxel orientations are calibrated by matching the wrench inferred from tactile readings to measured joint torques under static equilibrium, using differentiable kinematics and requiring no ground truth force sensor. Policies are trained with PPO in IsaacLab and transferred zero shot to Allegro hand with XELA uSkin, on two blind tasks including peg in hole insertion and ball balancing.
Critical strengths
- The representation is a well chosen abstraction and the chosen tasks and evaluation are strong enough to demonstrate its effectiveness.
- The calibration method is a practical contribution, estimating taxel orientations by matching tactile inferred wrench against joint torques under static equilibrium removes the dependence on a high precision force sensor.
Critical weaknesses
- The abstraction loses information from raw sensory data, since only the surface normal component is used and shear force is not used, while shear carries slip and tangential alignment information required for insertion.
- The calibration reliability and accuracy is not measured and reported.
Questions for authors
- What is the computational cost of the taxel to CoP mapping at inference?
- The masked sensor result shows CoP degrading more than simplified representations. Does a representation at intermediate fidelity exist that retains CoP's performance while matching binary's robustness, for example by including the active taxel count or a confidence term alongside CoP?
We thank the reviewer for the constructive comments. Both weaknesses are fair, and the second prompted new experiments, reported below.
W1. Only the normal component is used, and shear carries slip and tangential information. We agree this is a real limitation, and we state it in Sec. 3.4 and Sec. 6(B). Two clarifications. First, the restriction comes from the simulator, not the representation. The stress-distribution model includes shear (Sec. 3.2), and the calibration data excite shear responses (Sec. 3.3). We drop the tangential component because IsaacLab's shear estimates were unreliable for our fingertip geometries. A channel that disagrees between simulation and hardware would undermine the alignment the representation exists to provide. Second, the results show how far a normal-only signal goes: 78% blind insertion across six shapes, and 63% under OOD initialization. The policy is not blind to tangential interaction, since normal forces at several differently oriented fingertips, with contact locations and joint positions over time, partially constrain the net wrench on the peg. We expect shear to help further, particularly for slip, once simulated shear is reliable. We have made the normal-only scope explicit wherever the paper says "3D force vector" (Sec. 1, Sec. 3.1).
W2. Calibration reliability and accuracy are not measured. We agree, and have run two additional evaluations on the real-hardware calibration recording consisting of 2,400 frames. Full per-fold, per-joint and per-taxel results are in Appendix A. Accuracy. We ran 5-fold blocked cross-validation. Folds are contiguous time blocks with a 10-frame purge gap, so temporally adjacent frames never fall in both training and test sets. The joint-torque RMSE on held-out contacts is 0.266 ± 0.022 N·m, close to the 0.260 ± 0.006 N·m on training contacts, so the calibration does not overfit. Reliability. We ran 50 independent calibrations, from 5 data subsets (the cross-validation training sets) and 10 random initializations each, and measured the geodesic angle between learned orientations for each taxel. Across initializations on the same data, the median per-taxel spread is 0.9°, and 27 of 30 taxels stay below 10°. Across data subsets, the median spread is 3.2° and the maximum is 28.2°. Two taxels converge to different orientations from different initializations, with spreads of 33° and 52°. Held-out RMSE nevertheless varies by only 0.001 N·m (std) across initializations, so these two orientations are weakly identified by the torque objective and do not affect torque prediction.
Q1. Computational cost of the taxel-to-CoP mapping at inference. The mapping is closed-form. Each fingertip needs one weighted average for the contact position and one regularized 3×3 solve for the force. Cost is linear in the number of active taxels, with no iterative optimization and no learned encoder. It takes 0.15 ms on average and at most 0.4 ms per step for all four fingertip sensor arrays, against a control period of 50 ms. We now report this in Sec. 3.2.
Q2. cop degrades more under masking. Could an intermediate representation keep cop's performance with bin's robustness, for example with an active-taxel count or a confidence term? Two observations first. Under 40% masking, cop loses the most relative to its own unmasked result (78% to 62%), but it remains the highest in absolute terms: 62%, against 57% for vec, 52% for bin and 30% for taxel. The gaps to vec and bin are within trial noise, so we claim only that cop is no worse. Also, taxel masking is not among the training randomizations (Table 6), so this is a zero-shot test under an unseen corruption. The reviewer's suggestion fits our framework well. The mapping already yields two confidence signals per fingertip: the active-taxel count and the least-squares residual of the CoP fit. Because the mapping is bidirectional, we could also train with taxel dropout in simulation: simulated CoP to taxel forces, random masking, then back to CoP. The policy would then see CoP errors with the same structure as on hardware. We have added this to Sec. 4.1 as a promising future direction.