- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Joint-Force Matching (JFM), an auxiliary objective that uses contact mechanics to improve RL critic representations for dexterous grasping. JFM maps measured contact forces to joint loads through the hand’s kinematic Jacobians and trains a scalar energy function whose negative joint-angle gradient matches these loads. By sharing a backbone with the critic, this objective encourages mechanically informed features while retaining standard force-prediction and RL losses. Experiments on simulated LEAP-hand grasping show improved agreement with demonstration returns compared with force prediction alone, while a direct joint-load predictor achieves lower auxiliary loss but worse mean critic return error. Policy improvements are less conclusive: matched offline policies perform similarly, whereas a separate offline-to-online setup shows greater grasp coverage for JFM.
Critical strengths
- The idea of connecting contact forces with joint loads and use the signal to shape energy head is very interesting.
- The free-vector ablation is helpful. A vector head trained on same joint-load targets fits more accurately, yet produces worse mean critic return error than JFM. This supports the benefit of the energy-gradient constraint.
- Across all three training seeds, JFM’s value estimates are closer to the recorded demonstration returns than those of the force-prediction baseline.
- JFM retains its critic advantage when the predicted joint-load input is removed at evaluation, suggesting that the benefit is retained in the learned model rather than depending solely on that extra input.
Critical weaknesses
- Limited policy improvement under matched training. JFM covers 149 of 158 initial conditions versus 148 for the force-prediction baseline, providing little evidence that better critic estimates translate into better grasping performance.
- A comparison with a separate energy network, or a variant that prevents the energy loss from updating the shared backbone, would help establish whether shaping the critic’s representation is responsible for the gains.
- Missing metric. An initial condition counts as solved if any of 32 attempts succeeds, potentially masking low per-attempt success. The choice of 32 attempts should be justified, and average success rates over attempts with confidence intervals should be reported.
- Insufficient proof of generalizability. Experiments use one simulated hand, and training and validation contain similar demonstrations. Performance on unseen objects, other embodiments, and real hardware remains untested.
Questions for authors
- Can you compare against a separate energy network or a variant that blocks the JFM loss from updating the shared backbone to isolate the benefit of representation shaping?
- Why was coverage defined using 32 attempts per initial condition? Can you report aggregated success rates for each initial condition with confidence intervals too?
- Can you provide evaluation on held-out objects, other hand embodiments, and on real world hardware?
Reply to Reviewer 2juz: Higher Grasp Success, Gains from the Shared Representation, and Generalization
We thank Reviewer 2juz for recognizing JFM's value, including its energy-gradient constraint and its value estimates, which are significantly closer to the demonstrated returns than those of FP (force prediction).
JFM's policy does grasp more often than FP's under matched training: over three training seeds, its success rate (SR) per attempt is 55.0% against FP's 52.5% at H=50 (p=0.005; H: policy steps before the object's support is removed). Table 3 notes that JFM's and FP's policies cover 149 and 148 of the 158 initial conditions at H=100, near the ceiling of the coverage test. We additionally evaluated the SR per attempt, which separates them: all 32 attempts on each initial condition count, for all three training seeds (%, mean ± std [95% CI over initial conditions]):
JFM is the highest at both horizons. Paired on the same initial conditions and seeds, it is above FP at H=50 (p=0.005) and above FV at H=100 (p=0.012). The method with the better value estimates is also the one that grasps more often.
Shaping the critic's representation is responsible for the gain: in the comparison that separates the two, JFM's gain over FP disappears once the JFM loss is blocked from the shared backbone. Sec. 5.4 (Table 4) zeroed the predicted joint-load input of JFM's critic at evaluation: its return error (distance between value estimates and demonstrated returns; lower is better) stayed below FP's at all three seeds (6.30/7.55/7.37 against 6.58/7.76/7.48), so the gain lies in the trained critic. We additionally ran the comparison: we blocked the JFM loss from the shared backbone, which produces the critic's representation, so the loss still trains the energy network, which fits the joint loads (matching loss 1.4–1.6×10⁻³; JFM 1.1–1.3×10⁻³), but no longer shapes that representation; all else is unchanged. Return error on Table 1's 8,000 validation samples:
Without that shaping, the return error is the highest at every seed, and its excess over JFM's has a 95% CI excluding zero: the gain comes from that shaping. At seed 43 the blocked critic diverges on 201 samples; on the other 7,799 its error is 8.32 (JFM 7.60, FP 7.84).
The SR per attempt behind the coverage is 52–62% for every method at both horizons (table under item 1, with 95% CIs). For each of the 158 initial conditions (96 attempts each, Wilson 95% interval within ±10 points), the median SR is 74.5/69.8/69.8% (JFM/FP/FV) at H=100 and 61.5/54.2/60.4% at H=50, highest for JFM at both horizons.
32 attempts make coverage a reliable test: an initial condition from which a policy succeeds in 9% or more of its attempts is counted as covered with probability ≥ 95% (1 − 0.91^32). It is also the number of parallel environments in our simulation (App. B).
These points ask how far JFM extends beyond the tested hand and objects, a question for future work; they should not be grounds for dismissing JFM's value. The paper asks what in contact forces improves the critic's value estimates (Sec. 1) and answers by changing the supervision on the same hand, 71 objects and data (Sec. 5.2–5.4). For this question, simulation is the clearer design: it gives exact forces, Jacobians and returns, so differences between methods trace to the supervision; IQL (ICLR 2022) and SPR (ICLR 2021) are likewise evaluated in simulation. The result holds across 71 objects; unseen ones test transfer, a separate question from which supervision improves the critic. JFM is not tied to this hand: it maps contact forces to the joints through the hand's Jacobians, a construction that applies to any articulated hand (Sec. 3.1, 6). Each of these extensions builds on the supervision this paper identifies.
This similarity does not explain JFM's advantage: each validation demonstration is held out from training, one per combination of object, pose and variant (Sec. 5.1), and all methods share this split, so similarity affects them equally.