acceptodds
Under review as a conference paper at ICLR 2027

R²D-Grasp: Reward-Refined Diffusion for Physically Reliable Dexterous Grasping

Abstract

Existing dexterous grasp diffusion models can learn plausible and diverse grasp distributions from pose supervision, but they struggle to distinguish subtle pose variations that lead to different physical contact outcomes. We propose Physics-Selective DDPO, which first pretrains a grasp DDPM and then generates local candidates by branching from shared intermediate denoising states. Simulation outcomes are used to construct group-relative advantages and optimize physically sensitive denoising decisions, while the remaining stages preserve the pretrained generative behavior to improve grasp success without sacrificing grasp priors or diversity. Experiments on MultiDex and RealDex demonstrate improved grasp success and cross-object generalization.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Dexterous Grasping, Diffusion Models, Physics-Guided Reward Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_KzKFOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes R²D-Grasp, a method for post-training a pretrained, wrist-rotation-conditioned grasp diffusion model (DDPM) with binary success rewards from a non-differentiable physics simulator (Isaac Gym). Following TreeGRPO, the reverse diffusion process is split at a fixed step τ into shared denoising prefixes and branched suffixes. A between-prefix advantage is assigned to prefix transitions, and a between-prefix plus λ-weighted within-prefix advantage is assigned to suffix transitions, both optimized with a PPO objective. At inference, an external wrist-proposal module generates candidate wrist orientations, which are ranked by a training-free geometric energy; the diffusion model then generates the wrist translation and finger configuration for the retained orientations. The method is evaluated in the same simulator used for post-training, and on four grasp datasets.

Critical strengths
  • Physics-based post-training of grasp diffusion models with a non-differentiable simulator is a relevant and practical problem. Pose-supervised grasp generators do not optimize execution outcomes, and the paper shows a large improvement from post-training (Table 2: 68.0 to 85.3 on UniDexGraspNet, 55.0 to 85.1 on RealDex over the supervised DDPM), together with a smaller distribution shift than full-trajectory DDPO (KL 0.12 vs 0.15) and a lower invalid pre-grasp rate (2.5% vs 10.8%).
  • The paper evluates on four datasets, two execution protocols, a zero-shot transfer experiment, and ablations that separately vary the post-training strategy (Table 2), the wrist candidate source (Table 4a), and the yaw selection rule (Table 4b). The shared-prefix diagnostic (Fig. 3) is a useful sanity check that late denoising steps still change the physical outcome.
Critical weaknesses
  1. Novelty over TreeGRPO is unclear, TreeGRPO is not a baseline, and the gain over DDPO is marginal. The sampler is TreeGRPO with a single branch point (M = 2 prefixes, K = 4 suffixes). TreeGRPO already normalizes rewards within a group and propagates them up the tree, so the global/local split in Eq. 19 and the cancellation in Eq. 6 follow directly from it. The only differences are separate normalization of the two parts with a weight λ, training the prefix step and uniform instead of probability-weighted averaging. The introduction's claim that TreeGRPO applies one combined advantage across the whole trajectory contradicts TreeGRPO's own description of step-specific advantages. Despite this, TreeGRPO is not compared against, and λ and the branch step τ are never reported. This matters because the measured effect is small: in the only controlled comparison (Table 2), R²D-Grasp beats full-trajectory DDPO by 1.7 and 2.4 points with no error bars or seed count, and Fig. 3b shows the gain over DDPO ranging from roughly zero to about 3 points depending on the branch ratio.
  2. Most of the headline gain comes from components that are not the paper's contribution, and the evaluation setting is narrow. Table 4a shows the UniDexGrasp wrist predictor adds 17.4 and 21.6 points and the geometric ranking another 4.0 and 6.0, while the proposed credit assignment adds 1.7 and 2.4 (Table 2); the large jump over the supervised model is already obtained by plain DDPO. The geometric energy is only needed because the wrist orientation is excluded from the diffusion model, and the given justification (one rotation supports several translations) applies equally to a model that generates the rotation as well. Table 1a thus compares a multi-component pipeline against single end-to-end generators. The tabletop setting fixes palm-down orientation and wrist position, so wrist selection reduces to picking 4 of 16 yaw angles for a single settled object with an open-loop approach, while the dynamic baselines run closed-loop pipelines; gradient-based wrist steering is not compared. All results are in Isaac Gym with no sim-to-sim or real-robot transfer, and no table reports standard deviations. The premise that early steps fix the grasp mode and late steps refine contacts is not directly tested; Fig. 3 only shows that late steps can still change the outcome.
Questions for authors
  1. How does TreeGRPO (depth-1 tree with standard group normalization and leaf-to-root aggregation) perform under the same rollout budget as Table 2? This is the most important missing comparison.
  2. How many seeds were used for Tables 1 to 4, and what is the standard deviation? Fig. 3b reports five runs with error bars; the tables report none.
  3. Which wrist candidate source do the Table 1a baselines (GraspTTA, DexGraspAnything) use? If they do not receive the wrist predictor and geometric ranking, Table 1a conflates backbone, wrist handling, and post-training.
  4. Why is the wrist orientation not diffused jointly with the translation and finger configuration? Was this tried?
  5. Why M = 2 shared prefixes? With M = 2 the global advantage carries no magnitude information. Were larger M evaluated?
  6. What values of λ and τ were used, and how sensitive are the results to them?
Reviewer_qECgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes R²D-Grasp, a physics-feedback post-training framework for dexterous grasp diffusion. It employs shared-prefix denoising trees and separates global and local reward advantages to refine grasp generation. A geometric wrist-selection mechanism is additionally introduced at inference. Experiments across four datasets demonstrate improved grasp success and zero-shot cross-dataset generalization.

Critical strengths
  1. Well-motivated reward refinement. The distinction between global grasp-mode selection and local contact refinement provides a reasonable interpretation of the diffusion process.

  2. Promising results on the evaluated benchmarks. The method achieves consistent improvements over the selected baselines across four datasets.

Critical weaknesses
  1. Limited methodological novelty. Shared-prefix tree sampling and reward-based diffusion optimization have been explored in prior work. The proposed global/local advantage decomposition is intuitive, but its fundamental distinction from existing tree-based optimization approaches is insufficiently established.

  2. Missing comparison with closely related optimization methods. Although the paper builds on shared-prefix sampling principles from recent work, it does not provide a direct comparison with corresponding tree-based reward optimization methods under matched sampling and computational budgets.

  3. Insufficient comparison with recent state-of-the-art methods. The experimental comparisons primarily involve methods published up to 2025, despite the availability of relevant 2026 research on physics-guided dexterous grasp optimization.

  4. Incomplete advantage decomposition ablation. The paper compares full-trajectory DDPO and local-only refinement, but lacks a global-only variant. Consequently, the independent contribution of local suffix advantages remains unclear.

  5. Insufficient validation of the hierarchical credit-assignment assumption. Although different branching ratios are examined, the paper does not directly establish that early denoising steps consistently govern grasp modes while later steps primarily determine local contacts, or that the proposed advantage allocation is optimal across different branch positions.

  6. Limited robustness analysis. The approach rotation is fixed by wrist-candidate selection, but the consequences of missing a viable rotation are not analyzed. The sensitivity of geometric-selection hyperparameters and the distribution of failure modes also remain unclear.

  7. Missing computational analysis. The paper does not sufficiently quantify the cost of simulation-based post-training or the inference latency of the complete pipeline.

Questions for authors
  1. What fundamentally distinguishes the proposed advantage decomposition from existing tree-based diffusion reward optimization methods? Can the authors provide a matched comparison?

  2. Can the authors include a global-only advantage baseline to isolate the contribution of local refinement?

  3. Can the authors compare against relevant 2026 methods under a unified evaluation protocol, or explain why such comparisons are not feasible?

  4. How does the proposed credit assignment perform under different branch positions, and what evidence supports the assumed separation between grasp-mode and contact-level decisions?

  5. Can the authors provide controlled tabletop comparisons using the same execution controller?

  6. How sensitive is performance to wrist-candidate quality and geometric-selection hyperparameters?

  7. What are the post-training computational costs and full-pipeline inference latency?

Reviewer_AH4xOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose reward-refined diffusion (R2D-Grasp) to post-train a dexterous grasping diffusion generative model with physics feedback from simulation. To achieve this, they propose a local branch comparison mechanism and a wrist pose selection module. They demonstrated that their method outperforms certain baselines in the simulator. Although the idea is interesting, the overall presentation of this paper does not seem publication ready, and experimental validations can be improved.

Critical strengths

Strengths

  1. The idea of using physics-feedback as an RL reward to post-train diffusion dexterous grasp models is interesting.
  2. The RL formulation does not require human-tuned dense rewards.
Critical weaknesses

Weaknesses

  1. Paper presentation is not publication ready. Figure 3 (b) has different font, and the numbers above the bars are partially occluded by the error bars. Line 364 has a duplicated sentence. Equations 3 and 6 are broken into multiple lines without clear reasons. Table 2 is never explicitly cited by the main text, and the local-only refinement line seems to be missing some entries. The paper seems to be under-polished and written in a rush.
  2. Experimental comparisons do not include the newest baselines. For example, UniDexGrasp has a clear followup UniDexGrasp++, DemoGrasp has a clear followup DemoFuncGrasp. Comparing against and outperforming newest baselines can make the results more compelling and prove that RL-post-training is critical to improve grasp quality.
  3. No real-robot experiments.
Questions for authors

Questions:

  1. In Table 4, the authors ablated the wrist selection protocol towards a random strategy. However, have the authors tried letting the DDPM produce the writst pose together with the joint angles? How much is the benefit of using a separate module to produce the wrist pose?
  2. Why does the local-only refinement row in Table 2 do not contain the final three entries?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.