acceptodds
Withdrawn from ICLR 2027

StableGrasp: Reconstructing Physically Stable Human Hand Grasps from Single Images

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reconstructing a physically stable human grasp from a single RGB image is challenging because physically modeling grasps is itself difficult, and the problem requires recovering not only a visually constrained hand pose but also the underlying physical hand control. Existing methods either model only visual hand geometry without considering physics, or rely on less plausible physical modeling, which limits the physical validity of the resulting grasps. In this paper, we present StableGrasp, a differentiable simulation-based optimization framework that explicitly separates the visual hand pose from the control target that determines the grasping forces. Our method jointly optimizes hand geometry and control by minimizing the kinetic energy of the grasp in a differentiable simulator, while regularizing the hand geometry to preserve visual consistency and geometric plausibility. The reconstructed grasps are substantially more stable under rigorous physical simulation, without sacrificing the visual and geometric quality of the reconstruction. When projected back onto the input image, the grasp appears visually accurate. Experiments show that our approach produces far more stable grasps than alternative hand-control strategies and can also benefit visual-only grasp reconstruction pipelines by turning their outputs into physically stable grasps.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
hand-object interaction, grasp reconstruction, monocular 3D reconstruction, physical stability, simulation-based optimization, robotics, computer vision, computer graphics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_AGEaOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes StableGrasp, an optimization pipeline for reconstructing physically stable 3D human hand grasps from a single RGB image. The first stage uses an off-the-shelf hand pose estimator and an object point cloud. The results are optimized using geometric regularizers. In the second stage, a physics-based kinetic energy loss is used to jointly optimize the hand-object interaction in simulation. The method is evaluated on 362 static frames from DexYCB, HO3D, and CORe50 against a heuristic baseline and a baseline PPO policy, without any comparison to other baselines.

Critical strengths
  • The paper presents a clear formulation of static PD target optimization, with clear implementation details.
Critical weaknesses
  • There is no comparison to any related work, and the paper is overall incremental, with no clear motivation or distinguishable contribution compared to existing works.
  • Constructing physically possible grasps has been studied extensively, for example, in D-Grasp (Christen et al., 2022), Stability-Driven Contact Reconstruction (Zhao et al., 2022), Physical Interaction (Hu et al., 2022), Grasp'D / Fast-Grasp'D (Turpin et al., 2022; 2023), DeepSimHO (Wang et al., 2023), SpringGrasp (Chen et al., 2024), HOIC (Hu et al., 2024), Physics-Aware HOI Denoising (Luo et al., 2024), HandyPriors (Zhang et al., 2024b), and Grasp-and-Lift (Choi et al., 2026)
  • Recovering physically plausible dexterous grasps with PD targets from videos has also been studied in Do as I Do (Paliwal et al., 2026), CHORD (Zhu et al., 2026), ManipTrans (Liu et al., 2025), and DexTrack (Han et al., 2025)
  • The computational cost of optimization is also very high.
Questions for authors

Please provide a comparison to any existing physics-based reconstruction work.

1 comment
AuthorsOpenReview ↗

# Response to Reviewer AGEa

We thank the reviewer for the review and for the related works.

As requested, we compared our method with two published physics-based methods in this rebuttal. We ran DeepSimHO (Wang et al., 2023) with its released code and model. The code of Zhao et al. (2022) is not released, so we compare with a Zhao-style variant that keeps the defining choice of their physics stage. Details are in our response to Reviewer fZuj (Q6, part 2), and the results are summarized below. In addition, the two baselines in the paper follow prior work and were not invented by us. The PPO baseline is adapted from D-Grasp (Christen et al., 2022), the first work in the reviewer's list. The Pinching baseline follows a control rule that is widely used to execute a static grasp in physics simulation: the PD target is the grasp pose with the fingers moved further into the object, so that the hand squeezes it. DexGraspNet (Wang et al., 2023), GenDexGrasp (Li et al., 2023), BODex (Chen et al., 2025), Get a Grip (Lum et al., 2024), D(R,O) Grasp (Wei et al., 2025) and DexGraspAnything (Zhong et al., 2025) all test their grasps in simulation with this rule (details in our response to Reviewer fZuj, Q5).

To the best of our knowledge, our method is the first to reconstruct one exact hand control from a single image, so that the hand holds the object with its own time-invariant PD target, under full two-way physical dynamics, without moving from the grasp seen in the image or adapting its control to hold the object. We are also the first to significantly improve how stable a grasp is (which is crucial for visual reconstruction), not just the success rate of grasps not dropping the object.

We thank the reviewer for pointing out the important list of related works. In this list, 2 concurrent and 2 prior works, Do as I Do (Paliwal et al., 2026), CHORD (Zhu et al., 2026), ManipTrans (Li et al., 2025) and DexTrack (Liu et al., 2025), are missed by our paper. We agree that they are highly related to our work and we should discuss them in our paper.

This list of related works differ from our task in the following three ways, and we compare with a representative for each way.

  • 1. A fixed or simplified hand. In their models, the hand does not have to produce the forces on the object with its own joint torques: the hand is fixed in place (Grasp'D, Fast-Grasp'D, DeepSimHO); the contact forces are free variables (Hu et al., Luo et al.); the hand only has to avoid sliding and penetration at the contacts, and no forces are modeled (HandyPriors); or each fingertip is an ideal spring (SpringGrasp). As discussed in Section 2 of our paper (paragraph "Physics-aware Grasping Assuming Static Hands"), with a static hand, a stable object does not mean that an actuated hand can produce the required forces. DeepSimHO is a close published system with released code: when its output is tested in our two-way physics evaluation, where the hand must hold the object with its own actual joint torques, it succeeds in 22.0% of the 50 DexYCB images of our subset, against 96.0% for ours.
  • 2. A control target equal to the pose. As noted in the last paragraph of Section 2, Zhao et al. (2022) hold the hand at the reconstructed pose: the PD target always equals the visual pose. A finger then pushes on the object only if its visible pose goes into the object, so a firmer grip needs a pose that overlaps the object more, which is no longer a valid contact. As a result, even when the pose is optimized with our simulation settings and objective, the Zhao-style variant succeeds in 48.2% of the full test set, against 94.4% for ours. Our approach is better because a separate target lets the pose stay on the surface while the fingers squeeze.
  • 3. Control that changes over time. D-Grasp, HOIC, Grasp-and-Lift, Do as I Do, CHORD, ManipTrans and DexTrack learn or optimize control that changes at every timestep, so that the hand can produce or follow a given kinematic motion. As stated in the last paragraph of Section 2, our task is image reconstruction, not video, so we require our PD target to be static: it must keep the grasp stable without changing the visible grasp over time. Although the PPO baseline, adapted from D-Grasp, is allowed to change its PD targets at every step, it succeeds in 37.8% of the test examples, against 94.4% for ours. On the 128 examples where Ours, Pinching and PPO all succeed, its hand moves by 4.5 mm (MPJPE), against 0.1 mm for ours (see our response to Reviewer fZuj, Q4).

We agree that the optimization in the paper is slow. Its cost comes mainly from the differentiable MJX simulation, not from our formulation (see Appendix C.5). With a faster implementation of the same simulation (see Appendix C.6), the same method optimizes one example in about 2.5 min instead of about 22 min, on the same GPUs.

We thank the reviewer again for these references, and will add the discussion and comparisons above to the revised paper.

Reviewer_6oLGOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This article studies the reconstruction of physically stable hand grasping from a single RGB image. The core idea is that the visual hand pose determines the contact geometry, but cannot independently determine the force required to maintain grasping, so it is necessary to estimate the control target separately. The author evaluated a total of 362 images from DexYCB, HO3D, and CORe50, and compared each method on 357 common samples after excluding five failed PPO training cases. The stable success rates of StableGrasp on three datasets were 87.45%, 88.89%, and 78.95%, respectively. The complete process takes an average of 26.8 minutes per case, which belongs to the offline reconstruction method.

Critical strengths

1. The paper represents control through fixed PD objectives and jointly optimizes pose and control, so that the reconstructed results can be tested in bidirectional coupled dynamics. This modeling has a clear physical motivation and provides available post-processing directions for existing visual reconstruction processes.

2. Stability benefits are clear and include meaningful initialization ablation. These results support the effectiveness of the entire process, not just the improvement of a few qualitative cases.

Critical weaknesses

W1. The attribution of core earnings still lacks sufficient direct controlled controls. The current quantitative ablation mainly verifies pre optimization, without directly comparing "fixed visual pose, only optimizing control objectives" with jointly optimizing, and it is unclear how much contribution pose refinement and control optimization make respectively.

W2. In Table 3, MPJPE-3D increases from 7.7 to 11.3 mm on DexYCB and from 5.6 to 11.0 mm on HO3D relative to HORT. Corresponding [email protected] values decrease from 0.68 to 0.60 and from 0.72 to 0.59, respectively. The authors reasonably attribute these changes to discrepancies among the rigid hand proxy, reconstructed object geometry, and reference annotations. However, this explanation does not establish that the accuracy degradation primarily reflects necessary physical corrections. A clearer characterization of this trade-off is needed to substantiate the claim of preserving visual consistency.

Questions for authors

Q1. Could the authors compare the fixed pre optimized visual pose with only optimized PD objective and the complete joint optimization on the existing MJX settings and a representative subset?

Q2. Could the authors jointly report grasp stability and reconstruction error on the same examples? On a small subset, could varying the regularization weights characterize the trade-off between stability and reconstruction accuracy?

Q3. What evidence can support that the decrease in accuracy in Table 3 is mainly due to necessary contact correction?

3 comments
AuthorsOpenReview ↗

Response # 1 (W1, Q1): fixed visual pose, only optimizing control objectives

We thank the reviewer for the comments. As requested, we froze the visual pose at the warm-up result and optimized only the PD target, on the full test set of the paper:

Visual posePD targetSuccess rate
warm-up, frozenequal to the warm-up visual pose (no optimization)52.9%
warm-up, frozenoptimized (PD target only)87.1%
jointly optimizedjointly optimized (Ours)94.4%

Success rates are shown above. On examples where optimizing only the PD target fails, Ours succeeds on 27 examples. The reverse happens on 1.

How stable the grasp is: On the 310 examples where both succeed, the hand and the object move over 5 s by:

Object translationObject rotationHand motion (MPJPE)
PD target only1.26 mm1.35°0.77 mm
Ours0.32 mm0.34°0.14 mm

While control optimization does contribute a lot, joint optimization further improves the results consistently, both in success rates and in how stable a successful grasp is.

AuthorsOpenReview ↗

Response # 2 (Q2): stability and reconstruction error on the same examples & the trade-off with regularization weight ablations

Stability and reconstruction error on the same examples. The table below covers the 338 test examples with hand annotations (239 DexYCB and 99 HO3D; CORe50 has no hand annotation).

Success rate3D joint error (mm)[email protected][email protected]2D joint error (pixels)
HORT pose—7.090.6870.9295.05
Warm-up pose—10.880.6150.9005.74
Pinching79.9%10.880.6150.9005.74
Ours94.4%11.150.6050.8915.89

Pinching keeps the warm-up pose and changes only the PD target, so its errors equal those of the warm-up pose. The 2D joint error is the average L2 distance between the projected joints and the annotated 2D joints, on the 224 × 224 pixel image crop defined by HORT. HORT cuts a square box around the hand out of the input image and resizes it to 224 × 224 pixels, so a distance of 1 pixel in the crop equals, on average, 1.26 pixels in the input image (1.12 on DexYCB, 1.67 on HO3D and 1.03 on CORe50).

The trade-off explained by ablation study on λobs\lambda_{\mathrm{obs}}. The weight λobs\lambda_{\mathrm{obs}} scales the image term LobsL_{\mathrm{obs}}, which keeps the projected hand joints close to the 2D joint locations that HORT estimates from the image. On our chosen subset (as explained in the general comments), we multiplied the default λobs\lambda_{\mathrm{obs}} value used in our paper by 0, 0.3, 3 and 10, in both the warm-up and the full optimization, and report the results below. Following the same rule as above, we exclude 10 CORe50 examples for which hand annotations are not available, so the results are on 70 examples.

λobs\lambda_{\mathrm{obs}} (× our default)Success rate3D joint error (mm)[email protected][email protected]2D joint error (pixels)Ours (×1) better / worse
×091.4% (64/70)21.490.1020.34617.405 / 2
×0.394.3% (66/70)14.600.3900.7338.723 / 2
×1 (Ours)95.7% (67/70)11.620.6060.8925.90—
×385.7% (60/70)9.820.6560.9135.508 / 1
×1084.3% (59/70)9.090.6730.9225.339 / 1
HORT pose (no optimization)—7.550.6840.9245.17—

The table shows a trade-off between stability and closeness to the image. With a larger weight (×3 and ×10), the hand stays closer to the image and to the annotation, but fewer grasps are stable (85.7% and 84.3%). With a smaller weight (×0.3 and ×0), the hand moves further from the image, and the success rate does not become higher (94.3% and 91.4%). The weight therefore leaves room to choose. We chose ×1, which we consider a good balance: it has the highest success rate in the table (95.7%), and its 2D joint error (5.90 pixels) is close to that of HORT's pose (5.17 pixels).

AuthorsOpenReview ↗

Response # 3 (Q3): evidence that the decrease in accuracy is mainly due to necessary contact correction

Almost all of the increase in 3D joint error comes from correcting the contacts between the hand and the object, not from the physics-based optimization. This correction is necessary, because the initial (HORT) hand is not physically valid. The correction also barely changes the hand in the input image. The annotated hand is itself not physically valid either, and it must also move by a similar amount for it to physically hold the object. The increase in error against the annotation therefore mainly reflects a necessary correction. We explain these reasons below in detail.

1. Most of the error increase happens in the warm-up, which are meant to fix broken hand-object contact geometry. On the 338 annotated examples, the 3D joint error is 7.09 mm for HORT's pose, 10.88 mm after the warm-up and 11.15 mm for our final hand. The warm-up therefore accounts for 93% of the increase, and the physics-based joint optimization adds only 0.27 mm after it. The warm-up is designed to reduced unrealistic interpenetration between the hand and the object and closes the gaps between the fingers and the object surface, while keeping the hand's projection in the image. So most of the decrease in accuracy comes from this geometric contact correction, not from the physics-based optimization.

2. HORT's output hand penetrates deep inside the object, and any physically valid grasp must fix this. We measured how deep each hand penetrates inside the object, over the 180 distinct grasps among the 338 annotated examples (some examples are different views of the same grasp, hence the 180/338 here). For HORT's hand and object, the median penetration depth is 10.3 mm; 89% of the hands penetrates more than 5 mm inside. A real hand cannot penetrate deep an object, even accounting for the fingertip skin deformation. So a physically valid grasp has to fix these contact geometry errors. After the warmup, the median penetration depth is 3.5 mm; only 8% of the hands penetrates more than 5 mm inside.

3. The increase in 3D error is almost invisible in the input image, because it is mainly along image depth. For each joint, we split its error against the annotation into two parts, by projecting it onto the camera's line of sight through the joint: the part along the line of sight, which changes only the depth, and the part across it, which changes where the joint appears in the image. From HORT's hand to our final hand, the part along the line of sight grows 3.76 mm on average, while the part across it grows only 1.38 mm. As a result, the 2D joint error against the annotation increases by only 0.84 pixels (from 5.05 to 5.89 pixels on the 224 × 224 pixel image crop, as in Table 1 in comment #2). A single image says little about depth, so the 3D change lies mostly in the direction that one single view cannot determine.

4a. The annotated hands are not physically valid either, so being closer to them does not mean being more physically correct. The dataset annotations are made by fitting a hand model to the images and depth maps with geometric terms only, without any consideration of physical forces, friction or gravity, so neither checks whether the hand can hold the object. An annotated hand often penetrates inside the object on some fingertips and leaves contact gaps on the others. Over the 180 distinct grasps, the annotated hand penetrates a median of 5.0 mm inside the object. On the 50 DexYCB images of our subset, the annotated thumb, which a grasp normally relies on, ends more than 3 mm away from the object in 28% of the examples. We then simulated the annotated grasps on these 50 images. They succeed in only 48.0% (24/50), against 96.0% (48/50) for Ours on the same images. Even with the hand fixed in place ("one-way coupling physics", as defined in our paper Section 2 and our general comment), they succeed in only 64.0% (32/50). With the hand fixed, in 21 of the 50 grasps the object moves upward against gravity within the first 10 ms, by more than half the distance it would fall freely in that time. This means the annotated fingers push the object away instead of supporting it. A related paper, DeepSimHO (Wang et al., 2023), had a similar observation. To build their test set, they "only select those [annotated test samples] that remain stable after simulation", which means that some annotated grasps also fail in their simulation.

4b. Even the annotated hand must move by a similar amount to support stable grasps. Starting from the annotated hand, keeping its visual pose fixed and optimizing only the PD target, gives success rate 64.0% (32/50) on the DexYCB subset. If we first apply our warm-up to the annotated hand, which moves it by 7.40 mm on average, the same PD-target-only optimization gives 86.0% (43/50). Our full pipeline started from the annotation reaches 94.0% (47/50) and ends up 7.82 mm from the annotation on average. On the same images, our pipeline started from HORT moves the hand by a similar 8.2 mm.

Reviewer_fZujOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

StableGrasp takes a single RGB image of a hand holding an object and reconstructs the object mesh, MANO hand pose, and one fixed PD control target such that the two-way-coupled hand–object system stays near its initial state under forward simulation ("declare that the grasp is stable if xt ≈ x0 for t ≤ T"). HORT provides the visual initialization; after a regularizer-only warm-up, the visual pose and PD target are jointly optimized by minimizing accumulated kinetic energy over 20 differentiable MJX steps at 2 ms, and stability is then certified by 5 s of MuJoCo simulation at 0.2 ms with impratio 2000, success meaning the object's center of mass stays within 1 cm. On 357 author-curated examples (239 DexYCB, 99 HO3D, 19 CORe50), success rates are 87.45/88.89/78.95% versus Pinching 64.02/63.64/73.68% and an instance-specific PPO baseline at 39.33/32.32/47.37% (Table 2). Hand accuracy versus dataset annotations degrades from 7.7 to 11.3 mm MPJPE on DexYCB and 5.6 to 11.0 mm on HO3D (Table 3). Per-example cost is 26.77 ± 13.20 minutes on a Quadro RTX 6000 (Table 9).

Critical strengths

A falsifiable formulation, new within the cited literature. The paper replaces proxy criteria — force closure on a static hand, which by its own argument "does not establish whether an actuated hand can generate the required contact forces" — with a checkable dynamical definition, and correctly argues open-loop torques cannot realize it because residuals integrate into drift. Jointly inferring the visual pose and a single fixed PD target through a differentiable simulator is cleanly delineated from dynamic controllers (D-Grasp, Grasp-and-Lift) and from Zhao et al. (2022)'s stability-criterion optimization. The kinetic-energy objective is a direct, coordinate-free measure of violating the stability definition rather than a hand-crafted contact surrogate.

A matched in-engine comparison plus exceptional transparency. Pinching shares the engine, numerical settings (Table 7), object, hand proxy, PD gains, and 5 s evaluation, starting from the same warm-up pose; the 23–25-point success-rate gains on DexYCB/HO3D are the paper's solid core evidence, and all nine Table 2 percentages were verified to correspond to exact integer counts on the stated denominators. The paper self-reports its accuracy regression (Table 3), the cross-engine PPO caveat, the survivor-conditioned aggregation, a failure census (27 of 48 failures traced to object reconstruction), and solver settings down to tolerances. This transparency is what makes the claim boundaries locatable at all.

Critical weaknesses

The stability certificate runs in a configuration the engine's own documentation recommends against, and the training objective lives in a different contact regime, with neither gap ablated. Evaluation pairs impratio 2000 with a pyramidal friction cone (Table 7), while the MuJoCo XML reference defines impratio as the frictional-to-normal impedance ratio "for elliptic friction cones" and states "it is not recommended to use high impratio values with pyramidal cones" (verified at https://mujoco.readthedocs.io/en/stable/XMLreference.html#option-impratio). The abstract's "rigorous physical simulation" therefore rests on an unvalidated solver configuration; the Ours-vs-Pinching ordering likely survives, since the two share all settings, but the absolute success rates and the fidelity framing do not automatically. Meanwhile, the optimized quantity is 40 ms of kinetic energy at impratio 2 — a regime the paper itself says "lets the soft-constraint solver introduce non-physical slipping" — justified only by the assertion that "instability emerges within the first few timesteps," yet the paper's own failures drop at 0.28–1.27 s (Table 2). No ablation touches the horizon To, the timestep, impratio, or the optimization-to-evaluation transfer; the only mechanism ablation is warm-up (Table 4, verified budget-fair and arithmetically exact). An unexplained internal inconsistency compounds this: Table 4's drop times are 0.04–0.07 s with standard deviations as low as ±0.00, versus 0.28–1.27 s in Table 2, under a caption stating "Metrics and aggregation follow Table 2."

The comparative and visual-fidelity evidence is substantially narrower than the claims. No published system is ever run: both baselines are author-built control strategies applied to the paper's own warm-up output. Pinching adapts what the appendix describes as the normal-based grasp-closure procedure from DexGrasp Anything's evaluation code (δ = 5 mm), while the main text calls it "the approach of DexGraspAnything" — an evaluation heuristic promoted to a method-level baseline. PPO is a 100-iteration from-scratch per-instance adaptation, trained and scored in RaiSim with a mean-MANO-shape hand, from a warm-up pose that was contact-optimized for a different proxy in a different engine; the paper's claim that "we resolved key choices in favor of the baseline" is verifiable only for the choices it lists, and the PPO rows cannot characterize learned control in general. The closest single-RGB prior, Zhao et al. (2022), is discussed but never compared, and no hold-at-reconstructed-pose null baseline exists. All Table 2 columns except Success Rate are survivor-conditioned by the paper's own admission, so they average over different example sets per method; the CORe50 margin is a single grasp (15 vs 14 of 19). Finally, the abstract's "remaining visually consistent with the input images" has no number behind it: Table 2's IoU/PCK compare the simulation's initial and final states to each other, while the only input-referenced metric degrades 47–96% (Table 3), and gravity is approximated as the camera-down direction for HO3D/CORe50.

Questions for authors
  1. Re-run the 5 s evaluation with an elliptic friction cone and/or a moderate impratio, and report the optimization-regime versus evaluation-regime 2×2 stable/unstable contingency table. Do the absolute rates and the Ours-vs-Pinching gap persist? This decides whether "rigorous physical simulation" is defensible.
  2. Target-only ablation: freeze the visual pose at the warm-up result and optimize only the PD target under the same budget. Table 3 shows refinement moves the pose "marginally"; does the claimed joint optimization actually carry the success rates, or does the control target do all the work?
  3. Explain the Table 4 versus Table 2 drop-time gap (0.04–0.07 s ± 0.00–0.05 versus 0.28–1.27 s) under a caption stating identical aggregation: is the failure-onset distribution strongly bimodal, or does an undisclosed protocol difference exist?
  4. Report motion/IoU/PCK on the intersection of all methods' successes, your method's outcome on the 5 dropped PPO examples, and one input-referenced fidelity number for the final refined pose (e.g., reprojection error against the Lobs 2D targets, or silhouette IoU against the input).
  5. Add a hold-at-warm-up-pose null row (the strategy your own Related Work attributes to Zhao et al.) under your protocol, and report penetration/contact statistics of the warm-up grasp as transplanted onto the mean-shape RaiSim hand, so the PPO baseline's starting condition is auditable.
  6. Anchor against a published end-to-end system — Zhao et al. (2022) released a dataset but no code on its project page, so this may require reimplementation — report gravity-tilt sensitivity for HO3D/CORe50, and test a second visual initializer to support the abstract's plural "pipelines."
11 comments
AuthorsOpenReview ↗

Response #1 (Q1), part 1 of 3: friction cone, impratio and timestep

We thank the reviewer for the precise and constructive review.

All our evaluations run with the elliptic cone; "pyramidal" in Appendix Table 7 is a typo.

  • Optimization and evaluation differ in the following settings:
    • optimization: the pyramidal cone, impratio 2 and a 2 ms timestep;
    • evaluation: the elliptic cone, impratio 2000 and a 0.2 ms timestep.
  • We verified this in the code and by simulating stored evaluation runs again. The elliptic cone reproduces them exactly, while the pyramidal cone does not.
  • We have corrected the Appendix 7 table and will release the code.

Our evaluation settings are meant to be as physically faithful as possible, rather than being cherry-picked or in favor of any method.

  • Timestep. A smaller timestep integrates physics more accurately; a larger timestep adds unphysical numerical error. Evaluation uses a smaller timestep.
  • impratio. The warning that the reviewer quotes is about pyramidal cones. Our evaluation runs with elliptic cones, which, as the reviewer also quotes, are the cones that impratio is defined for. For elliptic cones, the same documentation entry (impratio) states that impratio is a parameter "having the general effect of reducing slow slippage without increasing the actual friction coefficient or guaranteeing exact sticking." In other words, a higher impratio does not make friction stronger than the friction coefficient allows in real physics; it only reduces slow slippage, which is artifical and unphysical, as we discuss below.
  • Slow slippage (as MuJoCo calls it), is completely unphysical.
    • In real physics, a contact does not slip at all if the friction force needed to keep it in place is smaller than the friction coefficient times the normal force. However, MuJoCo's documentation (slow slippage) states: "Even when the tangential force required for static equilibrium lies strictly inside the contact friction cone, MuJoCo's regularized soft-contact model does not guarantee an exact zero-velocity stick state", and "a persistent tangential load can produce a nonzero steady slip velocity."
    • As the first way to reduce this unphysical artifact, the documentation advises users to "Use elliptic cones and increase impratio", which is exactly what our evaluation does. Its section on solver settings (Algorithms) also states that "Elliptic cones correspond more closely to physical reality", and that "When contact slip is a problem, the best way to suppress it is to use elliptic cones, large impratio, and the Newton algorithm with very small tolerance." Our evaluation follows all three: elliptic cones, impratio 2000, and the Newton solver with a tolerance of 1e-12.
    • In addition, we built a simple experiment whose real-world outcome is known exactly, like a textbook physics exercise (see part 3). It shows that a higher impratio only brings the simulation closer to real physics, and does not introduce new unphysical behavior at all.
  • Different regimes. Whether our evaluation is valid does not depend on the settings of the optimizer.
    • The optimizer only outputs a visual pose and a PD target. No matter how they were found, they are judged by the same, most physically faithful evaluation.
    • The optimizer runs with softer, cheaper settings only to obtain gradients (see our response 3, part 2).
AuthorsOpenReview ↗

Response # 1 (Q1), part 2 of 3: the requested re-evaluation and the 2×2 table

The requested re-evaluation. Despite the above arguments, we report the requested experiments below, on the random 80-example subset described in our general comment. In every row, each method's optimized visual and target hands are the same; only the evaluation setting changes. "Better / worse" counts the examples where Ours succeeds and Pinching fails, and the reverse. The last three columns cover only the examples where both methods succeed.

Evaluation settingOursPinchingBetter / worseBoth succeedObject translation (mm), Ours / PinchingObject rotation (°), Ours / Pinching
impratio 243.8% (35/80)16.2% (13/80)24 / 2113.21 / 7.334.26 / 7.52
impratio 2086.2% (69/80)71.2% (57/80)14 / 2551.01 / 6.381.45 / 5.88
impratio 20093.8% (75/80)80.0% (64/80)13 / 2620.26 / 5.470.30 / 4.08
impratio 2000 (our evaluation)96.2% (77/80)82.5% (66/80)12 / 1650.27 / 5.200.28 / 3.87
2 ms timestep91.2% (73/80)78.8% (63/80)11 / 1620.35 / 5.200.41 / 4.06

As the table shows:

  • Ours leads Pinching under every setting.
  • Lowering impratio to 200, 20 or 2 lowers the success rates of both methods.
  • A 10× coarser timestep also loses grasps for both methods.
  • On the examples where both methods succeed, the object moves more with Pinching than with Ours under every setting.

Even under less physically faithful evaluation settings, our method stays well ahead. As argued above and in part 3, the lower absolute success rates here are completely due to unphysical, numerical artifacts, not a weakness of either method. This is exactly why we do not evaluate under such settings.

The 2×2 table. This table covers the same 80 examples. Here, "the optimization setting" means a 5 s simulation with the optimizer's pyramidal cone, impratio 2 and 2 ms timestep.

MethodStable under bothOnly under our evaluationOnly under the optimization settingNeither
Ours146303
Pinching660014
Target-only (defined in our response #2)367010
Zhao-style (defined in our response #6)733040
  • No grasp of any method is stable under the optimization setting yet unstable under our evaluation. The evaluation setting does not favor a different kind of grasp. It only reduces MuJoCo's unphysical "slow slippage" artifact.
  • Under the optimization setting, few grasps of any method last 5 s. This matches the slow slippage at impratio 2 shown in part 3.
  • Ours has the most grasps that are stable under both settings.
  • We emphasize again that the optimization setting only serves to provide gradients and does not change the physical settings of the evaluation or the validity of it. For any method, our valid evaluation does not require the optimizer itself to run with physically faithful settings.
AuthorsOpenReview ↗

Response # 1 (Q1), part 3 of 3: a simple experiment on slow slippage

This experiment is as simple as a high-school textbook physics exercise. Its outcome, if the experiment is done in real-world, is known exactly. It runs in MJX, with the contact parameters, friction cone and timestep the same as in our evaluation.

Setup.

  • A box of 4 kg, of size 0.1 × 0.1 × 0.2 m, is squeezed between two spherical fingertips of radius 2 cm. The friction coefficient is 1.
  • Each fingertip can only move horizontally, and a constant force N pushes it against the box. The fingertips start in contact with the box, and gravity pulls the box down.
  • Contact parameters: solref 0.02 1, solimp 0 0.99 0.001 0.5 2, margin 0, condim 4; elliptic cone, Newton solver, 0.2 ms timestep. Each run lasts 5 s.
  • We define the safety factor as the largest weight that friction at the two fingertips can support, 2μN, divided by the box's weight mg. At 1, friction is just enough; at 5, friction could support five times the weight.
  • We test safety factors of 0.8, 0.95, 1.2, 2 and 5, and impratio values of 1, 2, 20, 200 and 2000. We also test the optimizer's setting (pyramidal cone, impratio 2, 2 ms timestep).

Results.

  • Enough friction (safety factor 1.2 or more). A real box would not move. In the simulation, the box slides down at a constant speed. The speed is proportional to 1/impratio and does not depend on the grip force: 2.95 mm/s at impratio 20, 0.295 mm/s at 200 and 0.0295 mm/s at 2000. MuJoCo's soft-contact model predicts these speeds to 3–4 significant digits. At impratio 20, this sliding alone moves the box past our 1 cm success threshold in about 3.4 s, even at a safety factor of 5. At impratio 2000, the box moves only about 0.19-0.26 mm in 5 s.
  • Not enough friction (safety factor 0.8 or 0.95). The box falls at every impratio, including 2000. A higher impratio does not create grasps that real physics would not allow.
  • impratio 1 or 2. The box falls even at a safety factor of 5. The same happens with the optimizer's setting.

We set impratio to 2000 to reduce this unphysical "slow slippage" as much as possible, not to make our evaluation intentionally biased toward any method or unfairly easier. The code for this experiment has been added to our supplementary material.

Comments on the paper

1 comment
AuthorsOpenReview ↗

# General comment to all reviewers and the AC

We thank all reviewers and the AC for your time and comments. Information in this general comment applies to all our individual comments below.

MJX, which is based on MuJoCo, omits a velocity-dependent term in its contact computation, as its documentation states (Dual problem). We noticed that their implementation choice could lead to an unphysical simulation artifact in our original settings, which explains most failed grasp cases (both for our method and baselines) where the objects do not fall out of the hands immediately. We now use this simulator in a way that the MuJoCo documentation recommends to reduce this artifact as much as possible. Details are added in Section C.7 of the appendix.

We therefore ran all original experiments involving the simulation again with this correction, and updated the affected tables of the paper. All results in this rebuttal use this correction and are computed with a faster implementation of the same simulator (see Appendix C.6). We will release the code. All our claims and conclusions still hold.

Because of limited time, some additional experiments cover a fixed 80-example subset (50 DexYCB, 20 HO3D, 10 CORe50) of the full test set in our paper. It is drawn at random within each dataset from the examples whose objects decompose into at most 5 convex pieces. Time cost for the 5-second grasp simulations grows with the number of object convex pieces, so we use this rule only to avoid very long simulation time that could delay our comments. The rule depends only on the object's mesh and applies to all methods, so we claim that our results based on this subset is representative. We will finally report all results on the full test set.

We would also like to emphasize that Methods that evaluate grasp stability under one-way coupling grasp physics (i.e. with a "fixed" hand) cannot be faithfully compared with ours, as we have also argued in Section 2 of our paper. Although such methods are often called "physics-based", their evaluation differs in critical ways from ours and cannot be regarded as the same problem.

Many published works test grasp stability with one-way coupling: ObMan (Hasson et al., 2019), Grasping Field (Karunratanakul et al., 2020), GraspTTA (Jiang et al., 2021), CPF (Yang et al., 2021), Grasp'D (Turpin et al., 2022), DeepSimHO (Wang et al., 2023) and RegionGrasp (Wang et al., 2024) state this explicitly in their papers. GanHand (Corona et al., 2020), OakInk (Yang et al., 2022), FastGrasp (Wu et al., 2024), TOHGS (Liu et al., 2025), Force-Aware 3D Contact Modeling (Chen et al., 2026) and VolCo (Chen et al., 2026) also use one-way coupling physics for evaluation, as can be seen either in their supplemental material or their released code.

In one-way coupling, the hand is "fixed" in space. The object falls under gravity and touches the hand, but the hand never moves no matter how hard the object pushes on it. In other words, the hand can apply arbitrarily large forces without limit. A grasp therefore looks stable whenever the object is stuck against the hand, even if no real hand could press hard enough to keep it there. This is like using a stone-made hand on a statue to hold the object.

In our evaluation, the coupling is two-way. The hand is free in space and is moved by torques at the wrist and at every finger joint, each with a biomechanically realistic torque limit. The object pushes back on the hand, and a grasp is stable only if both the hand and the object is stable.

To compare the two kinds of evaluation directly, we designed an experiment on our aforementioned 80-example test set to evaluate the same grasp twice: once with the "stone" hand fixed in place by changing physical settings of the hand in the simulation, and once with our two-way physics. All other settings are identical. The results are below.

Visual Hand PoseHand fixed in placeTwo-way (ours)
HORT output71.2% (57/80)53.8% (43/80)
After warm-up88.8% (71/80)55.0% (44/80)
Zhao-style (defined in our response to Reviewer fZuj, Q6)93.8% (75/80)50.0% (40/80)
Ours92.5% (74/80)96.2% (77/80)

One-way coupling makes grasping easier, because the fixed hand can push with unlimited force. It also ignores control. A fixed-hand test cannot tell which grasps a hand with limited strength can keep. As shown in the table, success rates of different methods cannot be directly and faithfully compared, if they evaluate under different coupling physics. Note that Ours was optimized for two-way coupling only; we tested its output with the fixed hand directly, without optimizing it again.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.