acceptodds
Withdrawn from ICLR 2027

DiBiHand: Diffusion Guided Bilateral Hand Composition

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

This paper proposes DiBiHand, a novel diffusion-based framework for 3D hand mesh recovery. In contrast to conventional discriminative pipelines that directly regress hand parameters from images via multi-stage approaches such as detect, crop, flip, box, and joint analysis, DiBiHand formulates hand pose estimation as a reverse diffusion process, progressively denoising random samples drawn from a Gaussian prior to produce realistic hand meshes. The method employs the MANO (hand Model with Articulated and Non-rigid defOrmations) parametric model to represent hand poses, supporting our proposed adaptive axis-angle (A-axis) and continuous 6D rotation representations (Rot-6D) for stable and efficient rotation learning. For bilateral hand pose estimation, the model handles both left and right hands simultaneously through a unified pose representation that concatenates rotations for all joints. Training with random masking simulates partial observations and occlusions, enhancing the robustness of the model under real-world conditions. The diffusion-based formulation inherently captures the complex, multi-modal nature of hand articulations and provides a strong generative prior that can serve as a regularizer for conditional refinement tasks. Meanwhile, DiBiHand achieves state-of-the-art performance on single and bi-hand benchmarks, e.g., 4.6 JPE on FreiHand, 4.5 JPE on DexYCB, and an average 64% [email protected] on HInt. The code will be available online.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Diffusion prior, Bi hand 3D mesh recovery, adaptive axis-angle, 6D rotation representations

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_BNU2OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DiBiHand, a diffusion-based framework for 3D hand mesh reconstruction. During the training stage, DiBiHand trains a diffusion model with multiple hand datasets to learn hand prior information. During the testing stage, DiBiHand leverages this diffusion prior as a regularizer and treats hand mesh reconstruction as a fitting process. Extensive experiments demonstrate its effectiveness over multiple hand benchmarks.

Critical strengths
  1. The proposed diffusion prior as a regularizer is effective in hand reconstruction. Particularly, I think it is useful in scenarios that occlusion exists to refine the hand poses.
  2. Extensive experiments on both multiple in-door and in the wild datasets are provided. The results are convincing.
Critical weaknesses
  1. The main proposed algorithm is similar to DPoser, with a fitting loss and a regularization loss. The only difference in algorithm seems to be switching SMLP to MANO model and using different 2D keypoint detection methods.
  2. The improvement in hand mesh recovery is not significant compared to other methods in Fig.5. Limited qualitative comparisons are provided with hand reconstruction methods. Fig. 18 presents further results with whole body reconstruction methods. While the first three cases are better, the fourth case seems worse than other methods when severe hand-object occlusion exists.
  3. Some typos affect the readability of the paper. There is no AI use statement, which is required in ICLR submission rules. “Section ??” in Appendix A. “Eq. equation” double equation representations. “..” in L270.
  4. Discussions are limited in Ablation studies. Authors present the experimental results, but further discussions regarding the results are encouraged.
Questions for authors

What are the highlights or advantages of DiBiHand compared with the previous method DPoser? Are there any specific challenges in the hand reconstruction scenario?

Reviewer_hW9JOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper present DiBiHand, which learns an unconditional diffusion prior over MANO hand articulations and use it to regularize fitting to detected 2D keypoints for hand mesh recovery. The paper explores different pose parameterizations, with masked prior training and orientation-hypothesis selection. The method is technical sound, and results are competitive across several benchmarks.

Critical strengths
  1. The paper addresses a meaningful problem in 3D hand reconstruction, particularly under occlusion and ambiguous observations.
  2. Separating pose-prior learning from observation fitting provides useful flexibility. The learned prior can support 2D keypoint guided fitting and 3D-joint denoising without task-specific retraining. The reported results are promising.
Critical weaknesses
  1. The methodological contribution over DPoser-hand is insufficiently presented. DPoser-X already includes a diffusion prior over MANO articulations and uses it to regularize 2D-keypoint-guided hand fitting. DiBiHand also follows DPoser for its optimization in Sec.3.5. However, the paper does not clearly explain what core methodological advance distinguishes DiBiHand from DPoser-hand.
  2. Although the overall framework appears technically reasonable and the reported results are promising, the proposed contribution or modifications (rotation parameterization, masking, shared bilateral fitting, orientation-hypothesis selection) appear incremental, and with limited method novelty beyond existing framework.
  3. Comparisons with feed-forward methods like HaMeR is unfair, since DiBiHand is an optimization-based method. It remains unclear whether the reported improvements primarily come from the prior itself or fitting pipeline. A more direct comparison should evaluate fitting without a pose prior, fitting with a VPoser-style hand prior, fitting with DPoser-hand, and fitting with DiBiHand, using the same detected keypoints, initialization, loss term. Currently, Tab.1 only compare these priors in unconditional pose generation, not in 3D mesh recovery. Without controlled reconstruction comparisons, the paper does not demonstrate what additional fitting errors or ambiguities DiBiHand resolves beyond existing priors, or how much of its reconstruction improvement can be attributed to the proposed prior.
  4. The paper introduces A-axis and Rot6d as separately trained alternatives, but with only one prior used each fitting. The reported downstream results favor rot6d, so when and why A-axis be preferred over rot6d? Why the authors proposed two representation? Nor are the two priors used jointly to demonstrate complementary benefits. The issue is therefore not merely the absence of a combined-prior experiment, but the lack of a clear rationale for presenting both formulations as methodological contributions. I suggest authors to clarify their intended use cases and demonstrate the distinct value of A-axis beyond serving as an alternative parameterization that performs worse in the reported downstream evaluations.
  5. Could the prior gradient conflict with the 2D reprojection term, pulling an otherwise accurate estimate away from reliable keypoints toward a plausible but incorrect pose? I think the optimization process is fragile sometimes, a prior-weight sensitivity analysis under varying observation quality, may help clarify when the prior improves recovery and when it introduces unwanted bias.
  6. The motivation is not clear. Why diffusion prior is important? The introduction would benefit from a more focused explanation of why a diffusion prior is needed during fitting. In particular, the authors should clarify whether its primary purpose is to stabilize optimization, prevent implausible poses, or improve reconstruction under incomplete observations, and organize the motivation around these specific objectives.
Questions for authors
  1. What is the core methodological advance over DPoser-hand? Please clarify which limitations of the existing hand prior are addressed by the proposed modifications.

  2. Could the authors compare fitting without a pose prior, with a VPoser-style hand prior, with DPoser-hand, and with DiBiHand under the same keypoints, initialization, and fitting settings? This would help clarify whether the improvements come from the proposed prior or the fitting pipeline.

Reviewer_Ao61OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper learns an unconditional diffusion prior over canonical MANO articulations and uses it to regularize keypoint-based single- and two-hand fitting. It introduces bounded A-axis decoding, masked noise prediction, and orientation-hypothesis selection, reporting improvements on several reconstruction benchmarks. The masking and A-axis issues affect methodological validity, and the incremental value over prior-guided fitting remains unisolated. A corrected formulation and controlled reconstruction experiments are needed.

Critical strengths
  1. A reusable articulation prior supports generation, inverse kinematics, and reconstruction.
  2. Evaluation covers several datasets and distinguishes aligned, root-relative, and inter-hand errors.
  3. The compute breakdown includes the large keypoint detector and iterative fitting.
Critical weaknesses
  1. A-axis is inconsistent with the stated training coordinates. Figure 8 trains on raw MANO axis-angle values but applies πtanh(‖ω‖/π) only during decoding, shrinking even a perfectly recovered training vector; evaluating fitting through this decoder does not remove the prior-distribution mismatch. Specify any inverse transformation and report clean round-trip error; the smooth exponential map should not be conflated with the non-uniqueness of axis-angle coordinates.

  2. The distinction from the closest prior is insufficient. The cited DPoser-X (in ICCV 2025), discussed as DPoser in the submission, already studies hand diffusion priors, hand generation/fitting, and masked training, while Rot-6D is standard. A matched-data, matched-detector, matched-fitting comparison is needed to establish the technical increment.

  3. The bilateral contribution is not isolated. Equation (11) sums single-hand priors, with cross-hand coupling primarily through shared shape/camera, while HInt PCK directly depends on ViTPose observations. Detector-only, no-prior, alternative-prior, and independent-hand fitting controls are required to attribute the gains.

Questions for authors

Does Eq. (5) match the implementation, and what exactly is predicted at masked coordinates? Are training angles inverse-transformed for A-axis; if not, how is the claimed absence of bias reconciled with nonzero clean round-trip error?

How is the scalar prior energy used for hypothesis selection computed from the stochastic gradient in Eq. (12), and made comparable across hypotheses?

What results do DPoser-X, no masking, and independent-hand fitting obtain under the identical detector and optimization pipeline?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.