acceptodds
Under review as a conference paper at ICLR 2027

DisContact: Generating Hand Motion and Discrete Contact Maps with Geometry-Coupled Hybrid Diffusion

24 views

Abstract

Generating hand-object interactions requires modeling two tightly coupled but structurally different quantities: continuous articulated hand motion and discrete contact over the hand surface. Existing approaches place both in a shared continuous or latent space, which ignores this difference. We introduce DisContact, a hybrid diffusion framework that generates each modality in its native state space and couples them through the geometry they share. Hand motion follows a Gaussian DDPM, while per-vertex binary contact follows a Bernoulli D3PM whose stationary distribution is the empirical contact frequency of each vertex. At every denoising step, the current motion estimate is decoded through differentiable MANO kinematics into vertex-aligned geometry that conditions contact denoising, and pooled contact features refine the motion in return. A structured denoiser further exploits mesh connectivity and anatomical hand regions. On HoloAssist and ARCTIC, DisContact improves contact F1 and motion accuracy over prior work in nearly all settings, raising contact F1 on ARCTIC from 0.41 to 0.59. Controlled comparisons show that most of the benefit of coupling comes from explicit geometry rather than generic feature exchange, and new object-free diagnostics show that the generated motion and contact are more consistent with each other. These results indicate that keeping each modality in its own generative process, while linking them through the geometry they describe, is an effective way to generate structured hand–object interactions.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Hand-Object Interaction, Motion Generation, Contact Modeling, Diffusion Models, Discrete Diffusion, Multimodal Generative Modeling

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_QtKcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose DisContact. DisContact jointly generates hand motion and per vertex contact using Gaussian and discrete diffusion, respectively. This formulation lets the model retain it's original representation in each diffusion, while allow each diffusion stream to improve each other. At each denoising step, predicted motion is decoded through MANO to condition contact prediction, and pooled contact features then refine the motion estimate.

Critical strengths
  • Reasonable formulation. Modeling continuous motion and binary contact in their respective state spaces is well motivated and reasonable.

  • Meaningful geometric connection. Vertex aligned MANO geometry provides a clear correspondence between articulated motion and contact variables.

  • Promising experimental results. Improvements on two datasets and several controlled ablations support the overall framework, particularly its contact prediction.

Critical weaknesses
  • Lack video results. A paper on 4D, especially for generation papers, is expected to have extensive qualitative results, especially videos. Reviewers cannot assess a 4D generation paper without seeing the videos.

  • The contribution of contact to motion improvement is not sufficiently isolated. Motion refinement uses hidden contact branch features, which also contain motion geometry and image/text conditioning. Consequently, improved motion accuracy could arise from additional processing capacity or auxiliary supervision, rather than contact specific information. The coupling ablations do not separately isolate the contact motion pathway. A directional ablation and a capacity matched refinement baseline would make this contribution much clearer.

  • Motion and contact consistency does not establish valid hand object interaction. The object free diagnostics assess properties of the generated hand and contact labels, but cannot determine whether contacting vertices touch the actual object or whether penetration occurs. This limitation is acknowledged, but it restricts what the consistency results demonstrate. Object aware evaluation would strengthen the practical significance.

  • The evaluation provides limited evidence about generative quality and robustness. Results emphasize motion error and contact F1, with little analysis of sample diversity or uncertainty. Controlled ablations use a single sampling run, making smaller improvements difficult to assess reliably.

  • Lack of discussion with methods that use continuous contact maps (minor) It would be great to see discussions related to papers using continuous contact maps.

Questions for authors
  • How does motion accuracy change when only contact-to-motion refinement is removed, while motion to contact geometry conditioning is retained?

  • Does refinement still help when the contact branch retains its architecture but receives no contact-state input or contact supervision?

  • Can the generated contact be evaluated against actual object surfaces where geometry is available?

  • If contact is binary, it cannot model the distance between the hand and the object which also could be a very informing signal. Could using continuous, distance based contact maps potentially improve model performance? There is already a line of work that uses distance based continuous contact maps to enhance performance (JointHOI: Jointly Generating Contact Maps Enhances Hand Object Interaction Generation ECCV26, Bimart: A unified approach for the synthesis of 3d bimanual interaction with articulated objects CVPR25). It would be great if there is a discussion on the use of binary contact maps over continuous distance based contact maps, and also comparison to these work.

Reviewer_aThYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a new method to generate hand object contact simultaneously. The method, named DisContact, uses a hybrid diffusion framework to generate both the hand motion and binary contact on hand vertices. To make it work, the authors proposes several techniques such as using vertex-aligned geometry to condition contact generation, and let the denoiser to exploit mesh connectivity and anatomical hand regions. On HoloAssist and ARCTIC datasets, the method performs well.

Critical strengths
  1. The experimental results on HoloAssist and ARCTIC datasets outperform existing methods by a large margin.

  2. THe proposed method is well motivated. The method explicitly addresses both the different state spaces of motion/contact and their different indexing structures.

  3. The joint-consistency analysis seems useful. Evaluating whether motion and contact agree is more informative than reporting their individual errors alone.

Critical weaknesses
  1. As hand-object interaction method, the physical plausibility remains unverified. Object-free consistency measures can reward coherent hand/contact patterns without establishing compatibility with the actual object, including penetration or contact placement.

  2. The authors only present qualitative results on HoloAssist dataset. More qualitative results on ARCTIC dataset, and real-world images should be presented to help the readers fully understand the quality.

Questions for authors
  1. For experiments on HoloAssist and ARCTIC, was the model trained on each dataset seperately for each experiment? I want to make sure that the comparsion is fair wrt previous papers. This may be stated more clearly.
Reviewer_v9T5OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper jointly generates continuous hand motion with a Gaussian DDPM and binary vertex contacts with a marginal-preserving Bernoulli D3PM. MANO-derived geometry and contact-to-motion refinement couple the branches under the image, text, and sparse-contact-point setting of LatentAct. Geometry-mediated coupling is well motivated and supported by cross-dataset improvements and useful controls. This assessment is limited to the supplied main paper; failure to reproduce the coupling gain or resolve implementation details would lower it.

Critical strengths
  1. The representation respects the different structures of articulated motion and surface contact, using MANO for explicit cross-modal correspondence.

  2. The factorized Gaussian versus Bernoulli controls isolate contact state space more cleanly than the shared-denoiser comparison alone.

  3. Gains extend across two datasets, including ARCTIC contact F1 of 0.59 versus 0.41, with a substantial geometry-token ablation effect.

Critical weaknesses
  1. Joint geometric validity lacks object-aware verification. Enclosure, persistence, and rigidity can improve for a stable contact patch at the wrong object location, while ARCTIC MPJPE remains 11.24 cm (Tables 2 and 4). Use available ARCTIC object geometry for evaluation-only contact distance, penetration, and relative-slip diagnostics; this does not require changing the object-free input setting.
  2. Training stability is unmeasured. Each model is trained once, Tables 1 and 3 additionally use one sampling run, and twenty resamplings elsewhere do not quantify training variance (§4.1). Repeat the key state-space and geometry controls and verify protocol compatibility with imported baseline results.
  3. The benefit of stochastic generation relative to cost is unclear. Single-reference errors and contact F1 do not establish the value of 1,000-step sampling over deterministic prediction. Include conditional diversity/coverage, measured quality–latency trade-offs, and a matched motion backbone with a direct contact head.
Questions for authors
  1. Do object-relative contact distance, penetration, and slip improve alongside F1 on ARCTIC?
  2. Are the +0.02 F1 from vertex-specific marginals and +0.08 from geometry tokens stable across training seeds?
  3. How do a direct contact head and fewer-step sampling compare in quality, conditional diversity, and latency?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.