- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents IMAGIN-4D, a diffusion-based framework for image-guided 4D human-object interaction (HOI) generation. To improve fine-grained interaction control, the Spatially Factorized Image Encoder (SFIE) extracts supervised role-specific tokens for human pose, object pose, contact, and spatial relationships. A reference-frame localizer and window-gated conditioning provide temporal alignment, while the Frame-Aware Image Encoder (FAIE) retrieves frame-dependent visual cues. A role-aware conditioning mechanism integrates image, text, and waypoint information through separate conditioning pathways. Experiments on FBM and BEHAVE demonstrate improvements in image adherence, motion quality, and contact accuracy over image-conditioned baselines, although waypoint-following accuracy decreases.
Critical strengths
The paper addresses a meaningful limitation of HOI generation, as text and sparse trajectories alone cannot fully specify fine-grained interaction details. Using a reference image provides an intuitive way to control these details. The proposed supervised spatial factorization and frame-aware conditioning mechanisms are well motivated and complement each other. Experiments on FBM show substantial improvements in image adherence, motion quality, and contact accuracy, while the BEHAVE results provide additional evidence under real-image conditioning. Component ablations and image-flipping experiments further support the effectiveness of the proposed approach. The cross-domain and sketch-based evaluations also demonstrate the potential flexibility of the proposed image-conditioning framework.
Critical weaknesses
The evaluation does not fully support fine-grained interaction control, as the image-adherence metric omits reference-specific object orientation and hand-object contact details, which are not adequately captured by the separate contact metrics. IMAGIN-4D also shows higher waypoint errors, but the trade-off between image adherence and trajectory accuracy lacks systematic analysis. Although temporal comparisons and frame-offset experiments are provided, reference-frame localization accuracy remains unreported, and the contribution of frame-index conditioning in FAIE is not independently verified. Finally, while the component ablations use consistent settings, these differ from the final model, which additionally employs logit-normal diffusion-step sampling and hand-object coupling. Without isolating these training strategies, the contributions of the proposed architectural components to the final performance remain unclear.
Questions for authors
1.The image-adherence metric does not directly evaluate object orientation or hand-object contact details. Could the authors provide additional evaluations of these attributes and report image-adherence results for the image-free CHOIS baseline? 2.How accurately does the reference-frame localizer predict the target frame? Could the authors compare results using predicted and ground-truth reference frames and evaluate FAIE without frame-index conditioning to clarify its temporal contribution? 3.The ablation settings in Table 3 differ from those of the final model. Could the authors isolate the effects of the additional training strategies under consistent settings? A parameter-matched multi-token baseline without role-specific supervision would also help clarify the contribution of spatial factorization. 4.How does IMAGIN-4D compare with a simpler approach that predicts a 3D interaction state from the reference image and uses it for keyframe control? Could the authors also examine the trade-off between image adherence and waypoint accuracy under different guidance strengths?