- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper proposes a 3D-aware video diffusion model for human motion-conditioned video generation, using 3D human motion as a condition without relying on rendered 2D motion guidance. The authors tokenize 3D human pose with a VQ-VAE, separately encode the global trajectory, and combine them into motion tokens. These motion tokens are then injected into a pre-trained DiT-based video diffusion model through frame-wise cross-attention. The authors evaluate the proposed method on human motion control benchmarks, showing comparable or improved video quality compared to existing rendering-conditioned motion control video generation methods.
Critical strengths
-
The direct use of tokenized 3D human motion conditioning for video motion control is well-motivated. The proposed render-free 3D motion representation provides 3D motion information to the video generation model and avoids the view-dependent bias of rendering-based approaches.
-
The proposed method shows competitive video quality across the evaluated benchmarks. The authors provide useful ablations on input trajectory, body pose conditioning, and motion representation, which help justify their proposed design choices.
Critical weaknesses
1. Limited comparison with closely related prior work
The proposed render-free 3D motion tokenization is not sufficiently distinguished from closely related prior work, particularly MTVCrafter (ICLR 2026). It seems that MTVCrafter similarly tokenizes 4D human motion and uses the motion tokens to condition a video DiT for controllable video generation. Since direct 3D motion token conditioning is presented as a core contribution, a clear discussion of technical differences from MTVCrafter, with an empirical comparison, would be important for assessing the novelty of the proposed method.
[1] Ding et al., "MTVCrafter: 4D Motion Tokenization for Open-World Human Image Animation," ICLR 2026.
2. Insufficient evidence for 3D awareness
The core motivation of this paper is that render-free, direct 3D motion conditioning enables the video diffusion model to jointly reason about 3D motion and camera viewpoint, yielding more robust generation under viewpoint changes. However, the reported metrics primarily evaluate video quality rather than cross-view or 3D consistency. An experiment that generates the same underlying 3D motion under different, preferably unseen, camera viewpoints and measures cross-view consistency would provide more direct evidence for the claimed 3D awareness and robustness to viewpoint changes.
3. Limited analysis of 3D motion estimator dependency The render-free input formulation removes the need to rasterize 3D motion into 2D guidance, but it instead relies on 3D human motion estimator and human motion VQ-VAE to obtain the conditioning signal. This introduces an additional source of error that is not sufficiently analyzed. The authors argue that the proposed method is robust to imperfect combinations of trajectory and body pose, but it remains unclear how much error in the estimated 3D motion the model can tolerate. A quantitative analysis under different levels of motion estimation error would help clarify the robustness of the proposed conditioning pipeline.
Questions for authors
- Does the decomposition of global trajectory and body pose support independent control? For example, can the same body pose sequence be combined with different global trajectories, or vice versa, while maintaining plausible generation quality?
- The paper uses a pre-trained VQ-VAE mesh tokenization. Could the authors provide an ablation on the codebook size to clarify whether the performance is sensitive to the codebook capacity?
- The figure ordering should be consistent. Figure 4 is currently presented before Figures 2 and 3.