- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes PHE-DiT, a conditional flow matching framework for flexible-length dance-to-music generation. The method first extracts rhythmic phase from motion energy using the Hilbert transform, and then decomposes motion energy into slow and fast components for hierarchical conditioning across different transformer layers. Variable-length sequence processing is further introduced to support flexible-duration music generation within a single model. Experiments on AIST++ and TikTok demonstrate strong rhythmic alignment and music-quality results.
Critical strengths
-
The paper is well motivated by the multi-scale temporal correspondence between dance motion and music. The decomposition of motion information into phase, slow energy, and fast energy provides an intuitive conditioning structure.
-
PHE-DiT achieves strong rhythmic alignment and music-quality results on most metrics across both AIST++ and TikTok benchmarks.
-
The ablation study covers the major architectural choices, including phase conditioning, hierarchical splitting, split order, single-frequency conditioning, self-attention, and the layer split ratio.
Critical weaknesses
Weaknesses
-
The paper lacks an ablation study on music quality. Table 3 only evaluates rhythmic alignment metrics, while the main paper also claims improvements in audio quality based on FAD. It is therefore unclear how each proposed component affects music quality rather than only beat/onset alignment.
-
The qualitative comparisons do not consistently control generation length across methods. In Fig. A1, the baselines operate on fixed-length 5-second segments, while PHE-DiT generates the full sequence. The supplementary demo also presents outputs of different durations. Although these comparisons illustrate flexible-length generation, they make direct comparison of rhythmic alignment less controlled. It is also unclear whether the quantitative results in Tables 1 and 2 are evaluated over matched temporal ranges and segmentation protocols.
Minor Weakness
-
The paper overstates the fixed-length limitation of prior D2M methods. While several methods adopt fixed-length training or inference protocols, this limitation is not universal. For example, CMT uses video-duration-dependent beat-timing encoding to guide music generation and EOS prediction, while LORIS operates on predefined clip lengths. The authors should clarify whether the main contribution lies in unified variable-length training, full-sequence generation without chunking, or improved robustness across different durations.
-
The demo does not cover the ablation variants. Since phase-aware fusion and hierarchical slow/fast conditioning are the main designs of the paper, qualitative comparisons among these variants would help illustrate their perceptual effects.
-
There appears to be an incorrect bold marking in Table 3. For onset-level HSD, “w/o Self-attn + Phase” achieves 10.32, which is lower than the full model’s 14.47 and should therefore be the best value if lower is better.
Questions for authors
-
Why are the Hilbert transform and Butterworth filtering specifically chosen for phase extraction and energy decomposition? Were they selected through empirical comparison or mainly by design intuition? A lightweight comparison with functionally similar alternatives, such as peak/cycle-based phase estimation and Gaussian smoothing, moving-average filtering, or wavelet decomposition, would help clarify whether the gains come from the proposed conditioning design rather than these particular signal-processing operators.
-
What is the main source of improvement over prior dance-to-music baselines? Is the gain primarily due to decomposing motion into phase/slow/fast components, the hierarchical injection of these signals into different DiT layers, or the underlying flow-matching backbone?
-
Are all methods in Tables 1 and 2 evaluated over matched temporal ranges and segmentation protocols? Fig. A1 and the supplementary demo contain outputs of different durations across methods. Please clarify how generation length is controlled in quantitative evaluation, and whether length-matched qualitative comparisons can be provided.
-
How do the individual components affect perceptual and audio quality? Could the authors report FAD or subjective-quality results for the main ablation variants in Table 3?
-
Since slow and fast motion energy are explicitly modeled as separate conditions, can they be independently recombined at inference time? For example, what happens when the slow-energy component from one dance is combined with the fast-energy component from another? Such a compositional test could provide direct evidence that the two branches capture the claimed global and local rhythmic factors.
-
The hierarchical variants show relatively modest changes in mean BCS/BHS/F1 in some cases but much larger changes in CSD/HSD. Could the authors provide per-sequence distributions or further analysis explaining where this variance reduction comes from?