- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents RAMP, a dance-to-music generation framework that adapts a pretrained music model using dance, text, and lyric conditions. Its Style-Adaptive Rhythm Tokenizer (SART) combines style-conditioned expert routing with frame-level motion adaptation to represent dance rhythm. A two-stage training strategy first performs text/lyric adaptation and then introduces motion conditions, with a frozen Stage-1 teacher regularizing the student’s text-conditioned predictions. The paper also introduces Dance100K, a large-scale dataset of paired dance–music clips. Quantitative comparisons, ablations, prompt interventions, and a user study report improvements in rhythmic alignment, musical quality, and semantic consistency on five-second samples.
Critical strengths
-
Well-motivated hierarchical rhythm modeling. SART combines global style preferences with frame-level motion adaptation, providing a coherent way to capture both style-dependent rhythmic patterns and local movement variations.
-
A practically valuable adaptation framework. The method jointly considers dance–music alignment, musical quality, and semantic control. Staged adaptation and teacher regularization provide a sensible approach to incorporating motion conditions into a pretrained music model.
-
Substantial data and multifaceted evaluation. Dance100K provides a sizable real-world resource, while controlled ablations, prompt interventions, and a user study offer complementary evidence for the proposed design.
Critical weaknesses
-
Limited evaluation beyond short clips. Experiments focus on five-second segments, which primarily demonstrate local rhythmic alignment and audio quality. Although acknowledged, this limitation leaves musical development, phrase transitions, and sustained correspondence with longer choreography unverified.
-
Unclear song-level independence of the data split. Source-video disjointness prevents overlapping segments from the same video, but does not exclude different videos containing the same song or recording. Could the authors report an audio-overlap audit and, ideally, results under a song-disjoint split?
-
Limited methodological novelty. The combination of style-conditioned expert routing, staged adaptation, and teacher regularization appears incremental. The paper should more clearly distinguish SART from existing genre-adaptive rhythm modeling, such as GACA-DiT, and explain the conceptual advance beyond combining these components.
-
Incomplete positioning within music–motion research. Relevant context includes cross-modal representation and alignment methods such as MoMu-Diffusion, BeatDance, TokenDance, and ChoreoMaster, alongside controllable or flow-based dance generation methods such as EDGE, CustomDance, and FlowerDance. Discussing transferable mechanisms while distinguishing their task settings would better clarify the contribution of SART.
Questions for authors
See weakness. I will raise my score if my concerns are adequately addressed.
4 comments
Response to Reviewer s5yG — W2: Song-Level Independence and Audio-Overlap Audit
We thank the reviewer for identifying that source-video disjointness does not guarantee song-level independence.
1. Audio-overlap audit. Accordingly, we have added a cross-split audit using exact AIST music IDs where available and CLAP nearest-neighbor retrieval for sources without song IDs. We removed samples with a matched AIST music ID or cosine similarity . This conservative union removed 1,726/3,000 clips and retained 1,274. Because some sources lack canonical song IDs, we refer to this set precisely as a conservatively overlap-filtered subset, rather than claiming a strictly song-disjoint split.
2. Overlap-filtered evaluation. For the retained 1,274 IDs, we re-aggregate MDAphys, Beat F1, AudioBox PQ, and musicality from the corresponding Full-3K per-sample evaluation outputs, without retraining or regenerating audio. Beat MAE was not reported in the paper's main table; both Beat MAE values are computed under the same matched overlap-audit protocol.
| Evaluation set | MDAphys ↑ | Beat F1 ↑ | Beat MAE (ms) ↓ | AudioBox PQ ↑ | Musicality ↑ |
|---|---|---|---|---|---|
| Full-3K | 0.2978 | 0.6224 | 36.79 | 7.6686 | 2.9804 |
| Overlap-filtered (1,274 clips) | 0.2937 | 0.5980 | 36.98 | 7.6537 | 2.9471 |
The model retains comparable absolute performance after overlap filtering, with modest decreases in rhythm and audio-quality scores and nearly unchanged Beat MAE. The protocol and results are included in the revised manuscript.
Response to Reviewer s5yG — W3: Methodological Novelty beyond Component Combination
We thank the reviewer for raising this concern. RAMP retains the pretrained music-generation backbone by design; its methodological contribution addresses what dance-rhythm information should be represented and how it can be added while preserving the pretrained T2M behavior.
1. SART introduces global-to-local rhythm routing. GACA-DiT combines multi-scale spatiotemporal motion features, adaptive joint weighting, and query-based temporal alignment. SART instead constructs kinematically biased rhythm candidates and explicitly factorizes their routing into a clip-level style prior and a bounded frame-level motion residual. This global-to-local factorization provides persistent style preferences together with frame-level motion adaptation.
2. Teacher-constrained modality expansion preserves pretrained behavior. In Stage 2, a full-condition student learns dance--rhythm--text control, while its text/lyric-only prediction is constrained by a frozen Stage-1 teacher under matched text inputs, noisy latents, and flow timesteps. Because the teacher receives text/lyrics without motion, it provides a modality-isolated reference for the pretrained text-conditioned velocity field and is removed after training.
3. Accordingly, we have added two ablation experiments to verify these distinct roles. First, we isolate the two motion-conditioning branches:
| Conditioning | ABHS ↑ | MDAphys ↑ | Beat F1 ↑ | PQ ↑ | CLAP ↑ |
|---|---|---|---|---|---|
| General Dance only | 0.1084 | 0.2758 | 0.5893 | 7.2670 | 0.3071 |
| SART Rhythm only | 0.1103 | 0.2839 | 0.5928 | 7.2018 | 0.3006 |
| Full | 0.1117 | 0.2978 | 0.6224 | 7.6686 | 0.2923 |
SART-only improves all three rhythm metrics over General-only, and Full further improves MDAphys (0.2978), Beat F1 (0.6224), and PQ (7.6686), supporting complementary roles. Second, we have added an auxiliary text-only ground-truth FM baseline. Under the same Full-3K evaluation protocol, the paper model () also outperforms the auxiliary text-only ground-truth FM baseline (): PQ improves from 6.1928 to 7.6686, CLAP from 0.2203 to 0.2923, coherence from 2.7453 to 2.9947, and musicality from 2.6922 to 2.9804. Thus, SART supplies structured rhythm control, while teacher-constrained adaptation protects the pretrained musical and semantic behavior; the two mechanisms therefore serve complementary functions. We have made this distinction explicit in the revised method description and Figure 1.
Response to Reviewer s5yG — W4: Music–Motion Related Work
We thank the reviewer for identifying this missing context. We have added a dedicated “Music--Motion Cross-Modal Modeling” subsection covering all seven suggested studies and clarifying their relationship to RAMP:
- Cross-modal representation and alignment: MoMu-Diffusion and BeatDance contribute aligned latent spaces and beat-aware contrastive alignment, but focus on joint generation or retrieval rather than dance-conditioned waveform generation.
- Music-to-dance generation: TokenDance, ChoreoMaster, EDGE, CustomDance, and FlowerDance contribute token mapping, phrase structure, editability, and efficient diffusion/flow, but solve the reverse task and output motion rather than music.
The prior studies offer useful principles for alignment, tokenization, long-range synchronization, and controllability. RAMP addresses the complementary question of how dance kinematics can condition a pretrained text-native music generator. SART converts dance into a global-to-local mixture of kinematic rhythm views, while teacher-constrained adaptation introduces these controls while retaining the backbone's text/lyric interface. This positions RAMP as a task-specific bridge from dance kinematics to controllable music generation.
References
[1] You F, Fang M, Tang L, et al. MoMu-Diffusion: On learning long-term motion--music synchronization and correspondence[C]//Advances in Neural Information Processing Systems. 2024, 37.
[2] Yang K, Zhou X, Tang X, et al. BeatDance: A beat-based model-agnostic contrastive learning framework for music-dance retrieval[C]//Proceedings of the 2024 International Conference on Multimedia Retrieval. ACM, 2024: 11--19.
[3] Yang Z, Yang K, Tang X, et al. TokenDance: Token-to-token music-to-dance generation with bidirectional Mamba[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2026: 5520--5529.
[4] Chen K, Tan Z, Lei J, et al. ChoreoMaster: Choreography-oriented music-driven dance synthesis[J]. ACM Transactions on Graphics, 2021, 40(4): 1--13.
[5] Tseng J, Castellon R, Liu C K. EDGE: Editable dance generation from music[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 448--458.
[6] Tang X, et al. CustomDance: Customized 3D dance generation with coarse-to-fine human-centered interactive control[J]. arXiv preprint arXiv:2608.06722, 2026.
[7] Yang K, Tang X, Peng Z, et al. FlowerDance: MeanFlow for efficient and refined 3D dance generation[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026: 56--74.
Response to Reviewer s5yG — W1: Long-Form Evaluation
We thank the reviewer for highlighting this limitation. Accordingly, we have added a long-form evaluation on 81 continuous choreographies: 72 25-second AIST clips and 9 approximately 30-second crawler clips. Importantly, RAMP is trained only on five-second clips; these results therefore evaluate zero-shot temporal extrapolation. Each soundtrack is generated in a single diffusion pass, without long-duration fine-tuning, segment-wise generation, or audio stitching.
To assess musical development and phrase transitions, we derive structural boundaries independently from choreography and audio novelty curves and report objective cross-modal proxies. For both Gen. and Ref., Boundary F1 and offset compare audio-derived boundaries against the same choreography-derived boundaries, while novelty correlation compares the corresponding audio and choreography novelty curves. “Reference” denotes the ground-truth soundtrack processed by the same automatic protocol.
Boundary offset is averaged over successfully matched boundary pairs.
Despite being trained only on short clips, these automatic proxies provide evidence of structural continuity and sustained fine-grained dance--music correspondence over 25--30 seconds, with only modest temporal degradation. We have added the protocol, results, and this short-clip-training limitation to the revised manuscript.