acceptodds
Under review as a conference paper at ICLR 2027

ChoreoStudio: Planning What, When, and Who in Music-Driven Group Choreography

Abstract

Music-driven group dance generation must decide not only how each dancer moves, but also what formation the group should form, when it should change, and who should execute each transition. Existing methods can generate orderly and rhythmically synchronized group motion, but remain limited in formation evolution and inter-dancer interaction. In human choreography, formation transitions often align with changes in high-level musical structure, such as the progression from an introduction to a chorus or from a chorus to a climax. We present ChoreoStudio, which makes choreography planning explicit. A music disentangler first decomposes the audio into frame-level rhythmic features and segment-level structural features. Given the segment-level features and sparse labels, a frozen audio-language model acts as a scriptwriter, producing a global semantic description that guides the Director's choreography planning. Based on this context, the Director produces three coupled interaction decisions: a graph-structured formation representation specifies what spatial organization to produce, a transition gate determines when it should change, and a role-assignment vector indicates who should execute the transition. The transition gate and role assignment are further conditioned on the planned geometric change, coupling formation targets with their temporal and dancer-specific execution. At a finer temporal scale, frame-aligned music features condition the diffusion generator to control detailed body motion and preserve beat-level alignment. The segment-level interaction plan is injected in parallel through frame-aligned instruction tokens and cross-dancer attention biases, coordinating how the generator realize each transition. Experiments on AIOZ-GDance demonstrate that ChoreoStudio follows the planned interactions while achieving state-of-the-art performance in motion quality, group organization, and formation dynamics.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Group Dance Generation, Diffusion Model, Multimodal Generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_gsTeOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces ChoreoStudio, a structured choreography planning framework for music-driven group dance generation, aiming to improve formation evolution and inter-dancer coordination. The key idea is to explicitly decompose group choreography into three planning decisions, including formation geometry (what), transition timing (when), and dancer mobility (who), before generating the detailed body motions. A multi-scale music encoder extracts segment-level structural information used for director planning and frame-level rhythmic features for generating detailed motion. A Director is introduced to predict relative formation coordinates, dancer relations, transition timing, and mobility to construct a group choreography plan. The group plan is then encoded into dancer-specific conditioning features and cross-dancer attention biases to guide a diffusion-based motion generator. Additionally, a verifier evaluates the generated motion and provides feedback for iterative replanning. Experiments on AIOZ-GDance show that the proposed framework improves group-motion realism, coordination, and formation dynamics over existing baselines.

Critical strengths
  • This paper addresses a meaningful limitation of existing music-driven group dance generation methods, which often focus on local motion synchronization and coordination but lack coherent formation across different music intervals. Explicitly modeling group-level choreography is well-motivated and important for generating more natural and structured group dance performances.

  • The proposed idea of decomposing group choreography into formation geometry (what), transition timing (when), and dancer mobility (who) is interesting and meaningful for the problem. Jointly modeling these aspects is a promising direction for incorporating high-level choreography planning into the motion generator, rather than relying entirely on implicit coordination within the dance generator.

Critical weaknesses
  • Although the proposed what-when-who decomposition is interesting, I think the technical framework is not sufficiently justified to support this key claim of explicit choreography planning and guidance. Despite the elaborate formulation of formation geometry, dancer relations, transition timing, and mobility, these variables are mainly encoded into conditioning features or attention bias, and serve as inputs for the diffusion motion generator, rather than explicitly enforcing their constraints during denoising. Although the intervention experiments show some response to plan changes, it remains unclear whether the generator can reliably follow these choreography decisions or simply treats them as additional conditioning information.
  • The overall framework seems overly complicated relative to the actual improvements. The system introduces multiple conditioning features, planning variables, and a likely expensive verification and replanning process. However, the gains over some baselines (e.g., TCDiff++) are relatively modest, while removing the Director or Verifier even results in worse performance than this baseline (Tables 1 and 3). The paper lacks sufficient justification for why such a complex design is necessary, and computational efficiency is also not investigated. In turn, this also makes the methodology quite difficult to follow. Several important variables, such as the group state SS, the feedback FF, group representation hˉ\bar{h}, and state change Δ\Delta is not clearly defined or explained in the main text.
Questions for authors
  • Could the authors clarify how the proposed plan-to-motion interface provides explicit choreography guidance beyond conventional feature conditioning? How can we verify that the generated motion faithfully follows each choreography decision, particularly transition timing and dancer mobility, rather than simply responding to changes in conditioning features?

  • Could the authors further justify the necessity of the proposed what-when-who decomposition? What are the specific advantages of modeling these variables separately rather than using simpler spatial or trajectory-based conditioning?

  • Given the complexity of the entire framework, could the authors clarify the performance gains over simpler baselines such as TCDiff++ ( which jointly generates group dance through a more straightforward diffusion process without iterative verification and replanning)? Could the authors also provide a performance-efficiency analysis to better justify the complexity?

  • Could the authors analyse the computational overhead introduced by the Director, multimodal conditioning and iterative verification and replanning, compared with existing baselines?

3 comments
AuthorsOpenReview ↗

Clarifications on Explicit Planning and Efficiency

Thank you for the thoughtful comments. We have clarified the plan-to-motion interface and its limitations, expanded the explanation of the planning variables, and added a performance–efficiency analysis.

1. Explicit guidance and plan realization. We agree that the plan does not impose hard constraints at every denoising step. It guides the diffusion generator through frame-aligned instructions and relation-aware attention biases. Its explicitness lies in the intermediate decisions: formation, transition timing, and dancer mobility have defined meanings, are supervised using targets extracted from motion, and can be inspected and edited before generation. We have clarified this distinction in the method section.

To test whether these decisions affect the intended motion properties, Table 4 reports paired interventions that edit one plan component at a time while fixing the music, initial state, actor seeds, and diffusion noise. Dependent quantities are recomputed as required after each edit. These experiments use a single generation cycle without Verifier replanning or candidate selection. Formation, timing, and mobility edits reduce their respective errors toward the edited targets by 40%, 37%, and 29%; the generated motion changes in the requested direction in 78.54%, 74.85%, and 68.68% of cases, respectively.

To facilitate qualitative inspection, our supplementary demos also show the generated semantic descriptions alongside the corresponding motions, allowing readers to compare the intended high-level choreography with the visual result. Together with the controlled interventions, these examples make the guidance and its realization observable, although they do not guarantee faithful realization for every sample.

2. Why what–when–who? A target formation alone does not determine when a transition occurs or which dancers perform the movement: the same spatial change can be executed at different musical times or with different mobility allocations. We therefore represent formation, timing, and mobility as distinct but coupled decisions, with timing and mobility conditioned on the planned geometric change. This makes the alternatives separately editable while preserving their dependency. The component ablations in Table 3 show that removing transition timing, role assignment, or the relation graph degrades performance; the interventions further test whether edits to each decision produce the intended response. We have added this motivation to Section 3.3. We do not claim that these experiments establish superiority over every possible trajectory-only conditioning design.

3. Benefit relative to TCDiff++ and component ablations. Under our revised common 150-frame evaluation, three-dancer ChoreoStudio obtains GMR/GMC/TIF of 10.32 ± 0.36 / 83.31 ± 0.38 / 0.07 ± 0.01, compared with 11.42 ± 0.46 / 82.37 ± 0.37 / 0.10 ± 0.01 for our TCDiff++ reproduction. These results are scoped to the models evaluated under our protocol. We agree that removing the Director or Verifier can yield worse results than this baseline. The ablations indicate that these components contribute to the full system. Table 3 reports single-run ablations, whereas Table 1 reports ten-run means; we have clarified their different statistical settings.

4. Efficiency and overhead. We added an efficiency comparison for three-dancer, 150-frame generation and latency measurements for refinement configurations. Training uses two RTX 4090 GPUs, and inference is measured on one RTX 4090. With precomputed semantic descriptions, text embeddings, and music features, ChoreoStudio takes 17 s per sample under the measured configuration, compared with 11 s for TCDiff++. Preparation time for these conditions is excluded. We acknowledge the additional inference cost.

The measured ChoreoStudio components contain 64.24M parameters: 48.45M for the generator, 9.63M for the Director, and 6.16M for the music disentangler. This excludes the frozen omni-modal model and text encoder. A single 50-step cycle provides a lower-latency option at 9 s, compared with 18 s for two 50-step feedback cycles, with GMR/GMC of 15.43/79.32 versus 10.36/83.20. At the same 100-denoising-evaluation budget, two feedback cycles improve GMR/GMC over a single 100-step cycle (10.36/83.20 vs. 14.74/80.94; both 18 s). These results show the quality–latency trade-off. 5. Clarity. We expanded the main-text explanations of the group state and Verifier feedback, described their use in replanning, and clarified the intermediate plan. We also added a limitations section discussing inference cost and the fact that plan realization is guided rather than strictly enforced.

Dear Authors,

Thank you for the clarifications, which have mostly addressed my concerns. I would remain positive about the paper. However, I still believe the paper could further benefit from additional analysis and experiments by comparing with simpler spatial or trajectory-conditioning baselines, which can potentially highlight the effectiveness of the proposed what-when-who decomposition.

Best

AuthorsOpenReview ↗

Thank You for Your Feedback

Dear Reviewer,

Thank you for your positive assessment and for acknowledging our clarifications. We agree that a comparison with simpler spatial or trajectory-conditioning baselines would help clarify the contribution of the what–when–who decomposition.

A controlled comparison would require implementing and retraining these alternatives under the same generator architecture and evaluation protocol. As this goes beyond the experiments completed for the current response, we will consider it in future work.

Best regards,
The Authors

Reviewer_dZNBOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces a method, ChoreoStudio, for multi-dancer choreography generation from music. Unlike previous methods, the authors tackle two main challenges: (1) dancers are exchangeable prior to generation, whereas choreography naturally assigns asymmetric roles to different dancers, and (2) incorporating dancer formation explicitly during training.

During training, "teacher plans" are automatically extracted from ground-truth motions, and a Director module is trained via supervised learning to predict these explicit plans from music features and descriptions (derived from a frozen Qwen3-Omni model), while a diffusion model is trained to realize the full-body motion. At inference time, it operates in a closed-loop planning–generation–verification pipeline across refinement cycles.

The authors conduct an extensive evaluation and compare their performance against prior works, demonstrating high superiority across a range of metrics. The quantitative results show decent results and surpassing prior works.

Critical strengths

This paper tackles an under-explored and interesting task with limited data.


  • The paper is well-written, and the method is explained in detail.
  • The authors carefully model and address each sub-objective, using intuitively design choices

  • The authors address many common challenges of prior works and report both qualitative and quantitative comparisons.

  • The evaluation metrics cover different evaluation aspects, and the authors seem to surpass prior works.

  • The qualitative results are plausible and convincing.
Critical weaknesses
  • High Complexity: The overall method is extremely complex. Even though each component is claimed to highly contribute, the paper would benefit from exploring how to make the system less complex (a point the authors never address).


  • Lack of Latency Analysis: The paper lacks an explicit analysis of inference runtime and latency, which is an essential factor given the highly complex nature of the method.


  • Missing Limitations Section: The manuscript lacks a dedicated limitations section.

  • Missing citation and comparison to "CoheDancers: Enhancing Interactive Group Dance Generation through Music-Driven Coherence Decomposition" (Yang et. al.)

Questions for authors
  • Section 3.3: Given a segment of music and N dancers, why is masking needed, and what defines unavailable motion?


  • Line 280: There is a repetition of the word "pairwise."


  • The abstract contains too many low-level implementation details, making it hard to follow.

  • The authors are highly encourages to release the code, as independent components may contribute to future research in generative planning or conditional

2 comments
AuthorsOpenReview ↗

Efficiency Analysis, CoheDancers Comparison, and Clarifications

Thank you for the constructive feedback. We have added a quality–latency analysis, an efficiency comparison, a limitations section, and a comparison with CoheDancers. We also clarified the mask, revised the abstract, and corrected the typo.

1. Complexity and simplification. We agree that the benefits of the components should be considered alongside their cost. We extended Table 9 with inference latency. A single cycle with 50 denoising steps provides a faster alternative: 9 s, compared with 18 s for two 50-step feedback cycles, at the cost of higher GMR and lower GMC (15.43/79.32 vs. 10.36/83.20).

At the same budget of 100 denoising evaluations, two feedback cycles improve GMR/GMC over a single 100-step cycle (10.36/83.20 vs. 14.74/80.94; both 18 s) and over selecting the better of two independent 50-step samples (10.36/83.20 vs. 13.62/81.29; 18 vs. 17 s). Thus, reducing refinement cycles offers a quality–latency trade-off, while feedback improves quality under the matched budget.

2. Latency and efficiency. We added an efficiency comparison for three-dancer, 150-frame generation. Training uses two NVIDIA RTX 4090 GPUs, and inference is measured on one RTX 4090. ChoreoStudio takes 17 s per sample under the measured configuration, compared with 11 s for TCDiff++, 6 s for GCD, 4 s for ST-GDance, and 3 s for CoDancers. Semantic descriptions, text embeddings, and music features are precomputed; their preparation time is excluded from inference latency. The measured components total 64.24M parameters: 48.45M for the generator, 9.63M for the Director, and 6.16M for the music disentangler, excluding the frozen omni-modal model and text encoder. While planning and refinement add latency, the generator remains compact, and structured guidance helps it produce higher-quality group motion. Accelerating this process is future work.

3. Limitations. We added a dedicated limitations section in the appendix, discussing the cost of iterative refinement, the quality–latency trade-off, and limitations of relative-geometry planning, including the absence of explicit absolute stage-orientation control and richer interaction semantics.

4. CoheDancers. Thank you for pointing out this omission. We added CoheDancers to related work and a three-dancer, 150-frame comparison to the appendix. CoheDancers originally conditions on initial dancer poses. To compare without provided initial motion, we adapted its initialization using noise; the comparison therefore applies to this adapted setting rather than its original pose-conditioned protocol.

MethodGMR↓GMC↑TIF↓IDDV → GT
GT—86.820.0040.302
CoheDancers (adapted)13.52 ± 0.4781.45 ± 0.520.09 ± 0.020.218 ± 0.032
ChoreoStudio10.32 ± 0.3683.31 ± 0.380.07 ± 0.010.262 ± 0.038

Under this setting, ChoreoStudio achieves lower GMR and TIF, higher GMC, and IDDV closer to GT. We have made the initialization difference explicit.

5. Masking in Section 3.3. The mask distinguishes a valid dancer whose motion is unavailable from a padded dancer who does not belong to the group. In the first planning cycle, generation starts from music and the specified dancer count without a motion sequence, so valid dancers are marked motion-unknown. In later cycles, motion generated in the preceding cycle provides the group state. We clarified this distinction in the text.

6. Typo and abstract. We corrected the repeated “pairwise” and revised the abstract to emphasize the motivation, explicit planning, and main results while reducing implementation details.

7. Code release. We will release the code upon acceptance, including the training and evaluation pipelines and the independent planning components, to support reproducibility and future research.

AuthorsOpenReview ↗

Thank You for Your Positive Assessment and Feedback

Dear Reviewer,

Thank you very much for your positive assessment and for the time and care you devoted to reviewing our paper. We appreciate your constructive suggestions on model complexity, inference latency, limitations, and the comparison with CoheDancers. Your feedback helped us improve both the experimental analysis and the clarity of the paper.

We would be happy to provide any further clarification if needed.

Best regards,
The Authors

Reviewer_yJbtOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Task. Input: a music clip + tags for music style, dance style and dancer count N (2–7); output: an N × L (frame #) × 151 array of 3D dance; evaluated on AIOZ-GDance (1,309 train / 154 test sequences).

Problem. Existing group-dance generators keep dancers locally consistent but never decide the choreography itself (which formation, when to change it, which dancers move), so generated groups jitter in place instead of slowly reorganizing as real groups do.

Key insight. Insert an explicit, editable plan between music and motion: a trained Director outputs per music segment a target formation and relation graph (what), a transition switch (when) and a per-dancer mobility share (who), a diffusion generator synthesizes motion under this plan, and a rule-based verifier sends failures back for re-planning.

Result. On AIOZ-GDance with 2–4 dancers the paper reports the best group-realism and group-correlation scores against five baselines, closer-to-real slow formation dynamics on three new metrics, ablations and plan-editing interventions in its favour, and a 30-person user study, all from one checkpoint that serves every group size. The videos are good

What I learned. The "what/when/who" interface and the diagnostic that generators under-produce slow formation change are useful, but the baseline table mixes numbers copied from other papers' protocols with the authors' own, lists the strongest baseline below its own published numbers (which would beat the proposed method), so I feel the gains doubtful.

Critical strengths
  1. The diagnosis is right and reusable. Splitting inter-dancer motion into slow (formation) and fast (jitter) components shows that current generators keep the fast part but under-produce the slow part (IDDV 0.18 vs 0.30 for real data), and the resulting metrics apply to any group-motion method.

  2. The plan is an explicit, testable interface. Formation, timing and mobility are extracted automatically from ground-truth root trajectories and can be edited at test time, and the intervention experiments (Table 4) are the right kind of evidence for such an interface.

  3. The video demos show visual performance are good!

Critical weaknesses

Table 1 appears to mix numbers copied from other papers' protocols with the authors' own, contrary to 4.1. The GCD, CoDancers and ST-GDance 1- and 3-dancer entries (e.g. 39.24/9.64/2.53, 30.22/80.22/0.19, 26.34/74.22, 14.76/81.24/0.11) are digit-for-digit the rows of the ST-GDance paper (BMVC 2025, a 120-frame 3-dancer protocol), the GCD and CoDancers 4-dancer rows (36.28/81.82/0.13, 26.44/75.34/0.10) are the GCD and TCDiff papers' rows, while TCDiff++ is listed with GMR 15.77/11.57/13.63 although its own paper reports 12.07/9.75/10.02, all of which would beat ChoreoStudio (13.56/10.42/11.94). The headline comparison therefore cannot be trusted as it stands, and this is decisive for my recommendation. The authors need to explain what happened.

Questions for authors
  1. For every entry in Tables 1 and 7, state whether it was produced by your own retraining under the 150-frame protocol or taken from a published paper; several entries are identical to the ST-GDance, GCD and TCDiff papers, which 4.1 says you did not do.
  2. Why is TCDiff++'s GMR 1.8–3.6 points worse than its published values? Please evaluate the original authors' released checkpoints (you already use TCDiff++'s in App. C.2) under your protocol and add a ground-truth row to Table 1.
  3. Why is the best baseline's beat-alignment score (0.224, Table 2) so far below ground truth (0.363), and which method is that "best baseline" (4.2 says TCDiff++, App. A.4 and Fig. 5 say TCDiff / ST-GDance)?
  4. Related work on multi-dancer generation cites only DuetGen (2025) for the two-dancer case and omits the earlier Duolando (ICLR 2024),
3 comments
AuthorsOpenReview ↗

Corrected Evaluation Protocol and Baseline Comparisons

Thank you for carefully checking our results against the original papers. Your comments helped us identify errors in both the comparison and its description. We sincerely apologize. We audited the model/checkpoint sources and evaluation settings, corrected the affected results, and revised the paper.

1. Mixed protocols and provenance (Critical Weakness; Q1). You are correct: the original Table 1 reused earlier results without adequately checking protocol compatibility. These included 120-frame one- and three-dancer results for GCD, CoDancers, and ST-GDance, and published 150-frame four-dancer results for GCD and CoDancers. The original Section 4.1 therefore incorrectly described the comparison as using a shared protocol.

For the revised Table 1, we retrained and evaluated all four baselines—GCD, CoDancers, TCDiff++, and ST-GDance—using their official public implementations. We retained the original hyperparameters where feasible, with smaller batch sizes where required by GPU memory limits. Every generated entry, including ChoreoStudio, is now evaluated by us under a shared 150-frame protocol for the indicated dancer count. Section 4.1 and the evaluation appendix describe this setting. Table 1 reports mean ± SD over ten complete generation/evaluation runs, with fixed checkpoints and test clips and different sampling seeds; these are not independent training runs.

Table 7 uses 720 frames. Its original TCDiff++ three-dancer entry reused published long-sequence scores without verifying their applicability to our setting or identifying their source. Our own three-dancer evaluation replaces it with GMR/GMC/TIF = 14.38/81.25/0.15. All other entries were already evaluated by us. Thus, all generated entries in revised Tables 1 and 7 are our own evaluations. Appendix C.2 now specifies the 720-frame model sources: GCD, CoDancers, ST-GDance, and ChoreoStudio were trained for this length; TCDiff++ uses our previously trained 720-frame checkpoint.

2. TCDiff++ discrepancy and GT (Q2). We could not locate author-released TCDiff++ weights in the official repository. The “existing checkpoint” in Appendix C.2 was our own previously trained checkpoint, not the original authors’ weights; we have clarified this. Our reproduction uses the official code but smaller batch sizes than those reported in the paper due to GPU memory limits. It therefore uses independently trained weights and a modified training configuration. Smaller batches may affect optimization and contribute to the reproduction gap, but we have not isolated their effect in a controlled comparison.

DancersTCDiff++ publishedTCDiff++ reproducedChoreoStudio
212.0715.43 ± 0.4513.20 ± 0.40
39.7511.42 ± 0.4610.32 ± 0.36
410.0212.96 ± 0.6711.97 ± 0.28

The gap persists across repeated sampling; the SDs capture sampling variability at fixed weights, not variation across training runs. We acknowledge that the published TCDiff++ GMR values are lower than ChoreoStudio’s. We restrict our ranking claims to models evaluated under our common protocol. Table 1 now also includes GT references for Div, PFC, GMC, and TIF; dashes omit real-set self-comparisons for FID and GMR, and GT is excluded from rankings.

3. Baseline identity and beat alignment (Q3). The baseline with BAS 0.224 is TCDiff++. The inconsistent names were labeling errors. We changed “Best baseline” to “TCDiff++” and corrected the corresponding references in Section 4.2, Appendix A.4, and Figure 5. The separate ST-GDance series and the underlying scores remain unchanged.

We use the Beat Alignment Score introduced by FACT (Li et al., ICCV 2021). Motion beats are detected as local minima of joint velocity. For each motion beat, the distance to the nearest music beat is converted into a score by a Gaussian kernel, then averaged across motion beats, dancers, and test sequences. The same evaluator is used for generated and real motion, with σ = 2.12 frames; the “Music-Dance Alignment” subsection of Appendix A provides the full definition.

TCDiff++ obtains 0.224, versus 0.363 for real motion and 0.351 for ChoreoStudio. TCDiff++ uses music features including beat information, but its reported training objective has no separate beat-alignment loss. ChoreoStudio additionally considers beat alignment when its Verifier scores generated candidates. This difference may contribute to the gap, but our current results do not isolate its individual effect.

4. Duolando (Q4). We added Duolando (Li et al., ICLR 2024) to Section 2.1 and the bibliography, describing its generation of a follower’s motion conditioned on music and a leader’s motion alongside DuetGen. Thank you for identifying this omission.

AuthorsOpenReview ↗

Thank You for Your Reassessment

Dear Reviewer,

Thank you very much for the time and care you devoted to reviewing our paper and reassessing it after our response. We are especially grateful for your thorough examination of the baseline results and evaluation protocols. Your detailed feedback helped us identify and correct important inconsistencies, improve the transparency of our experiments, and strengthen the paper.

We truly appreciate your constructive suggestions and positive reassessment.

Best regards,
The Authors

Since the authors provide unified evaluations, I increased my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.