acceptodds
Under review as a conference paper at ICLR 2027

X-Dancers: Group Choreography as Masked Completion of a Formation-Grounded Group Motion Tensor

Abstract

Music-driven 3D group choreography requires jointly modeling individual motion, evolving formations, global trajectories and their synchronization with music. Existing methods model these factors separately and rely on dedicated mechanisms for different choreographic operations, limiting flexibility and generalization. We present **X-Dancers**, a unified framework that formulates group choreography as *bidirectional masked generation* over a continuous group motion tensor, where each cell couples a dancer's motion representation with its root state to jointly represent motion, formation and trajectory. To learn their joint distribution, we introduce the *ChoreoGraph Transformer*, which uses interleaved temporal and formation-grounded spatial attention to reason over observed group context, while masked conditional diffusion generates the unknown tensor cells. A spatial-temporal-channel masking strategy exposes the model to diverse conditioning patterns, enabling choreography synthesis, long-duration continuation, cast extension and trajectory control via different conditioning masks without task-specific retraining. The experimental results demonstrate its state-of-the-art motion fidelity, music synchronization and formation quality, together with strong zero-shot generalization to long-duration choreography.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
human motion generation, music-to-dance synthesis, multi-person motion synthesis, group motion synthesis

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_j9T1OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

X-Dancers formulates music-driven group choreography as masked completion of a continuous group motion tensor that combines individual motion representations with explicit root states. Its ChoreoGraph Transformer interleaves temporal attention with formation-grounded spatial attention, using descriptors computed from currently visible dancer positions. Spatial-temporal-channel masking enables one model to support choreography synthesis, continuation, cast extension, and trajectory control without task-specific retraining. Experiments on AIOZ-GDANCE, component ablations, and a user study demonstrate promising motion quality and group coordination under the reported trajectory-conditioned evaluation protocol.

Critical strengths
  1. Unified formulation. Combining motion and root states in one tensor enables synthesis, continuation, cast extension, and trajectory control through different masks without task-specific retraining.
  2. Formation-aware attention. Spatial attention uses the currently visible formation and updates as generation proceeds, providing a suitable mechanism for reasoning about evolving group geometry.
  3. Targeted ablation support. Removing formation descriptors while retaining explicit root states increases GC-FID from 7.64 to 13.72, supporting their contribution beyond raw position information.
  4. Strong experimental results. Under the trajectory-conditioned protocol, SC-FID improves from 16.21 to 7.25 over TCDiff++. Five-seed results and a 20-participant user study provide additional support.
Critical weaknesses
  1. Limited evaluation of autonomous formation generation. Quantitative comparisons provide ground-truth root trajectories, so they primarily demonstrate trajectory-conditioned motion quality. How does the method perform in fully masked composition without these trajectories?
  2. Confounded masking ablation. Spatial-only and temporal-only variants also remove channel masking, which directly matches the evaluation setting. Could the authors retain channel masking while independently removing spatial or temporal masks to isolate their contributions?
  3. Incomplete positioning within music-driven dance generation. Beyond the cited approaches, relevant work includes autoregressive models such as Transflower and MEGADance, flow-based methods such as DisCoRD and FlowerDance, and retrieval-guided systems such as ChoreoMaster, MotionRAG-Diff, and CustomDance. Discussing their relationship to the proposed framework would better clarify its contributions to fidelity, long-duration generation, and editing/control.
Questions for authors

See Weaknesses. I will raise my score if my concerns are adequately addressed.

4 comments
AuthorsOpenReview ↗

Response to Reviewer j9T1: W1

W1: Limited evaluation of autonomous formation generation. Quantitative comparisons provide ground-truth root trajectories, so they primarily demonstrate trajectory-conditioned motion quality. How does the method perform in fully masked composition without these trajectories?


We agree with the reviewer that a comparison under ground-truth root trajectories primarily demonstrates trajectory-conditioned motion quality and leaves autonomous formation generation unevaluated. To answer how the method performs without these trajectories, we have re-evaluated all methods and ablations in fully masked composition, where no root state is supplied and every method generates the formation together with the motions from the music and the cast size alone. X-Dancers attains the best formation quality among all methods in this setting, and it remains the best method on five of the six metrics. The ablations attribute the result to the joint progressive generation of root state and motion token and to formation-frame attention. The previous protocol is retained as a trajectory control experiment, in which the same model follows prescribed root paths most closely. The updated results are reported as the mean and standard deviation over five sampling seeds.

Table R1. Baseline comparison in fully masked composition (Table 1 of the revised paper). Mean ± std over five seeds. Best in bold, second best underlined.

MethodSC-FID ↓BAS ↑Div ↑FSD (mm) ↓GC-FID ↓Coll. (%) ↓
GCD49.20±1.7249.20^{\pm 1.72}0.365‾±0.002‾\underline{0.365}^{\pm \underline{0.002}}9.27±0.079.27^{\pm 0.07}16.25±0.0016.25^{\pm \mathbf{0.00}}90.43±2.3690.43^{\pm 2.36}2.054±0.0782.054^{\pm 0.078}
CoDancers29.56±0.8529.56^{\pm 0.85}0.334±0.0010.334^{\pm \mathbf{0.001}}7.13±0.037.13^{\pm \mathbf{0.03}}14.48‾±0.00\underline{14.48}^{\pm \mathbf{0.00}}35.28±1.1435.28^{\pm 1.14}1.936±0.1251.936^{\pm 0.125}
TCDiff28.39±0.6428.39^{\pm 0.64}0.361±0.002‾0.361^{\pm \underline{0.002}}11.70±0.0911.70^{\pm 0.09}21.93±0.01‾21.93^{\pm \underline{0.01}}53.84±0.6853.84^{\pm 0.68}2.829±0.1912.829^{\pm 0.191}
FreeDance52.15±1.3852.15^{\pm 1.38}0.356±0.0010.356^{\pm \mathbf{0.001}}5.02±0.035.02^{\pm \mathbf{0.03}}31.46±0.0831.46^{\pm 0.08}102.57±2.71102.57^{\pm 2.71}1.415‾±0.236\underline{1.415}^{\pm 0.236}
TCDiff++21.53‾±0.42‾\underline{21.53}^{\pm \underline{0.42}}0.364±0.002‾0.364^{\pm \underline{0.002}}13.81±0.04‾\mathbf{13.81}^{\pm \underline{0.04}}15.47±0.0015.47^{\pm \mathbf{0.00}}25.68‾±0.53‾\underline{25.68}^{\pm \underline{0.53}}1.660±0.0131.660^{\pm \mathbf{0.013}}
X-Dancers (Ours)17.28±0.13\mathbf{17.28}^{\pm \mathbf{0.13}}0.368±0.001\mathbf{0.368}^{\pm \mathbf{0.001}}12.63‾±0.06\underline{12.63}^{\pm 0.06}9.31±0.00\mathbf{9.31}^{\pm \mathbf{0.00}}18.97±0.22\mathbf{18.97}^{\pm \mathbf{0.22}}0.974±0.027‾\mathbf{0.974}^{\pm \underline{0.027}}
Ground Truth–0.367\mathrm{0.367}9.48\mathrm{9.48}4.70\mathrm{4.70}–0.956\mathrm{0.956}

X-Dancers remains the state-of-the-art method when the formation is generated autonomously. As no root trajectory is supplied, GC-FID and the collision rate now measure the formation that each method generates, and X-Dancers attains the best GC-FID and the collision rate closest to that of the ground truth. It also achieves the best SC-FID, BAS and FSD, and the autonomous formation is therefore not obtained at the cost of the motion quality of each dancer. TCDiff++ is the strongest baseline and generates trajectories and motions in one stage, yet it distinguishes dancers by identity encodings without conditioning their poses on the formation, which yields a higher diversity at a lower fidelity of the generated choreography. Our method instead makes motion synthesis aware of the formation by coupling a continuous motion token with an explicit root state in every cell, and by grounding inter-dancer attention in the evolving formation through the formation-frame descriptors computed from the root states. As the dance motions are correlated and refined together with the evolving formation, the model learns to generate casts with natural spacing and coherent formations while keeping the motion fidelity of each dancer, although no formation or root trajectory is provided in advance.

The ablations in fully masked composition trace the formation quality to the design choices. The ablation study is re-evaluated under the same setting and is given as Table R2 in our response to W2. Removing the formation descriptors while keeping the explicit root state in every cell (C2) raises GC-FID from 18.97 to 27.91 and the collision rate from 0.974% to 1.105%. Replacing formation-frame attention with graph attention (C1) raises them further to 32.42 and 1.427%. The root states alone therefore do not make the attention across dancers aware of the formation, and it is the formation frame built from them that does. The effect of the descriptor ablation, which the reviewer lists among the strengths, thus persists when the formation itself is generated.

AuthorsOpenReview ↗

Response to Reviewer j9T1: W2

W2: Confounded masking ablation. Spatial-only and temporal-only variants also remove channel masking, which directly matches the evaluation setting. Could the authors retain channel masking while independently removing spatial or temporal masks to isolate their contributions?


Thank you for this suggestion, which removes a confound from our masking ablation. Following it, we have trained the two requested variants. The conclusion of the masking ablation holds with channel masking retained. Each mask family still contributes on its own, and the gain of STC-Mask comes from combining spatial and temporal completion and not from a match between channel masking and the evaluation setting. The updated study is presented Table 2 of the revised paper and is attached below as Table R2.

Table R2. Ablation study in fully masked composition (Table 2 of the revised paper). Mean ± std over five seeds. Best in bold, second best underlined.

MethodSC-FID ↓BAS ↑Div ↑FSD (mm) ↓GC-FID ↓Coll. (%) ↓
A1: VQVAE-based masked transformer54.29±2.0154.29^{\pm 2.01}0.356±0.0000.356^{\pm \mathbf{0.000}}4.52±0.384.52^{\pm 0.38}15.81±0.2515.81^{\pm 0.25}105.93±3.40105.93^{\pm 3.40}1.327±0.1281.327^{\pm 0.128}
A2: Diffusion transformer37.50±0.9037.50^{\pm 0.90}0.364±0.001‾0.364^{\pm \underline{0.001}}8.60±0.08‾8.60^{\pm \underline{0.08}}13.94±0.1613.94^{\pm 0.16}69.18±1.4669.18^{\pm 1.46}1.982±0.0721.982^{\pm 0.072}
B1: Spatial-only masking29.51±0.8529.51^{\pm 0.85}0.362±0.001‾0.362^{\pm \underline{0.001}}6.63±0.756.63^{\pm 0.75}11.06±0.0911.06^{\pm 0.09}32.59±0.8132.59^{\pm 0.81}1.209±0.0311.209^{\pm 0.031}
B2: Temporal-only masking26.93±0.1826.93^{\pm 0.18}0.365±0.0000.365^{\pm \mathbf{0.000}}9.84±0.269.84^{\pm 0.26}10.13±0.02‾10.13^{\pm \underline{0.02}}47.28±1.3547.28^{\pm 1.35}1.626±0.0071.626^{\pm \mathbf{0.007}}
B3: Spatial + channel masking (new)27.64±1.1927.64^{\pm 1.19}0.363±0.001‾0.363^{\pm \underline{0.001}}6.04±0.516.04^{\pm 0.51}9.42‾±0.13\underline{9.42}^{\pm 0.13}31.65±0.7331.65^{\pm 0.73}1.185±0.0241.185^{\pm 0.024}
B4: Temporal + channel masking (new)24.80±0.15‾24.80^{\pm \underline{0.15}}0.365±0.001‾0.365^{\pm \underline{0.001}}8.72±0.198.72^{\pm 0.19}9.86±0.039.86^{\pm 0.03}36.20±0.6136.20^{\pm 0.61}1.315±0.010‾1.315^{\pm \underline{0.010}}
C1: Graph attention22.73±0.5422.73^{\pm 0.54}0.358±0.0000.358^{\pm \mathbf{0.000}}6.95±0.126.95^{\pm 0.12}10.57±0.1810.57^{\pm 0.18}32.42±0.48‾32.42^{\pm \underline{0.48}}1.427±0.0321.427^{\pm 0.032}
C2: w/o formation descriptors19.34‾±0.41\underline{19.34}^{\pm 0.41}0.374±0.000\mathbf{0.374}^{\pm \mathbf{0.000}}11.61‾±0.25\underline{11.61}^{\pm 0.25}10.29±0.0410.29^{\pm 0.04}27.91‾±0.57\underline{27.91}^{\pm 0.57}1.105‾±0.016\underline{1.105}^{\pm 0.016}
Ours (full STC-Mask)17.28±0.13\mathbf{17.28}^{\pm \mathbf{0.13}}0.368‾±0.001‾\underline{0.368}^{\pm \underline{0.001}}12.63±0.06\mathbf{12.63}^{\pm \mathbf{0.06}}9.31±0.00\mathbf{9.31}^{\pm \mathbf{0.00}}18.97±0.22\mathbf{18.97}^{\pm \mathbf{0.22}}0.974±0.027\mathbf{0.974}^{\pm 0.027}

B3 keeps the spatial and channel masks and removes the temporal masks, and B4 keeps the temporal and channel masks and removes the spatial masks, with channel masking at its original probability in both. In addition, the entire ablation study is now evaluated in fully masked composition, where no root state is visible at the start of sampling, and channel masking therefore no longer matches the evaluation setting of any variant. As shown in Table R2, removing either family still degrades every metric relative to the full strategy, and the two families contribute different abilities. Without temporal masks (B3), individual motion suffers most, with the higher SC-FID and the lower diversity of the two variants, as the model is never trained to complete time steps from their temporal context. Without spatial masks (B4), group quality suffers most, with the higher GC-FID and collision rate, as the model is never trained to complete a dancer from the other dancers. Channel masking is beneficial in itself, as B3 and B4 improve on B1 and B2 in SC-FID, FSD, GC-FID and collision rate at a slightly lower diversity, yet both variants remain far from the full strategy, with a GC-FID of 31.65 and 36.20 against 18.97. Section 4.2 and Appendix B of the revised paper define B3 and B4 and include this discussion.

AuthorsOpenReview ↗

Response to Reviewer j9T1: W3 & Ending

W3: Incomplete positioning within music-driven dance generation. Beyond the cited approaches, relevant work includes autoregressive models such as Transflower and MEGADance, flow-based methods such as DisCoRD and FlowerDance, and retrieval-guided systems such as ChoreoMaster, MotionRAG-Diff, and CustomDance. Discussing their relationship to the proposed framework would better clarify its contributions to fidelity, long-duration generation, and editing/control.


Thank you for pointing us to these works. We agree that discussing them clarifies the positioning of the paper, and we have added the discussion to Section 2.2 and Appendix J of the revised paper. All seven comparison works address single-dancer generation, and our proposed X-Dancers differs from them in modeling the motion and the placement of all dancers jointly and in unifying composition, continuation, cast extension and trajectory control as conditioning masks of one model. Their relationship to X-Dancers in the three aspects that the reviewer names is summarized below.

For long-duration generation, the autoregressive models reach long sequences through causal prediction. Transflower predicts a distribution over the next poses with a normalizing flow conditioned on past poses and music through a multimodal transformer, and MEGADance predicts quantized dance codes with genre-routed mixtures of autoregressive experts and a sliding window for long sequences. X-Dancers obtains continuation from a temporal prefix mask within bidirectional masked generation, and the same model thereby also conditions on future and cross-dancer context, which a causal factorization does not expose.

For fidelity, the flow-based methods improve the quality or the efficiency of single-dancer motion in a continuous space. DisCoRD decodes discrete motion tokens into continuous motion with rectified flow, and FlowerDance generates an entire sequence in a few steps with MeanFlow and a BiMamba backbone. X-Dancers shares the continuous formulation, as each cell is sampled by a diffusion head trained with a velocity objective, but it generates continuous tokens directly instead of decoding discrete ones, and each cell couples the motion token with the root state.

For editing and control, the retrieval-guided systems derive their fidelity and control from an external motion library and from phrase-level user choices. ChoreoMaster synthesizes dance along a motion graph built from a dance motion library under a choreomusical embedding, MotionRAG-Diff refines segments retrieved from such a graph with a diffusion model for long-term music, and CustomDance lets users select retrieved phrases that a music-conditioned diffusion model connects by inpainting. X-Dancers is library-free and exposes control through the conditioning mask over dancers, time steps and root channels, which the model learns to follow under STC-Mask. The two forms of control are complementary, as retrieved or user-selected phrases could be supplied to X-Dancers as visible cells.


We thank the reviewer again for the careful review and the constructive suggestions, which have aligned the evaluation with the joint-generation claim and clarified the positioning of the paper. We hope that our responses address the concerns, and we would be glad to provide any further clarification.

Dear Reviewer j9T1,

As the discussion period is closing, we would like to kindly ask whether our response has addressed your concerns. If any point remains unclear, we would be glad to clarify it before the deadline. We sincerely thank you for the time and the constructive feedback.

Best,

Paper 50089 Authors

Reviewer_a7fHOpenReview ↗
Rating
3(+1)
Confidence
3(−1)
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper works on music-driven group dance generation. The main idea is to write a group dance as an N x T grid, where N is dancer # and T is time. Each cell holds a motion token for that dancer at that step plus its root position and heading on the floor. Generation is then done as masked completion over this grid: the model sees some cells and fills in the rest. The paper talks on four tasks: (1) generating from music, (2) continuing a long piece window by window, (3) adding dancers to an existing cast, and (4) following user-given paths. These tasks are all just different ways of choosing which cells to hide.

The model is a transformer that alternates attention over time (within one dancer) and attention across dancers (within one time step). Since dancers have no natural order, the cross-dancer attention gets extra geometric features from the currently known root positions. Masked cells are produced by a small diffusion head, following the MAR style of continuous-token masked generation. Training randomly mixes several masking patterns so one model handles all four tasks.

Experiments are on AIOZ-GDANCE, after the authors cleaned the annotations. They compared with a bunch of recent group dancing works and report the best number on all of them. There is an ablation over token type (continuous vs. VQ), generation paradigm, masking families and the attention design, a short attention visualization, and a 20-person user study. Continuation, cast extension and trajectory control are shown with qualitative figures and videos only. The supplementary video shows favourable results.

The biggest issue to me, which makes me lean to reject is it writes "Quantitative comparisons supply the same ground-truth root trajectories to all methods." As I understand, the trajectory should be one of the choreograph. Therefore I don't feel the current evaluation is significant enough.

Critical strengths
  1. One model does all four tasks (generate from music, continue a dance, add dancers, follow given paths) just by changing which cells are masked. Previous methods needed a separate module for each. This is the main reason I think the paper is useful.
  2. The ablation for “formation-frame attention” is convincing: removing it (row C2) makes GC-FID go from 7.64 to 13.72 with nothing else changed. This shows the module itself works, independent of the other concerns I have about Table 1.
Critical weaknesses

Line 304: "Quantitative comparisons supply the same ground-truth root trajectories to all methods.". That is very weird to me. As root trajectory is one part of the choreograph, shouldn't the it be generated? Furthermore, what the supp video shows is different and you highlighted the improvement on trajectory.

Questions for authors

Explain the critical weakness and provide comparisons with generated trajectories.

4 comments
AuthorsOpenReview ↗

Response to Reviewer a7fH: W1 [1/2]

W1: Line 304: "Quantitative comparisons supply the same ground-truth root trajectories to all methods.". That is very weird to me. As root trajectory is one part of the choreograph, shouldn't the it be generated? Furthermore, what the supp video shows is different and you highlighted the improvement on trajectory.


We agree with the reviewer that the root trajectory is part of the choreography and should be generated. We have therefore re-evaluated all methods and ablations under full generation, where the entire choreography is generated from the music and the cast size alone, which is also the setting of the supplementary videos. ** X-Dancers remains the best method on five of the six metrics in this setting, and the ablations attribute the result to the joint progressive generation of root state and motion token and to formation-frame attention. ** The previous protocol is retained as a trajectory control experiment, in which the same model follows prescribed root paths most closely. The re-evaluated results are reported as the mean and standard deviation over five sampling seeds.

Table R1. Baseline comparison under full generation (Table 1 of the revised paper). Mean ± std over five seeds. Best in bold, second best underlined.

MethodSC-FID ↓BAS ↑Div ↑FSD (mm) ↓GC-FID ↓Coll. (%) ↓
GCD49.20±1.7249.20^{\pm 1.72}0.365‾±0.002‾\underline{0.365}^{\pm \underline{0.002}}9.27±0.079.27^{\pm 0.07}16.25±0.0016.25^{\pm \mathbf{0.00}}90.43±2.3690.43^{\pm 2.36}2.054±0.0782.054^{\pm 0.078}
CoDancers29.56±0.8529.56^{\pm 0.85}0.334±0.0010.334^{\pm \mathbf{0.001}}7.13±0.037.13^{\pm \mathbf{0.03}}14.48‾±0.00\underline{14.48}^{\pm \mathbf{0.00}}35.28±1.1435.28^{\pm 1.14}1.936±0.1251.936^{\pm 0.125}
TCDiff28.39±0.6428.39^{\pm 0.64}0.361±0.002‾0.361^{\pm \underline{0.002}}11.70±0.0911.70^{\pm 0.09}21.93±0.01‾21.93^{\pm \underline{0.01}}53.84±0.6853.84^{\pm 0.68}2.829±0.1912.829^{\pm 0.191}
FreeDance52.15±1.3852.15^{\pm 1.38}0.356±0.0010.356^{\pm \mathbf{0.001}}5.02±0.035.02^{\pm \mathbf{0.03}}31.46±0.0831.46^{\pm 0.08}102.57±2.71102.57^{\pm 2.71}1.415‾±0.236\underline{1.415}^{\pm 0.236}
TCDiff++21.53‾±0.42‾\underline{21.53}^{\pm \underline{0.42}}0.364±0.002‾0.364^{\pm \underline{0.002}}13.81±0.04‾\mathbf{13.81}^{\pm \underline{0.04}}15.47±0.0015.47^{\pm \mathbf{0.00}}25.68‾±0.53‾\underline{25.68}^{\pm \underline{0.53}}1.660±0.0131.660^{\pm \mathbf{0.013}}
X-Dancers (Ours)17.28±0.13\mathbf{17.28}^{\pm \mathbf{0.13}}0.368±0.001\mathbf{0.368}^{\pm \mathbf{0.001}}12.63‾±0.06\underline{12.63}^{\pm 0.06}9.31±0.00\mathbf{9.31}^{\pm \mathbf{0.00}}18.97±0.22\mathbf{18.97}^{\pm \mathbf{0.22}}0.974±0.027‾\mathbf{0.974}^{\pm \underline{0.027}}
Ground Truth–0.367\mathrm{0.367}9.48\mathrm{9.48}4.70\mathrm{4.70}–0.956\mathrm{0.956}

X-Dancers remains the state-of-the-art method under the full choreography synthesis setting. It achieves the best SC-FID, BAS, FSD, GC-FID and collision rate among all methods. TCDiff++ generates trajectories and motions in one stage and is the strongest baseline, yet it distinguishes dancers by identity encodings without conditioning their poses on the formation, which yields a higher diversity at a lower fidelity of the synthesized choreography. Our method instead makes motion synthesis aware of the formation by coupling a continuous motion token with an explicit root state in every cell, and by grounding inter-dancer attention in the evolving formation through the formation-frame descriptors computed from the root states. As the dance motions are correlated and refined together with the evolving formation, the model learns to generate casts with natural spacing and coherent formations while keeping the motion fidelity of each dancer, even when no formation or root trajectory is provided in advance. The group dance motions for baseline comparison presented in the supplementary videos are generated under the full choreography synthesis setting, which matches with the updated quantitative results.

AuthorsOpenReview ↗

Response to Reviewer a7fH: W1 [2/2]

The previous protocol is kept as a trajectory control experiment. The comparison under ground-truth root trajectories is now Table 3 of the revised paper. It is extended with the tracking errors ADE and FDE, the mean and the final planar distance between the generated and the requested root paths, and is attached above as Table R3. The SC-FID and GC-FID of all methods are lower under ground-truth trajectories in Table R3 than under full generation in Table R1. This is expected, as the supplied trajectories carry the global displacement of each dancer and the formation geometry of the real choreography, both of which are components of the evaluated features. Our method leads in both settings. Without any trajectory, it generates natural formations and root trajectories from scratch, with the best GC-FID of 18.97 against 25.68 for the second-best method and the lowest collision rate of 0.974%, close to the 0.956% of the ground truth. With predefined trajectories, its generations follow the conditions most closely, with the lowest ADE and FDE of 7.33 cm and 13.09 cm, while retaining the best motion and group fidelity. Both abilities come from one model without retraining, as the root state of every cell is either generated jointly with the motion token or supplied as a visible channel.

Table R3. Trajectory control under ground-truth root trajectories (Table 3 of the revised paper). Mean ± std over five seeds. Best in bold, second best underlined.

MethodSC-FID ↓BAS ↑Div ↑FSD (mm) ↓GC-FID ↓Coll. (%) ↓ADE (cm) ↓FDE (cm) ↓
GCD37.27±0.1037.27^{\pm 0.10}0.361‾±0.001‾\underline{0.361}^{\pm \underline{0.001}}10.52±0.2110.52^{\pm 0.21}21.38±0.03‾21.38^{\pm \underline{0.03}}34.63±0.0834.63^{\pm 0.08}3.261±0.1063.261^{\pm 0.106}13.57±0.1813.57^{\pm 0.18}19.30±0.1519.30^{\pm 0.15}
CoDancers28.93±0.08‾28.93^{\pm \underline{0.08}}0.307±0.001‾0.307^{\pm \underline{0.001}}9.26±0.039.26^{\pm \mathbf{0.03}}22.29±0.0122.29^{\pm \mathbf{0.01}}30.72±0.0230.72^{\pm \mathbf{0.02}}2.574±0.0122.574^{\pm 0.012}11.63±0.0811.63^{\pm 0.08}18.27±0.0618.27^{\pm 0.06}
TCDiff20.58±0.1820.58^{\pm 0.18}0.352±0.0000.352^{\pm \mathbf{0.000}}11.24±0.3411.24^{\pm 0.34}18.13±0.03‾18.13^{\pm \underline{0.03}}18.47±0.0718.47^{\pm 0.07}2.286±0.0932.286^{\pm 0.093}8.79‾±0.01‾\underline{8.79}^{\pm \underline{0.01}}13.55‾±0.01\underline{13.55}^{\pm \mathbf{0.01}}
FreeDance24.35±0.1024.35^{\pm 0.10}0.354±0.0020.354^{\pm 0.002}8.35±0.09‾8.35^{\pm \underline{0.09}}27.21±0.0927.21^{\pm 0.09}26.20±0.0926.20^{\pm 0.09}1.557‾±0.284\underline{1.557}^{\pm 0.284}15.83±0.2515.83^{\pm 0.25}19.42±0.1819.42^{\pm 0.18}
TCDiff++16.21‾±0.07\underline{16.21}^{\pm \mathbf{0.07}}0.358±0.0000.358^{\pm \mathbf{0.000}}12.61‾±0.18\underline{12.61}^{\pm 0.18}14.65‾±0.01\underline{14.65}^{\pm \mathbf{0.01}}12.63‾±0.05\underline{12.63}^{\pm 0.05}1.739±0.007‾1.739^{\pm \underline{0.007}}10.35±0.0010.35^{\pm \mathbf{0.00}}15.62±0.0115.62^{\pm \mathbf{0.01}}
X-Dancers (Ours)7.25±0.08‾\mathbf{7.25}^{\pm \underline{0.08}}0.365±0.000\mathbf{0.365}^{\pm \mathbf{0.000}}13.05±0.23\mathbf{13.05}^{\pm 0.23}10.58±0.01\mathbf{10.58}^{\pm \mathbf{0.01}}7.64±0.03‾\mathbf{7.64}^{\pm \underline{0.03}}1.008±0.006\mathbf{1.008}^{\pm \mathbf{0.006}}7.33±0.00\mathbf{7.33}^{\pm \mathbf{0.00}}13.09±0.02‾\mathbf{13.09}^{\pm \underline{0.02}}
Ground Truth–0.367\mathrm{0.367}9.48\mathrm{9.48}4.70\mathrm{4.70}–0.956\mathrm{0.956}––

We believe that the two settings together characterize the method more completely than the previous version, and we are grateful to the reviewer for pointing out this issue.


We thank the reviewer again for the time and effort. Please feel free to follow up with any further questions, which we would be more than happy to address.

Dear Reviewer a7fH,

Thank you very much for taking the time to revisit our work and for raising your score. We appreciate the constructive feedback, which has helped us improve the paper.

Best,

Paper 50089 Authors

Since the authors provide quantitative evidence with generated trajectories, I raise the score.

Reviewer_A1HWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes X-Dancers, a unified framework for music-driven 3D group choreography. It represents a group performance as a joint motion tensor that couples continuous per-dancer motion tokens with explicit root states for placement and trajectory.

The ChoreoGraph Transformer interleaves temporal attention with formation-frame attention. The formation frame is recomputed from currently visible root states so that inter-dancer attention is grounded in an evolving formation rather than a fixed dancer ordering.

Critical strengths

The unified joint formulation is elegant and well motivated. Coupling motion tokens and root states in one tensor, and treating composition, continuation, cast extension, and trajectory control as mask patterns over the same distribution, is a coherent conceptual advance over task-specific modules.

Formation-frame attention addresses a real problem: the dancer axis is a set whose meaningful geometry is induced by the formation. Recomputing descriptors from visible root states and using them in inter-dancer attention is a sensible design, and the attention visualization provides useful qualitative evidence that neighborhoods reorganize with the formation.

The empirical comparison is strong on several metrics. The method substantially improves single-dancer FID, group FID, and collision rate over the reported baselines, and the ablations support continuous representations, spatial-temporal-channel masking, and formation-relative geometry.

Critical weaknesses

The main quantitative evaluation appears to supply the same ground-truth root trajectories to all methods, and the ablation discussion refers to matching a root-conditioned evaluation. If so, Table 1 primarily tests motion generation given ground-truth placement, not the joint generation of motion and formation that is the paper’s central claim. The proposed group FID’s spatial features and the collision metric would then be largely determined by the supplied roots rather than by the model’s formation synthesis. This is the most serious weakness because the core contribution is joint modeling of motion and placement.

The unified choreographic operations are not quantitatively evaluated. Continuation, cast extension, and trajectory control are demonstrated mainly through a qualitative gallery and an overall user study. There are no metrics for trajectory tracking error, long-horizon drift, temporal consistency across windows, formation coherence after cast extension, or collision behavior during extension. For a paper whose main selling point is one model for five operations, this evidence gap is significant.

The evaluation uses only one dataset, AIOZ-GDANCE, and a cleaned subset. It is unclear whether all baselines were retrained on the same cleaned subset and under the same root-conditioned protocol. The phrase “full AIOZ-GDANCE benchmark” is also confusing after excluding clips in preprocessing. Baseline fairness and the effect of the cleaning pipeline need clarification.

The proposed group-level metric is not validated. Group-Choreo FID is central to the group-quality claim, but the paper does not report correlation with human judgments, sensitivity to descriptor design, or comparison with alternative group metrics. The user study is small: twenty participants, twelve sequences, three methods, and no reported significance tests or inter-rater reliability.

Physical plausibility remains weak. The reported foot sliding distance is more than double the ground-truth value, and collisions are nonzero. The paper emphasizes state-of-the-art results but does not discuss this gap or whether post-processing could reduce it.

Cast size is limited to seven dancers, and the paper states that zero-shot extension beyond the training maximum causes severe collisions. Megacrew choreography is out of reach. Since “group choreography” often involves larger casts, this is a material limitation rather than a minor one.

The root-channel loss weight is extremely small relative to the motion channels, despite the paper’s emphasis on explicit root states and joint placement generation. If roots are supplied at evaluation, this is less visible, but it raises doubts about how well the model generates roots in fully masked composition.

The formation descriptor is recomputed from visible root states, but at the start of a fully masked composition there are no visible roots. The paper does not analyze how the initial commitment order affects formation emergence, nor whether the model can recover from poor early root commitments.

Questions for authors

Are the main quantitative results in Table 1 and the ablations computed with ground-truth root trajectories supplied to every method? If yes, please provide free-generation results without ground-truth roots, including formation quality, trajectory plausibility, and collision rate under generated roots. If no, please clarify exactly which metrics use supplied roots.

Can you provide quantitative evaluations for continuation, cast extension, and trajectory control, such as trajectory tracking error, long-horizon drift, cross-window consistency, collision rate during extension, and formation coherence after adding dancers?

Were all baselines retrained on the cleaned GDANCE subset under the same protocol, or are some numbers taken from prior papers? How many clips were used for each baseline, and how do results change on the original uncleaned split?

Can you validate Group-Choreo FID against human judgments, report sensitivity to its descriptor components, and compare it with alternative group-level metrics? Please also report significance tests and inter-rater reliability for the user study.

Why is the root-channel loss weight so small, and how well does the model generate root trajectories when all root channels are initially masked?

How sensitive are composition and cast extension to the random commitment order and to early root-state errors? Can poor early formation commitments be corrected later?

Does the slot canonicalization by first-step polar angle preserve the claimed permutation equivariance in practice? How does cast extension behave when new dancers should occupy non-trailing slots?

Can you discuss the foot-sliding gap relative to ground truth and whether the method can be combined with foot-contact post-processing without harming motion quality?

16 comments
AuthorsOpenReview ↗

Response to Reviewer A1HW

We sincerely thank the reviewer for the careful reading and for the positive assessment of our unified formulation of group choreography, the formation-frame attention and the reported empirical gains. We address every weakness and question below, and all new results are included in the revised manuscript. The main changes of the revision are summarized as follows.

  • Evaluation under fully automated generation. The main comparison and the ablation study are re-evaluated without ground-truth root trajectories (W1, Q1).
  • Quantitative evaluation of the choreographic operations. Trajectory control, continuation and cast extension are evaluated with tracking errors, long-horizon drift, consistency across windows, formation coherence and collision rates (W2, Q2).
  • Dataset and baseline protocol. The refinement of AIOZ-GDANCE and the retraining of all baselines on the refined data are documented (W3, Q3).
  • Validation of GC-FID and of the user study. GC-FID is validated with controlled perturbations and against GMR, and the user study is reported with significance tests (W4, Q4).
  • Physical plausibility. The foot-sliding gap is stated as a limitation, and a foot-contact refinement is evaluated as post-processing (W5, Q8).
  • Casts beyond seven dancers. Prescribed formations of 10 to 16 dancers are generated without retraining (W6).
  • Root-channel loss weight. The value reported in the appendix is corrected, and root generation with all root channels masked is evaluated (W7, Q5).
  • Commitment order. The sensitivity to the commitment order and to early root errors is analyzed, and the commitment order of the sampler and its description in the appendix are updated (W8, Q6).
  • Slot assignment in cast extension. The agreement with the slot convention and the extension into non-trailing slots are evaluated (Q7).
AuthorsOpenReview ↗

Response to Reviewer A1HW: W1 & Q1 [1/2]

W1 & Q1 [1/2]: Quantitative Evaluation under Full Generation Setting

W1&Q1: Are the main quantitative results in Table 1 and the ablations computed with ground-truth root trajectories supplied to every method? If yes, please provide free-generation results without ground-truth roots, including formation quality, trajectory plausibility, and collision rate under generated roots. If no, please clarify exactly which metrics use supplied roots.


We agree with the reviewer that this is the most important issue. In the submission, both Table 1 and the ablations were indeed computed with ground-truth root trajectories supplied to every method, and all six metrics were therefore conditioned on the supplied roots. We have therefore re-evaluated all methods and ablations under fully automated generation, where every method receives only the music and the cast size and generates the root trajectories together with the motions. X-Dancers remains the best method on five of the six metrics in this setting, and the ablations attribute the result to the joint progressive generation of root state and motion token and to formation-frame attention. The previous protocol is retained as a trajectory control experiment, in which the same model follows prescribed root paths most closely.

Table R1. Baseline comparison under fully automated generation (Table 1 of the revised paper). Mean ± std over five seeds. Best in bold, second best underlined.

MethodSC-FID ↓BAS ↑Div ↑FSD (mm) ↓GC-FID ↓Coll. (%) ↓
GCD49.20±1.7249.20^{\pm 1.72}0.365‾±0.002‾\underline{0.365}^{\pm \underline{0.002}}9.27±0.079.27^{\pm 0.07}16.25±0.0016.25^{\pm \mathbf{0.00}}90.43±2.3690.43^{\pm 2.36}2.054±0.0782.054^{\pm 0.078}
CoDancers29.56±0.8529.56^{\pm 0.85}0.334±0.0010.334^{\pm \mathbf{0.001}}7.13±0.037.13^{\pm \mathbf{0.03}}14.48‾±0.00\underline{14.48}^{\pm \mathbf{0.00}}35.28±1.1435.28^{\pm 1.14}1.936±0.1251.936^{\pm 0.125}
TCDiff28.39±0.6428.39^{\pm 0.64}0.361±0.002‾0.361^{\pm \underline{0.002}}11.70±0.0911.70^{\pm 0.09}21.93±0.01‾21.93^{\pm \underline{0.01}}53.84±0.6853.84^{\pm 0.68}2.829±0.1912.829^{\pm 0.191}
FreeDance52.15±1.3852.15^{\pm 1.38}0.356±0.0010.356^{\pm \mathbf{0.001}}5.02±0.035.02^{\pm \mathbf{0.03}}31.46±0.0831.46^{\pm 0.08}102.57±2.71102.57^{\pm 2.71}1.415‾±0.236\underline{1.415}^{\pm 0.236}
TCDiff++21.53‾±0.42‾\underline{21.53}^{\pm \underline{0.42}}0.364±0.002‾0.364^{\pm \underline{0.002}}13.81±0.04‾\mathbf{13.81}^{\pm \underline{0.04}}15.47±0.0015.47^{\pm \mathbf{0.00}}25.68‾±0.53‾\underline{25.68}^{\pm \underline{0.53}}1.660±0.0131.660^{\pm \mathbf{0.013}}
X-Dancers (Ours)17.28±0.13\mathbf{17.28}^{\pm \mathbf{0.13}}0.368±0.001\mathbf{0.368}^{\pm \mathbf{0.001}}12.63‾±0.06\underline{12.63}^{\pm 0.06}9.31±0.00\mathbf{9.31}^{\pm \mathbf{0.00}}18.97±0.22\mathbf{18.97}^{\pm \mathbf{0.22}}0.974±0.027‾\mathbf{0.974}^{\pm \underline{0.027}}
Ground Truth–0.367\mathrm{0.367}9.48\mathrm{9.48}4.70\mathrm{4.70}–0.956\mathrm{0.956}

X-Dancers remains the state-of-the-art method when placement and motion are generated jointly. It achieves the best SC-FID, BAS, FSD, GC-FID and collision rate among all methods. As no root trajectory is supplied, the formation features of GC-FID and the collision rate now measure the formation that each model generates, which removes the dependence on supplied roots that the reviewer points out. X-Dancers attains the best GC-FID, and its collision rate of 0.974% is the closest to the 0.956% of the ground truth. The plausibility of the generated trajectories enters GC-FID through its formation features, which include the speed of the group centroid and the temporal change of the formation geometry. TCDiff++ is the strongest baseline and generates trajectories and motions in one stage, yet it distinguishes dancers by identity encodings without conditioning their poses on the formation, which yields a higher diversity at a lower fidelity of the generated choreography. GCD and FreeDance encode the placement of each dancer inside its motion embedding or quantized motion token without modeling it explicitly, and they therefore tend to neglect the patterns of reasonable formations, which results in the two highest SC-FID and GC-FID values once the trajectories have to be generated. TCDiff generates the trajectories in a first stage without any cue from the dance motions, and its motion stage has to follow them even when they are noisy and unnatural, which is consistent with its highest collision rate and inferior group fidelity.

Our method instead makes motion synthesis aware of the formation by coupling a continuous motion token with an explicit root state in every cell, and by grounding inter-dancer attention in the evolving formation through the formation-frame descriptors computed from the root states. As the dance motions are correlated and refined together with the evolving formation, the model learns to generate casts with natural spacing and coherent formations while keeping the motion fidelity of each dancer, although no formation or root trajectory is provided in advance.

AuthorsOpenReview ↗

Response to Reviewer A1HW: W1 & Q1 [2/2]

W1 & Q1 [2/2]: Quantitative Evaluation under Full Generation Setting

The ablations under fully automated generation confirm the design choices. Following the reviewer's suggestion, all ablation variants are now re-evaluated under the same setting in Table R2 above. Removing the formation descriptors while keeping the explicit root state in every cell (C2) raises GC-FID from 18.97 to 27.91 and the collision rate from 0.974% to 1.105%. Replacing formation-frame attention with graph attention (C1) raises them further to 32.42 and 1.427%. The root states alone therefore do not make the attention across dancers aware of the formation, and it is the formation frame built from them that does. For the joint generation, replacing bidirectional masked completion with joint diffusion over the same tensor (A2) raises GC-FID to 69.18, which supports the progressive completion of placement and motion under the visible group context.

The previous protocol is kept as a trajectory control experiment. The comparison under ground-truth root trajectories is now Table 3 of the revised paper, where every metric is labeled as conditioned on supplied roots, and it is given with the tracking errors ADE and FDE as Table R3 in our response to W2. The SC-FID and GC-FID of all methods are lower there than under fully automated generation in Table R1. This confirms the observation of the reviewer, as the supplied trajectories carry the global displacement of each dancer and the formation geometry of the real choreography, both of which are components of the evaluated features. Our method leads in both settings. Without any trajectory, it generates natural formations and root trajectories from scratch, with the best GC-FID of 18.97 against 25.68 for the second-best method and the lowest collision rate. With predefined trajectories, its generations follow the conditions most closely, with the lowest ADE and FDE of 7.33 cm and 13.09 cm, while retaining the best motion and group fidelity. Both abilities come from one model without retraining, as the root state of every cell is either generated jointly with the motion token or supplied as a visible channel.

Table R2. Ablation study under fully automated generation (Table 2 of the revised paper). Mean ± std over five seeds. Best in bold, second best underlined.

MethodSC-FID ↓BAS ↑Div ↑FSD (mm) ↓GC-FID ↓Coll. (%) ↓
A1: VQVAE-based masked transformer54.29±2.0154.29^{\pm 2.01}0.356±0.0000.356^{\pm \mathbf{0.000}}4.52±0.384.52^{\pm 0.38}15.81±0.2515.81^{\pm 0.25}105.93±3.40105.93^{\pm 3.40}1.327±0.1281.327^{\pm 0.128}
A2: Diffusion transformer37.50±0.9037.50^{\pm 0.90}0.364±0.001‾0.364^{\pm \underline{0.001}}8.60±0.08‾8.60^{\pm \underline{0.08}}13.94±0.1613.94^{\pm 0.16}69.18±1.4669.18^{\pm 1.46}1.982±0.0721.982^{\pm 0.072}
B1: Spatial-only masking29.51±0.8529.51^{\pm 0.85}0.362±0.001‾0.362^{\pm \underline{0.001}}6.63±0.756.63^{\pm 0.75}11.06±0.0911.06^{\pm 0.09}32.59±0.8132.59^{\pm 0.81}1.209±0.0311.209^{\pm 0.031}
B2: Temporal-only masking26.93±0.1826.93^{\pm 0.18}0.365±0.0000.365^{\pm \mathbf{0.000}}9.84±0.269.84^{\pm 0.26}10.13±0.02‾10.13^{\pm \underline{0.02}}47.28±1.3547.28^{\pm 1.35}1.626±0.0071.626^{\pm \mathbf{0.007}}
B3: Spatial + channel masking27.64±1.1927.64^{\pm 1.19}0.363±0.001‾0.363^{\pm \underline{0.001}}6.04±0.516.04^{\pm 0.51}9.42‾±0.13\underline{9.42}^{\pm 0.13}31.65±0.7331.65^{\pm 0.73}1.185±0.0241.185^{\pm 0.024}
B4: Temporal + channel masking24.80±0.15‾24.80^{\pm \underline{0.15}}0.365±0.001‾0.365^{\pm \underline{0.001}}8.72±0.198.72^{\pm 0.19}9.86±0.039.86^{\pm 0.03}36.20±0.6136.20^{\pm 0.61}1.315±0.010‾1.315^{\pm \underline{0.010}}
C1: Graph attention22.73±0.5422.73^{\pm 0.54}0.358±0.0000.358^{\pm \mathbf{0.000}}6.95±0.126.95^{\pm 0.12}10.57±0.1810.57^{\pm 0.18}32.42±0.48‾32.42^{\pm \underline{0.48}}1.427±0.0321.427^{\pm 0.032}
C2: w/o formation descriptors19.34‾±0.41\underline{19.34}^{\pm 0.41}0.374±0.000\mathbf{0.374}^{\pm \mathbf{0.000}}11.61‾±0.25\underline{11.61}^{\pm 0.25}10.29±0.0410.29^{\pm 0.04}27.91‾±0.57\underline{27.91}^{\pm 0.57}1.105‾±0.016\underline{1.105}^{\pm 0.016}
Ours (full)17.28±0.13\mathbf{17.28}^{\pm \mathbf{0.13}}0.368‾±0.001‾\underline{0.368}^{\pm \underline{0.001}}12.63±0.06\mathbf{12.63}^{\pm \mathbf{0.06}}9.31±0.00\mathbf{9.31}^{\pm \mathbf{0.00}}18.97±0.22\mathbf{18.97}^{\pm \mathbf{0.22}}0.974±0.027\mathbf{0.974}^{\pm 0.027}
Reviewer_2jPvOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents X-Dancers, a unified framework for music-driven 3D group choreography generation. It represents a group performance as a motion tensor in which each dancer-time cell contains a continuous motion token and an explicit root state. A ChoreoGraph Transformer combines temporal attention with formation-frame attention, which uses formation-relative descriptors to model inter-dancer relationships as the group formation evolves. The proposed STC-Mask allows a single model to support composition, long-duration continuation, cast extension, and trajectory control through different conditioning masks without task-specific retraining.

Critical strengths
  1. A unified formulation for joint group choreography modeling. Representing individual motion and root states jointly, and formulating generation as masked completion, provides a coherent way to support composition, continuation, cast extension, and trajectory control within a single model. This is a meaningful advantage over approaches that require separate task-specific mechanisms.
  2. A meaningful formation-frame attention with supporting experiments. Formation-frame attention derives spatial descriptors from visible root states and updates them as the formation evolves. The ablation results show that removing these descriptors degrades both individual motion fidelity and group-level distributional quality.
  3. A flexible STC-Mask training strategy. The proposed spatial-temporal-channel masking strategy exposes the model to different patterns of partially observed choreography during training. This enables a single trained model to support composition, continuation, cast extension, and trajectory control through different conditioning masks, without task-specific retraining.
Critical weaknesses
  1. The evidence for long-duration continuation is incomplete. The quantitative evaluation primarily focuses on 8-second clips, while longer continuations are demonstrated mainly through qualitative examples. The paper does not systematically assess how motion quality, formation stability, and collision rates vary with sequence length, or quantify temporal continuity at window boundaries.
  2. Missing inference efficiency analysis. Although training time is reported, inference time and peak GPU memory are not provided. These omissions make it difficult to assess the practical computational cost and scalability of the proposed continuation strategy.
  3. Limited quantitative validation of cast extension. Although the method supports adding dancers during an ongoing performance, cast extension is demonstrated mainly through qualitative examples. Quantitative evaluation of collision rates, formation consistency, and coordination between newly added dancers and the existing cast would provide stronger evidence for this capability.
  4. Minor typographical and punctuation errors. For example, line 276 is missing a period after “one” before the sentence beginning “A one-step estimate,” and “synthsis” in line 287 should be corrected to “synthesis.”.
Questions for authors
  1. Long-duration generation. Could the authors provide quantitative results for multiple sequence lengths, such as 8, 16, 32, and 60 seconds, including motion quality, formation stability, collision rates, and continuity across overlapping windows?
  2. Inference efficiency. Could the authors report inference time and peak GPU memory for an 8-second window and longer sequences, along with the hardware and sampling settings?
  3. Cast extension. Could the authors provide quantitative evaluations of cast extension, including collision rates, formation consistency, and changes in existing dancers' motions before and after adding new dancers?
5 comments
AuthorsOpenReview ↗

Response to Reviewer 2jPv

We sincerely thank the reviewer for recognizing the joint choreography formulation, the formation-frame attention and the flexible STC-Mask training. Following the suggestions, we have added quantitative evaluations of long-duration continuation and cast extension and an inference efficiency analysis, which are included in the revised manuscript and attached below. As Q1 to Q3 request the results that W1 to W3 call for, we answer each pair together.


W1 & Q1: Long-duration continuation

W1: The evidence for long-duration continuation is incomplete. The quantitative evaluation primarily focuses on 8-second clips, while longer continuations are demonstrated mainly through qualitative examples. The paper does not systematically assess how motion quality, formation stability, and collision rates vary with sequence length, or quantify temporal continuity at window boundaries.

Q1: Long-duration generation. Could the authors provide quantitative results for multiple sequence lengths, such as 8, 16, 32, and 60 seconds, including motion quality, formation stability, collision rates, and continuity across overlapping windows?


Thank you for the suggestion on long-duration generation. Following it, we have evaluated continuation at 8, 16, 32 and 44 s on all test tracks longer than 44 s, with measures of motion quality, formation stability, collision rate and continuity at the window boundaries. Formation stability and continuity at the window boundaries are maintained as the sequence grows, and SC-FID and GC-FID remain in a stable range. Every window is completed by the same model as the temporal block pattern of STC-Mask, from a prefix that carries the motion tokens and the root states of all dancers, which continues the formation instead of re-planning it. * he longest evaluated length is 44 s and not the suggested 60 s, as only 10 of the 150 test tracks reach 60 s, which is too few for distribution-level metrics. Sequences of 60 s are included in the efficiency analysis of our response to W2 and Q2.

Table R1. Continuation at increasing length. Spread ratio is the group spread of the final 8 s over that of the first 8 s, and PJ the peak jerk (m/s³). → marks values that are better the closer they are to the reference (GT or in-window).

LengthSC-FID ↓GC-FID ↓Coll. (%) ↓ (ours / GT)Spread ratio → (ours / GT)PJ → (seam / in-window)
8 s19.37\mathrm{19.37}21.36\mathrm{21.36}0.738 / 0.9840.738 \,/\, 0.984––
16 s23.82\mathrm{23.82}23.74\mathrm{23.74}0.652 / 0.9270.652 \,/\, 0.9271.07 / 1.001.07 \,/\, 1.001204 / 13681204 \,/\, 1368
32 s27.51\mathrm{27.51}26.53\mathrm{26.53}0.579 / 0.8120.579 \,/\, 0.8121.13 / 1.001.13 \,/\, 1.001295 / 12411295 \,/\, 1241
44 s24.60\mathrm{24.60}22.08\mathrm{22.08}0.416 / 0.7230.416 \,/\, 0.7231.10 / 0.991.10 \,/\, 0.991327 / 12821327 \,/\, 1282

As shown in Table R1, formation stability and temporal continuity do not deteriorate with the length. For formation stability, the spread ratio stays between 1.07 and 1.13, against 1.00 for the ground truth, and no longer increases beyond 32 s. The collision rate does not rise with the length either, and it stays below that of the ground truth at every length, which we do not count as a gain by itself. For continuity at the window boundaries, the peak jerk at the seams remains at the level of that inside the windows, at most 5% above it, as the cells around each seam are completed again under temporal context on both sides, which is the in-betweening pattern of STC-Mask. For motion and group quality, SC-FID and GC-FID remain in a stable range, and their variation across the rows stems from the reference set, as each length is compared with its own ground-truth clips, whose number falls from 587 at 8 s to 146 at 44 s. It does not indicate an accumulated error, which would grow with the number of chained windows, whereas the 44 s sequences obtain lower SC-FID and GC-FID than the 32 s ones. Overall, the formation, the continuity across window boundaries and the motion quality remain stable over 44 s of continuation, more than five times the length of a training window, which supports the long-duration capability of X-Dancers quantitatively.

AuthorsOpenReview ↗

Response to Reviewer 2jPv: W2 & Q2

W2 & Q2: Inference efficiency

W2: Missing inference efficiency analysis. Although training time is reported, inference time and peak GPU memory are not provided. These omissions make it difficult to assess the practical computational cost and scalability of the proposed continuation strategy.

Q2: Inference efficiency. Could the authors report inference time and peak GPU memory for an 8-second window and longer sequences, along with the hardware and sampling settings?

Thank you for pointing out the missing inference analysis. Following your suggestion, we have added an inference efficiency analysis to Appendix A of the revised paper, together with the hardware and sampling settings. The cost of X-Dancers follows from its window-based design, in which all dancers of a window share one forward pass and a longer sequence is generated one 8 s window at a time. Generation therefore scales linearly in time at constant memory, stays faster than real time at every tested length, and is 4.3 to 7.4 times faster than prior methods on a 24 s sequence. The results are listed below with their table numbers.

  • Inference cost by sequence length. Table R2 below, with FLOPs, time, RTF and peak memory at 8, 16, 24 and 60 s for 2, 5 and 7 dancers.
  • Efficiency comparison with prior methods. Table R3 below, with FLOPs, parameters and time for a sequence of 24 s.

All measurements use one RTX 4090 at batch size 1 with bfloat16 mixed precision and cover the complete pipeline from audio features to SMPL motion. The sampler uses K = 18 iterations, a classifier-free guidance scale of 3, a 25-step Euler solver for the diffusion head, and 8 s windows with a 4 s stride. As shown in Table R2, an 8 s window is generated in about 1.9 s for two to seven dancers, as all dancers share one forward pass. Every further window adds about 3.6 s for seven dancers, as the cells around its seam are completed a second time, and time and FLOPs therefore grow linearly with the number of windows. For the same reason, the real-time factor (RTF), the generation time divided by the duration of the sequence, rises from 0.23 at 8 s to 0.81 at 60 s. It converges to about 0.9, the 3.6 s of a further window over the 4 s of motion that the window adds, and generation therefore remains faster than real time at any length. The peak memory stays between 0.92 and 0.94 GB, as the windows are processed one at a time.

Table R2. Inference cost of the complete pipeline for N = 2 / 5 / 7 dancers. FLOPs are counted as multiply-adds and given in units of 10¹² (T).

Length# WindowsFLOPs (T)Time (s)RTF (N = 7)Peak memory (GB)
8 s1\mathrm{1}3.43.4 / 4.64.6 / 5.45.41.91.9 / 1.91.9 / 1.91.90.23\mathrm{0.23}0.920.92 / 0.920.92 / 0.930.93
16 s3\mathrm{3}14.614.6 / 17.417.4 / 19.319.38.38.3 / 9.29.2 / 9.29.20.58\mathrm{0.58}0.920.92 / 0.920.92 / 0.930.93
24 s5\mathrm{5}25.925.9 / 30.330.3 / 33.233.214.714.7 / 16.516.5 / 16.516.50.69\mathrm{0.69}0.920.92 / 0.930.93 / 0.930.93
60 s14\mathrm{14}76.676.6 / 88.088.0 / 95.695.643.643.6 / 49.649.6 / 48.848.80.81\mathrm{0.81}0.930.93 / 0.940.94 / 0.940.94

For the comparison with prior methods, every method generates a sequence of 24 s, which corresponds to 720 frames, on the same GPU at batch size 1. As shown in Table R3, X-Dancers is 4.3 to 7.4 times faster than the baselines in the wall-clock time of the complete sampling, which is the end-to-end measure, as the samplers differ in their number of network evaluations. Its forward pass is also the cheapest, as its transformer always processes one 8 s window whereas the networks of the baselines process the complete sequence. It is worth noting that X-Dancers has the largest number of parameters among the compared models, 189.5 M in total, of which 117.5 M belong to the ChoreoGraph Transformer and 68.5 M to the diffusion head. However, this does not translate into a higher inference cost. Instead, our inference time efficiency outperforms the baseline methods by a large margin.

Table R3. Efficiency comparison on a 24 s sequence (720 frames). FLOPs of one forward pass and time of the complete sampling, averaged over casts of 2, 5 and 7 dancers. Best in bold, second best underlined.

MethodFLOPs (G) ↓Params (M) ↓Time (s) ↓
GCD90.37‾\underline{90.37}68.20‾\underline{68.20}91.6\mathrm{91.6}
CoDancers138.24\mathrm{138.24}60.65\mathbf{60.65}68.2‾\underline{68.2}
TCDiff++152.60\mathrm{152.60}79.71\mathrm{79.71}118.3\mathrm{118.3}
X-Dancers (Ours)35.49\mathbf{35.49}189.49\mathrm{189.49}15.9\mathbf{15.9}
AuthorsOpenReview ↗

Response to Reviewer 2jPv: the rest parts

W3 & Q3: Quantitative evaluation of cast extension

W3: Limited quantitative validation of cast extension. Although the method supports adding dancers during an ongoing performance, cast extension is demonstrated mainly through qualitative examples. Quantitative evaluation of collision rates, formation consistency, and coordination between newly added dancers and the existing cast would provide stronger evidence for this capability.

Q3: Cast extension. Could the authors provide quantitative evaluations of cast extension, including collision rates, formation consistency, and changes in existing dancers' motions before and after adding new dancers?


We appreciate the suggestion to quantify cast extension. To this end, we have evaluated the extended casts with the collision rate, the formation consistency and the change of the existing dancers that the review asks for. The existing dancers are preserved exactly, and the extended casts stay close to real casts of the same size. Cast extension is the spatial block pattern of STC-Mask, in which the existing dancers stay visible and formation-frame attention expresses every new cell in the frame of their root states, and the new dancers are therefore placed and moved relative to the existing formation.

We evaluate the extensions 2 → 3, 2 → 5 and 3 → 5 under fully automated generation. On all test clips whose ground truth has N′ dancers, N dancers are generated from the music and kept as visible rows while the appended rows are generated, and the extended cast is compared with the ground truth of the same clips.

Table R4. Cast extension from N to N′ dancers. All metrics are computed on the extended cast against real clips of N′ dancers.

N → N′SC-FID ↓BAS ↑Div ↑FSD (mm) ↓GC-FID ↓Coll. (%) ↓
2 → 320.25\mathrm{20.25}0.362\mathrm{0.362}9.76\mathrm{9.76}9.46\mathrm{9.46}18.82\mathrm{18.82}0.473\mathrm{0.473}
2 → 516.97\mathrm{16.97}0.365\mathrm{0.365}7.54\mathrm{7.54}8.81\mathrm{8.81}20.35\mathrm{20.35}0.568\mathrm{0.568}
3 → 518.21\mathrm{18.21}0.365\mathrm{0.365}10.83\mathrm{10.83}8.12\mathrm{8.12}19.46\mathrm{19.46}0.613\mathrm{0.613}
Ground Truth (N′ = 3)–0.361\mathrm{0.361}12.57\mathrm{12.57}––0.579\mathrm{0.579}
Ground Truth (N′ = 5)–0.359\mathrm{0.359}8.62\mathrm{8.62}––1.015\mathrm{1.015}

As shown in Table R4, the extended casts stay close to real casts of the same size. For the existing dancers, supplied cells are never resampled and every dancer is decoded independently, which reproduces their motions exactly, up to numerical precision. For formation consistency and coordination, which GC-FID measures through the distances and the relative velocities between the dancers, the GC-FID of 18.82 to 20.35 is at the level of the 18.97 that composition from scratch attains in Table 1 of the revised paper, and SC-FID is likewise close to its 17.28. Rows with different N′ are compared with different ground-truth clips, and the two extensions to five dancers, which share their reference, reach similar SC-FID and GC-FID, which shows that the fidelity of the extended cast depends little on how many dancers are given.

It is also worth noting that the collision rates of the extended casts lie below those of real casts of the same size in the ground truth samples. The ground truth is reconstructed from monocular in-the-wild videos and retains residual collisions between the dancers, whereas X-Dancers is trained with a collision term that penalizes dancers closer than a margin. Thus, the collision rate improvement is not a gain by itself and has to be evaluated together with GC-FID. As GC-FID stays at the level of composition from scratch, the lower rates are not obtained at the cost of the formation.


W4: Typographical errors

W4: Minor typographical and punctuation errors. For example, line 276 is missing a period after "one" before the sentence beginning "A one-step estimate," and "synthsis" in line 287 should be corrected to "synthesis.".

Thank you for the careful reading. We have added the missing period after "one" in line 276, corrected "synthsis" to "synthesis" in line 287, and proofread the entire manuscript again.


We thank the reviewer again for the constructive suggestions. We would be glad to provide any further clarification.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We sincerely thank all reviewers and the area chair for their time and constructive feedback, and for pointing out both the strengths of our work and the aspects to improve. We are especially encouraged by the positive comments on the unified joint formulation, with which one model performs four choreographic operations through conditioning masks alone (Reviewers j9T1, a7fH, A1HW, 2jPv), the formation-frame attention (Reviewers j9T1, a7fH, A1HW, 2jPv), the ablation that supports it (Reviewers j9T1, a7fH, A1HW), the flexible STC-Mask training (Reviewer 2jPv), and the quality of the supplementary results (Reviewer a7fH).

We would like to re-emphasize the novelty and contributions of X-Dancers.

  1. Joint choreography modeling. X-Dancers models the motions and the global placements of all dancers as one joint distribution over a group motion tensor, in which every cell couples a continuous motion token with an explicit root state. Music-conditioned bidirectional masked generation thereby produces motion, formation and trajectory together.
  2. Formation-grounded spatial reasoning. Formation-frame attention in the ChoreoGraph Transformer builds a formation frame from the root states visible at each stage of generation, and the spatial structure of the cast is thereby generated and refined together with the choreography.
  3. Unified choreographic operations. STC-Mask trains the model under randomized masks over dancers, time steps and channels, which serve as conditioning interfaces at inference. One trained model thereby performs composition, continuation, cast extension and trajectory control without task-specific retraining.
  4. State-of-the-art synthesis quality. With the root trajectories generated together with the motions, X-Dancers achieves the best motion fidelity, music synchronization and formation quality among all methods, and it generates faster than real time.

We have responded to every reviewer individually and revised the paper and the supplementary material accordingly, with the main changes marked in blue. The key updates are summarized below.

Additional Experimental Results

  • Baseline comparison under fully automated generation, without ground-truth root trajectories (A1HW, a7fH, j9T1; Table 1 in paper).
  • Ablation study under fully automated generation, with new variants B3 and B4 (A1HW, a7fH, j9T1; Table 2 in paper).
  • Trajectory control with ADE and FDE (A1HW, a7fH, j9T1; Table 3 in paper).
  • Continuation up to 44 s (2jPv, A1HW; Table 5 in paper).
  • Cast extension (2jPv, A1HW; Table 6 in paper).
  • Casts beyond seven dancers under prescribed formations (A1HW; Table 7 in paper).
  • Inference efficiency (2jPv; Tables 8 and 9 in paper).
  • Validation of GC-FID (A1HW; Table 10 in paper).
  • Statistics of the user study (A1HW; Table 4 in paper).
  • Commitment order and early root errors (A1HW; Tables 11 and 12 in paper).
  • Foot-contact refinement (A1HW; Table 13 in paper).
  • Slots of new dancers in cast extension (A1HW; Table 14 in paper).

Paper Updates

Added content

  • Added Table 3 and the quantitative findings on trajectory control, cast extension and continuation to Section 4.1, and casts of 10, 14 and 16 dancers to Figure 3.
  • Added Appendix A.2 to A.7 with the additional experiments, the statistical analysis of the user study to Appendix A.1 and the slot analysis to Appendix G.3.
  • Added the masking variants B3 and B4 to Table 2, Section 4.2 and Appendix B.
  • Added a discussion of recent single-dancer methods to Section 2.2 and Appendix J.
  • Added the effect of the dataset refinement (Figure 8) and the protocol of every baseline to Appendix H.
  • Added the definitions of the operation metrics to Appendix I.

Revised content

  • Revised the evaluation protocol of Tables 1 and 2 and their analysis in Sections 4.1 and 4.2.
  • Revised the limitations in Section 4.3.
  • Corrected the root-channel loss weight in Appendix D.
  • Updated the description of the sampler in Section 3.4 and Appendices D and G.
  • Revised the description of GC-FID in Appendix I and the user-study figure (Figure 5).
  • Clarified the permutation equivariance in Appendix E.1.
  • Replaced "full AIOZ-GDANCE benchmark" with "refined AIOZ-GDANCE" and corrected the reported typos with a full proofread.

Supplementary Video Updates

  • Added 03_large_cast_generalization.mp4, which shows zero-shot generalization to casts beyond seven dancers under prescribed formations (Appendix A.3).

We hope that these updates clarify and strengthen the contributions of our work, and we would be happy to address any further questions or concerns. If our responses and the revision have properly addressed your concerns, we would sincerely appreciate it if you could kindly re-evaluate our work. Thank you again for your valuable feedback and efforts.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.