- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents DisCodec, a motion codec that separates style and content into different latent representations. It uses a variational information bottleneck, a learned style basis, and SNR-adaptive transform quantization to reduce information leakage between the two representations. The authors evaluate the method on motion style transfer, text-to-motion generation, and co-speech gesture generation. The results are promising, particularly for transferring styles without retraining for each new style. However, the results on unseen styles are mixed, and it is not yet clear how well the learned representations generalize beyond the tested datasets.
Critical strengths
-
The method separates style and content in the motion representation itself, rather than relying on a separate style adapter for each new style. This makes it potentially useful for applications where new reference motions become available at inference time.
-
The authors evaluate the codec on three different tasks and include ablations of the main components. This provides some evidence that the representation can be reused across different generation settings.
Critical weaknesses
-
The results on unseen styles are not consistently strong. In the 100STYLE experiment, DisCodec has lower unseen-style recognition accuracy than some competing methods. This raises questions about how well the learned style space actually generalizes to styles that were not present during training.
-
The evidence for disentanglement is still limited. The paper mainly relies on linear probes, reconstruction metrics, and downstream generation results. These are useful measurements, but they do not fully establish that style and content are independently represented. It would help to show more directly whether changing the style latent preserves the action and whether changing the content latent leaves the style intact.
Questions for authors
-
In Table 1, why is the unseen-style recognition accuracy of DisCodec trained on 100STYLE considerably lower than that of GenMoStyle and MotionPuzzle? How should this result be reconciled with the claim of effective zero-shot style transfer?
-
How sensitive are the results to the choice of the number of style basis vectors and the quantization parameter? Do the main conclusions hold under different settings?
-
Can the authors provide a more direct evaluation of disentanglement, showing that style changes preserve action content and content changes preserve style, particularly for unseen styles?
5 comments
Rebuttal WANA (part 2)
W1/Q1: On unseen SRA being lower than baselines.
Unseen SRA as a metric is skewed towards generations that are closest to the input style.
And high performance by GenMoStyle and MotionPuzzle on this metric is understandable as these methods copy the style (MotionPuzzle more than GenMoStyle) and do not keep the content-related parts of motion.
This is clearly evident by high performance on UnseenSRA but high FID and low CRA (which is the accuracy of the content classifier on unseen content motions).
As described in lines 403-412, MotionPuzzle completely copies the style reference motion, and GenMoStyle also follows a similar pattern. This is also evident in video example (Suppl. Video 12:09-12:26).
For GenMoStyle specifically, this follows the trend of uni-directional disentanglement where style information is taken out of content latents in a proper way (low style ← ) but the method fails to disentangle content from style (content ← ) and find the right balance between SRA and CRA/FID (see Table 1 in paper).
Note that trajectory for the motion-to-motion comparison table comes from GT to be consistent with baselines, as noted in Ln. 412. The difference between our method and baselines (especially GenMoStyle and MotionPuzzle) is more apparent when we show the trajectory generated by each method. (Suppl. Video 12:26-12:42).
We can see in the first example (Suppl. video 12:26-12:36) of unseen Armaching style, MotionPuzzle completely follows the style and GenMoStyle walks in its own place (being close to the style trajectory). DisCodec produces motion that follows the trajectory but walks in a slow manner with one arm upheld to show the semantics of the style. A similar pattern can be seen for the next example (Suppl. Video 12:36-12:42).
Due to low content adherence, GenMoStyle/MotionPuzzle try to copy style motion’s trajectory and lower body instead of the content motion (evident from 1st video example 12:26-12:36 in Suppl. Video). As content usually resides in lower body motion, we can further demonstrate the difference by comparing the correlation of foot rhythm to content motion for all methods below, across 100style unseen test set.
Correlation can be calculated between output's and content example’s foot-contact signals (share of feet planted per frame with near-zero velocity).
| 100STYLE, unseen pairs | MotionPuzzle | GenMoStyle | Ours |
|---|---|---|---|
| Correlation to Content Foot rhythm | 0.06 | 0.33 | 0.54 |
Note that all the styles and identities used in visual results are unseen by our method.
Since metrics like UnseenSRA struggle to provide a holistic answer, we therefore conduct a perceptual evaluation against GenMoStyle and we find that evaluators prefer stylized motions from our method 71.4% more than those from GenMoStyle in terms of style matching (Fig. 5).
Additionally, this pattern is similar to text-to-motion baselines where MoMo scores great on UnseenSRA but has lower FID and text adherence. This is shown by the pareto frontier shown in Sec 4.2. Our method again finds the optimal balance between zero-shot style transfer and content adherence.
Q2: Sensitivity to the number of basis vectors and the quantisation step
We retrained the codec with K ∈ {4, 8, 32, 64} basis vectors and ∈ {0.125, 0.5, 1.0}, one change each. trades forward leak smoothly (0.348 → 0.193 → 0.064 → 0.054) with MPJPE increasing 32.7 → 35.3 mm. Higher provides better forward leak and worse MPJPE (stronger rounding in ). This can be seen in the Suppl. Video 18:34-18:40 where =1.0 (right most) shows higher foot sliding. Our default settings balance style and content reasonably. Throughout this range, disentanglement is easy to achieve without any representational collapse.
Regarding basis vectors, the conclusions hold for K ≥ 16: transfer is stable or better above 16 (K = 64 reaches unseen SRA 0.276, while reducing CRA). This is shown in Suppl. Video 18:58-19:11, where small K=4 also does not follow the lean back style.
| Cell @250k iter | MPJPE (mm)↓ | Seen SRA↑ | Unseen SRA↑ | CRA↑ | Fwd leak style←↓ | Rev leak content←↓ |
|---|---|---|---|---|---|---|
| Default (K 16, 0.25), — ablation table "full" row | 33.5 | 0.479 | 0.184 | 0.374 | 0.193 | 0.521 |
| K=4 | 35.3 | 0.131 | 0.019 | 0.310 | 0.310 | 0.234 |
| K=8 | 34.5 | 0.338 | 0.078 | 0.284 | 0.254 | 0.303 |
| K=32 | 33.0 | 0.544 | 0.239 | 0.290 | 0.185 | 0.535 |
| K=64 | 33.6 | 0.559 | 0.276 | 0.313 | 0.172 | 0.461 |
| =0.125 | 32.7 | 0.425 | 0.195 | 0.299 | 0.348 | 0.491 |
| =0.5 | 33.4 | 0.468 | 0.163 | 0.279 | 0.064 | 0.564 |
| =1.0 | 35.3 | 0.498 | 0.230 | 0.340 | 0.054 | 0.631 |
Dear Rev. WANA,
We wanted to check if the rebuttal has answered your concerns or if you have more questions. Let us know if you have any further questions before Author-Reviewer Discussion deadline.
Best, Authors.
Rebuttal WANA (part 1)
Please note relevant parts of Suppl. Video and Paper are highlighted with [Rev. WANA]. Following discussion/analyses will be incorporated in paper after author-reviewer discussion.
We thank the reviewer for their constructive feedback and are happy to know they see value in separating style and content in the motion representation itself and found evidence of generalizability to downstream tasks.
We address the reviewer’s concerns below:
W2/Q3: Direct Evaluation of Disentanglement: “does changing style keep the action, and changing content keep the style?”
Thanks for pointing it out. We directly evaluate disentanglement by creating a factorial test grid of 12 content clips (rows) × all 10 held-out styles from 100STYLES (columns), keeping same clips for every method. Every output is read by the two classifiers we used for SRA and CRA.
For each classifier, we measure how its full probability output (not top-1) varies along rows and columns of the R×C grid:
1. Take its probability vector prc for every cell, and the grand mean pˉ over all cells.
2. Total variation T=∑r,c∥prc−pˉ∥2.
3. Row variation: average each row over its C styles, pˉr, and compute Row=C⋅∑r∥pˉr−pˉ∥2 (Content-driven change).
4. Column variation: average each column over its R content clips, pˉc, and compute Col=R⋅∑c∥pˉc−pˉ∥2 (Style-driven change).
5. Shares: row share = Row / T and column share = Col / T. The remainder is pair-specific.
From these shares, we summarise it in two ways:
Style Selectivity: column share − row share : +1: the output's style changes only when the style clip changes; −1: only when the content clip changes (the output keeps the content clip's own style).
Content Selectivity: row share − column share;+1: the output's action changes only with the content clip; −1: only with the style clip (the output performs the style clip's action)
Style specificity: Col / (Col + Row): of the change in the style reading that one of the two inputs explains, the fraction explained by the style clip.
Content specificity: Row / (Row + Col): of the change in the action reading that one of the two inputs explains, the fraction explained by the content clip.
1 = the reading follows only the correct input, 0.5 = it follows both inputs equally, 0 = it follows only the wrong one.
Ideal disentanglement should have both selectivities > 0, both specificities > 0.5.
DisCodec is the only method that meets both conditions. Each baseline takes one factor from the wrong input. MoST keeps the content clip's own style: style selectivity −0.57, specificity 0.07, i.e., it largely returns the content clip. Motion Puzzle takes the motion from the style clip (content selectivity −0.230, specificity 0.19), and GenMoStyle's motion follows the style clip more than the content clip (−0.068, 0.44).
Note that GT trajectories are used to consistent with baseline. If we use generated trajectories, baseline performance would worsen.
Visual Results. We also visualize interpolation between two styles while keeping the content fixed (Suppl. Video 17:53-18:18) . The style changes from left to right, while the content stays fixed. Conversely, fixing style and varying content produces those actions in one style (Suppl. Video 18:18-18:33).
This demonstrates DisCodec's bi-directional disentanglement rather than copying either reference. Note that all the styles and identities used in visual results are unseen by our method.