acceptodds
Under review as a conference paper at ICLR 2027

Disentangled Representations for Zero-shot Behavioral Style Transfer

Abstract

Human behavior is exhibited through bodily motion that not only achieves its intent-driven or communicative goals, but also demonstrates stylistic and idiosyncratic patterns. Existing approaches for generative motion synthesis learn to transfer these styles to novel motions by learning adapters or designing task-specific guidance objectives. However, disentangling style and content within the core motion representation remains an open problem. We present DisCodec, a general framework for building disentangled representations of style and content in human motion, which can be used for zero-shot style transfer at the decoding stage. Our approach learns these representations through a neural motion codec trained with a Variational Information Bottleneck and SNR-adaptive Transform Quantization to disentangle style from the motion's action content. Two streams of encoders decompose any reference motion into style and content latents, which are then combined to reconstruct the motion, while also ensuring that these latents represent mutually distinct information about the motion. Therefore, any generative motion synthesis task learned using the proposed representation not only inherits zero-shot style transfer capability but can also sample styles from the learned style subspace. We benchmark the style transfer performance against existing approaches and independently demonstrate applications of DisCodec representation in text-to-motion synthesis and co-speech gesture generation tasks. Reader is urged to watch accompanying video demo.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Human Motion Generation, Style Transfer, Gesture Synthesis

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.80 over 5 reviews, above 85% of the others.
Reviewer_WANAOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents DisCodec, a motion codec that separates style and content into different latent representations. It uses a variational information bottleneck, a learned style basis, and SNR-adaptive transform quantization to reduce information leakage between the two representations. The authors evaluate the method on motion style transfer, text-to-motion generation, and co-speech gesture generation. The results are promising, particularly for transferring styles without retraining for each new style. However, the results on unseen styles are mixed, and it is not yet clear how well the learned representations generalize beyond the tested datasets.

Critical strengths
  1. The method separates style and content in the motion representation itself, rather than relying on a separate style adapter for each new style. This makes it potentially useful for applications where new reference motions become available at inference time.

  2. The authors evaluate the codec on three different tasks and include ablations of the main components. This provides some evidence that the representation can be reused across different generation settings.

Critical weaknesses
  1. The results on unseen styles are not consistently strong. In the 100STYLE experiment, DisCodec has lower unseen-style recognition accuracy than some competing methods. This raises questions about how well the learned style space actually generalizes to styles that were not present during training.

  2. The evidence for disentanglement is still limited. The paper mainly relies on linear probes, reconstruction metrics, and downstream generation results. These are useful measurements, but they do not fully establish that style and content are independently represented. It would help to show more directly whether changing the style latent preserves the action and whether changing the content latent leaves the style intact.

Questions for authors
  1. In Table 1, why is the unseen-style recognition accuracy of DisCodec trained on 100STYLE considerably lower than that of GenMoStyle and MotionPuzzle? How should this result be reconciled with the claim of effective zero-shot style transfer?

  2. How sensitive are the results to the choice of the number of style basis vectors and the quantization parameter? Do the main conclusions hold under different settings?

  3. Can the authors provide a more direct evaluation of disentanglement, showing that style changes preserve action content and content changes preserve style, particularly for unseen styles?

5 comments
AuthorsOpenReview ↗

Rebuttal WANA (part 1)

Please note relevant parts of Suppl. Video and Paper are highlighted with [Rev. WANA]. Following discussion/analyses will be incorporated in paper after author-reviewer discussion.

We thank the reviewer for their constructive feedback and are happy to know they see value in separating style and content in the motion representation itself and found evidence of generalizability to downstream tasks.
We address the reviewer’s concerns below:

W2/Q3: Direct Evaluation of Disentanglement: “does changing style keep the action, and changing content keep the style?”

Thanks for pointing it out. We directly evaluate disentanglement by creating a factorial test grid of 12 content clips (rows) × all 10 held-out styles from 100STYLES (columns), keeping same clips for every method. Every output is read by the two classifiers we used for SRA and CRA.
For each classifier, we measure how its full probability output (not top-1) varies along rows and columns of the R×C grid:
1. Take its probability vector prcp_{rc} for every cell, and the grand mean pˉ\bar{p} over all cells.
2. Total variation T=∑r,c∥prc−pˉ∥2T = \sum_{r,c} \Vert p_{rc} - \bar{p} \Vert^2.
3. Row variation: average each row over its C styles, pˉr\bar{p}_r, and compute Row=C⋅∑r∥pˉr−pˉ∥2\mathrm{Row} = C \cdot \sum_r \Vert \bar{p}_r - \bar{p} \Vert^2 (Content-driven change).
4. Column variation: average each column over its R content clips, pˉc\bar{p}_c, and compute Col=R⋅∑c∥pˉc−pˉ∥2\mathrm{Col} = R \cdot \sum_c \Vert \bar{p}_c - \bar{p} \Vert^2 (Style-driven change).

5. Shares: row share = Row / T and column share = Col / T. The remainder is pair-specific.

From these shares, we summarise it in two ways:

  • Style Selectivity: column share − row share : +1: the output's style changes only when the style clip changes; −1: only when the content clip changes (the output keeps the content clip's own style).

  • Content Selectivity: row share − column share;+1: the output's action changes only with the content clip; −1: only with the style clip (the output performs the style clip's action)

  • Style specificity: Col / (Col + Row): of the change in the style reading that one of the two inputs explains, the fraction explained by the style clip.

  • Content specificity: Row / (Row + Col): of the change in the action reading that one of the two inputs explains, the fraction explained by the content clip.
    1 = the reading follows only the correct input, 0.5 = it follows both inputs equally, 0 = it follows only the wrong one.

Ideal disentanglement should have both selectivities > 0, both specificities > 0.5.

100STYLE (12 × 10)Style selectivity↑Content selectivity↑Style specificity↑Content specificity↑
Copy-content control−1.000+1.0000.001.00
Copy-style control+1.000−1.0001.000.00
MoST−0.571+0.7050.070.97
GenMoStyle (unsupervised)+0.367−0.0680.860.44
Motion Puzzle+0.261−0.2300.810.19
DisCodec (100STYLE codec)+0.721+0.1950.960.66

DisCodec is the only method that meets both conditions. Each baseline takes one factor from the wrong input. MoST keeps the content clip's own style: style selectivity −0.57, specificity 0.07, i.e., it largely returns the content clip. Motion Puzzle takes the motion from the style clip (content selectivity −0.230, specificity 0.19), and GenMoStyle's motion follows the style clip more than the content clip (−0.068, 0.44).
Note that GT trajectories are used to consistent with baseline. If we use generated trajectories, baseline performance would worsen.

Visual Results. We also visualize interpolation between two styles while keeping the content fixed (Suppl. Video 17:53-18:18) . The style changes from left to right, while the content stays fixed. Conversely, fixing style and varying content produces those actions in one style (Suppl. Video 18:18-18:33).
This demonstrates DisCodec's bi-directional disentanglement rather than copying either reference. Note that all the styles and identities used in visual results are unseen by our method.

AuthorsOpenReview ↗

Rebuttal WANA (part 2)

W1/Q1: On unseen SRA being lower than baselines.

Unseen SRA as a metric is skewed towards generations that are closest to the input style.
And high performance by GenMoStyle and MotionPuzzle on this metric is understandable as these methods copy the style (MotionPuzzle more than GenMoStyle) and do not keep the content-related parts of motion.
This is clearly evident by high performance on UnseenSRA but high FID and low CRA (which is the accuracy of the content classifier on unseen content motions).
As described in lines 403-412, MotionPuzzle completely copies the style reference motion, and GenMoStyle also follows a similar pattern. This is also evident in video example (Suppl. Video 12:09-12:26).
For GenMoStyle specifically, this follows the trend of uni-directional disentanglement where style information is taken out of content latents in a proper way (low style ← zcz_c) but the method fails to disentangle content from style (content ← zsz_s) and find the right balance between SRA and CRA/FID (see Table 1 in paper).

Note that trajectory for the motion-to-motion comparison table comes from GT to be consistent with baselines, as noted in Ln. 412. The difference between our method and baselines (especially GenMoStyle and MotionPuzzle) is more apparent when we show the trajectory generated by each method. (Suppl. Video 12:26-12:42).
We can see in the first example (Suppl. video 12:26-12:36) of unseen Armaching style, MotionPuzzle completely follows the style and GenMoStyle walks in its own place (being close to the style trajectory). DisCodec produces motion that follows the trajectory but walks in a slow manner with one arm upheld to show the semantics of the style. A similar pattern can be seen for the next example (Suppl. Video 12:36-12:42).

Due to low content adherence, GenMoStyle/MotionPuzzle try to copy style motion’s trajectory and lower body instead of the content motion (evident from 1st video example 12:26-12:36 in Suppl. Video). As content usually resides in lower body motion, we can further demonstrate the difference by comparing the correlation of foot rhythm to content motion for all methods below, across 100style unseen test set.
Correlation can be calculated between output's and content example’s foot-contact signals (share of feet planted per frame with near-zero velocity).

100STYLE, unseen pairsMotionPuzzleGenMoStyleOurs
Correlation to Content Foot rhythm0.060.330.54

Note that all the styles and identities used in visual results are unseen by our method.

Since metrics like UnseenSRA struggle to provide a holistic answer, we therefore conduct a perceptual evaluation against GenMoStyle and we find that evaluators prefer stylized motions from our method 71.4% more than those from GenMoStyle in terms of style matching (Fig. 5).
Additionally, this pattern is similar to text-to-motion baselines where MoMo scores great on UnseenSRA but has lower FID and text adherence. This is shown by the pareto frontier shown in Sec 4.2. Our method again finds the optimal balance between zero-shot style transfer and content adherence.

Q2: Sensitivity to the number of basis vectors and the quantisation step

We retrained the codec with K ∈ {4, 8, 32, 64} basis vectors and δδ ∈ {0.125, 0.5, 1.0}, one change each. δδ trades forward leak smoothly (0.348 → 0.193 → 0.064 → 0.054) with MPJPE increasing 32.7 → 35.3 mm. Higher δδ provides better forward leak and worse MPJPE (stronger rounding in zcz_c). This can be seen in the Suppl. Video 18:34-18:40 where δδ=1.0 (right most) shows higher foot sliding. Our default settings balance style and content reasonably. Throughout this range, disentanglement is easy to achieve without any representational collapse.

Regarding basis vectors, the conclusions hold for K ≥ 16: transfer is stable or better above 16 (K = 64 reaches unseen SRA 0.276, while reducing CRA). This is shown in Suppl. Video 18:58-19:11, where small K=4 also does not follow the lean back style.

Cell @250k iterMPJPE (mm)↓Seen SRA↑Unseen SRA↑CRA↑Fwd leak style←zcz_c↓Rev leak content←zsz_s↓
Default (K 16, δδ 0.25), — ablation table "full" row33.50.4790.1840.3740.1930.521
K=435.30.1310.0190.3100.3100.234
K=834.50.3380.0780.2840.2540.303
K=3233.00.5440.2390.2900.1850.535
K=6433.60.5590.2760.3130.1720.461
δδ=0.12532.70.4250.1950.2990.3480.491
δδ=0.533.40.4680.1630.2790.0640.564
δδ=1.035.30.4980.2300.3400.0540.631

Dear Rev. WANA,

We wanted to check if the rebuttal has answered your concerns or if you have more questions. Let us know if you have any further questions before Author-Reviewer Discussion deadline.

Best, Authors.

Reviewer_vZvnOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces DISCODEC, a dual-stream motion codec intended to separate content and style for reference-based stylization and downstream text-to-motion and speech-to-gesture generation.

Critical strengths

Its noise-adaptive transform quantizer is a concrete, potentially useful design, and the paper reports improvements in reconstruction fidelity and several content-preservation metrics, alongside component ablations and perceptual comparisons.

Critical weaknesses
  1. The added significance beyond existing work is insufficiently established. Motion style transfer is practically relevant, but label-free reference-based transfer, probabilistic style representations, and representation-level code swapping already have precedents in Motion Puzzle, GenMoStyle, and VQ-Style, respectively. The paper does not clearly identify which remaining bottleneck its codec resolves or what substantial new capability it offers. Although the architecture has meaningful differences, its contribution beyond another combination of disentanglement techniques remains unclear.

  2. The disentanglement mechanism and its benefits are insufficiently supported. Separate VIB constraints and joint reconstruction do not by themselves ensure semantic separation of content and style. The hypothesis that style leakage resides in low-SNR directions also lacks direct evidence, as full-latent probes and retained-variance statistics do not distinguish selective style removal from generic compression or denoising. Moreover, the p.9 ablation shows that SNR quantization reduces style-from-content probe accuracy from 0.725 to 0.193, but lowers unseen-style SRA from 0.233 to 0.184 and increases content-from-style probe accuracy from 0.389 to 0.521. These results indicate a trade-off rather than consistently improved bidirectional disentanglement or zero-shot transfer. The paper should reconcile these findings and provide stronger evidence for the proposed mechanism’s joint content–style benefits.

Questions for authors
  1. What specific unresolved bottleneck does DISCODEC address? Beyond capabilities already demonstrated by existing reference-based and representation-level stylization methods, what new capability or transferable insight does DISCODEC provide? Could you substantiate its advantage through matched comparisons, particularly in label requirements, generalization, or downstream adaptation cost?

  2. What evidence localizes style leakage to low-SNR directions? Could you provide style and content probes restricted to low- versus high-SNR directions, together with controlled removal experiments? Comparisons against PCA truncation, random-direction removal, or fixed-step transform quantization at matched capacity or reconstruction distortion would help determine whether the effect is style-specific.

  3. How should the quantization ablation be interpreted? Why does SNR quantization reduce forward leakage while worsening unseen-style recognition and reverse leakage? Can you demonstrate a consistent improvement in the joint content–style trade-off, rather than primarily reconstruction fidelity, using clearly specified evaluation protocols and uncertainty estimates?

8 comments
AuthorsOpenReview ↗

Rebuttal vZvn (part 1)

Please note relevant parts of Suppl. Video and Paper are highlighted with [Rev. vZvn]. Following discussion/analyses will be incorporated in paper after author-reviewer discussion.

We thank the reviewer for feedback and are glad they found SNR-adaptive Transform Quantization concrete and potentially useful.

W1/Q1: What bottleneck does DisCodec resolve, beyond Motion Puzzle, GenMoStyle and VQ-Style?

We share the motivation and goal of zero-shot style transfer with the above-mentioned works. However, the DisCodec framework provides one stylization recipe for two important problems together due to its design:
1) learning a parametric style space in a label-free manner
2) performing bi-directional disentanglement, i.e., not only disentangled in the style branch but also in the content branch (unlike GenMoStyle).

Designing one approach to these problems enables generalization across motion-to-motion, text-to-motion and speech-to-gesture tasks while also demonstrating disentanglement in both style and content directions.

In terms of bottlenecks our framework resolves, our parametric basis gives user style control through coefficients (see Suppl. Video 01:47-09:37). In contrast, MotionPuzzle and VQ-Style (label-based) lack a structured & interpretable style space.
GenMoStyle's post-hoc Gaussian prior is unstructured and not controllable along semantic axes (see controllability discussion below).

Another bottleneck is finding the right way to store the disentangled content information in the content latents. GenMoStyle and MotionPuzzle show quite high zero-shot style transfer in reference-based stylization (Table 1). However, they fail to capture the content information properly, resulting in worse CRA and leakage of content information in the style prior. GenMoStyle is only uni-directional i.e. it removes style from content latents, whereas ours is bi-directional which matches the goal of separate latents.
This is also evident from the visual comparison results shown in the video (Suppl. Video 11:37-12:10), where GenMoStyle does not adhere to style relevant hand movements and MotionPuzzle completely fails.

In the following table we provide a complete rundown of reference-based transfer baselines:

MethodLabel RequirementParametric or Interpretable style spaceGenerator trained on the content code
MoSTYesNoNo
Motion PuzzleNoNoNo
GenMoStyle (unsupervised)No (needs sequence grouping)No. (only unstructured Gaussian prior)No: content code exists, but T2M demo is a cascade (off-the-shelf T2M motion → their encoder)
VQ-StyleYes (contrastive + MI on labels)NoNo: transfer is code swapping; no generative model is trained on the codes
DisCodec (ours)NoYes (16-d orthogonal basis, can also be sampled by the DiT or controlled by interpretable axes)Yes: DiT and ActionPlan generate zcz_c from text, no cascade required

On bottleneck of controllability:
Controllability and style control are one of the major bottlenecks for motion generation.
As DisCodec additionally provides a parametric space for style control, new ways and dimensions of control can emerge due to this design (without any explicit training for this control task). We can find new semantic directions of style control in 16-d parametric space, just by fitting simple linear regression probes (Suppl. Video 07:11-09:37). These control directions currently include HandSpread & WristHeight and can be used to stylize any downstream generation method with an interpretable axis. This capability is beyond the existing architectures and is only possible due to DisCodec’s representational design.

Inspired by the controllable motion synthesis literature (Line 156-161), we perform probing analysis on 16 style coefficients and demonstrate that we can find directions of interpretable properties of motion. The properties can include Hand spread(distance between the two hands) and Wrist height (height of both wrists). They can serve as new semantic axes for style control. We add additional sliders for these semantic axes (Suppl. Video 07:11-09:13) and demonstrate the generalizability of our learned style space by editing style along these new dimensions of control (for which our method was not trained at all). This provides more semantically meaningful directions of control for any off-the-shelf motion generation method and would only require just a single decoder pass for stylization.
Note that this is not limited to these attributes. Future work can explore and develop this further in the current design.

Probing Setup: We fit a regression probe for each attribute. The probe measures if the attribute is linearly readable from a clip's 16 style coefficients. We show Pearson correlation between ridge prediction and measured attributes on test clips.

Attributer, predicted vs measured
Hand spread0.897
Wrist height0.927
AuthorsOpenReview ↗

Rebuttal vZvn (part 2)

W2 / Q2: Is the leak really in low-SNR directions, or is the grid generic compression?

Localisation.
We thank the reviewer for pointing this out and have added additional analysis to motivate the choice of SNR-based quantization. We show localisation through probes on a codec trained without SNR quantization and trained with a continuous content latent. The style sits in the directions that carry almost no variance, the content in the directions that carry almost all of it. The 32 lowest-SNR directions hold 0.001% of the content code’s variance, yet a probe reads the style from them at 0.490 (full code 0.793) and the content at only 0.278 (full code 0.726). The 32 highest-SNR directions hold 99.9% of the variance and read the content at 0.621, but the style at only 0.320. The 91 directions the δ = 0.25 grid erases hold 0.05% of the variance and read the style as well as the full code (0.782 vs 0.793). This validates our design of rounding off low-SNR directions and using SNR to decide which directions to remove.

Direction subsetkShare of varianceStyle ← subsetContent ← subset
All directions128100%0.7930.726
Lowest SNR320.001%0.4900.278
Highest SNR3299.9%0.3200.621
Random Selection (mean of 5)3226%0.4760.523
Lowest SNR640.007%0.7200.376
Highest SNR6499.99%0.6160.725
Set erased by default δδ=0.25 grid910.05%0.7820.574
Set kept by the grid3799.95%0.3800.638

Style-specific, not generic compression. We further show that random removal also hurts MPJPE performance on codec with continuous content latents. We also demonstrate this random removal experiment on the pre-quantized mean of content latents inside SNR-quantized codec (same as DisCodec). Dropping the 92 lowest-SNR directions costs 0.0 mm MPJPE and cuts style 0.79 → 0.36 while content goes 0.73 → 0.64. Dropping 92 random directions costs 76–111 mm (three draws), and no random removal stays within +10 mm.

Codec cellDirections removedLowest-SNR removed: ΔMPJPE / style / contentRandom removed (mean of 3 draws): ΔMPJPE / style / content
Continuous; ref. MPJPE 33.2; probes on full: style 0.79, content 0.7316+0.0 mm / 0.76 / 0.72+22 mm / 0.78 / 0.73
32+0.0 mm / 0.72 / 0.72+51 mm / 0.76 / 0.73
64+0.0 mm / 0.62 / 0.72+62 mm / 0.67 / 0.68
92+0.0 mm / 0.36 / 0.64+97 mm / 0.53 / 0.52
SNR-quantized-trained (pre-quantized mean); ref. MPJPE 31.3; probes on full: style 0.79, content 0.7316+0.0 mm / 0.77 / 0.73+28 mm / 0.78 / 0.72
32+0.0 mm / 0.75 / 0.75+36 mm / 0.76 / 0.72
64+0.0 mm / 0.62 / 0.72+53 mm / 0.68 / 0.69
92+0.0 mm / 0.37 / 0.62+87 mm / 0.52 / 0.59

Further Comparisons. We compare SNR-adaptive quantization against a fixed-step transform quantizer, PCA truncation and random-direction removal at matched distortion. We express each content code zcz_c (train and test) in the eigenbasis used by the SNR quantizer, then sweep each method's strength: the SNR step δδ, the fixed quantizer's step, or the number of lowest-variance (PCA) or random directions zeroed.
At each setting we (i) decode test clips from the transformed zcz_c with their own style codes and record the MPJPE increase over the clean decode (distortion), and (ii) refit style/content probes on transformed training codes and score them on transformed test codes. We then compare forward leakage (style ← zcz_c) at equal distortion, +1 mm MPJPE: less remaining style means more selective removal.
At +1 mm, the SNR quantizer leaves 0.223 style accuracy vs. 0.241 (fixed-step) and 0.266 (PCA); clean code 0.793. Random removal cannot reach this budget: zeroing just 16 random directions already costs >10 mm.

Forward leakage probe accuracies (style ← zcz_c) at +1 mm:

Codec cellSNR quantizer (ours)Fixed-step transformPCA truncationRandom removal (cannot reach +1 mm)
Continuous0.2000.2420.272+28 mm already at 16 directions; style remains 0.77
SNR-quantized-trained (pre-quantized mean)0.1380.1490.195+28 mm already at 16 directions; style remains 0.78
AuthorsOpenReview ↗

Rebuttal vZvn (part 3)

Q3: How should the quantization ablation be interpreted? Can you demonstrate a consistent improvement in the joint content–style trade-off, rather than primarily reconstruction fidelity,

DisCodec is trained with reconstruction loss as a primary goal. Therefore, we prioritize a design that provides us with good reconstruction and disentanglement. SNR quantization helps us achieve both.
The ablation performed is meant to show that balancing disentanglement across two branches is a trade-off of putting information in either branch. SNR-adaptive quantization acts as a disentanglement push on the content branch, which understandably takes information out of zcz_c. Now for a motion codec with the primary goal of reconstruction fidelity, the network needs to place small content information elsewhere, hence the increase in the reverse leakage. The trick is to find the right balance of disentanglement pressure on both streams while also keeping reconstruction fidelity. Using SNR-adaptive quantization provides the reconstruction performance while giving us a way to control the disentanglement bi-directionally. Please refer to sensitivity analyses that shows disentanglement pipeline is stable for different parameters (see answer to Q1 of Rev. 2T2d).
In terms of joint content-style tradeoff, we see a consistent improvement in downstream tasks where DisCodec pushes the pareto frontier between unseen style transfer and content adherence (Sec. 4.2). Text-to-motion generation methods either perform good zero-shot transfer with bad text adherence (MoMo) or only follow input text without style transfer (LoRA-MDM/SMooDi). DisCodec provides the balanced version of both following text as well as performing zero-shot style transfer, hence pushing the pareto-front.
We will add this explanation/discussion in the final version of the draft after author-reviewer discussion.

SNR-quantization helps learn better in downstream tasks.
We can show this by training text-to-motion DiT and ActionPlan methods on both continuous and SNR-quantized latents (first two rows of ablation table on p.9):

GeneratorSRA top-1↑SRA top-3↑Unseen SRA↑
SNR quantized latents: paper’s DiT 12×1024 (= Table 2 row)0.3470.5220.267
Continuous latents: identical DiT0.2490.4010.226
SNR grid latents: paper’s ActionPlan0.3180.5150.239
Continuous latents: identical ActionPlan0.2250.3960.201
Reviewer_mAQDOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

DISCODEC learns separate global style and temporal content representations for human motion, combining variational bottlenecks, an orthogonal style basis, temporal-crop consistency, and SNR-adaptive quantization of content latents. A conditioned decoder enables reference-based style transfer without style-specific adaptation, with applications to text-to-motion and co-speech gesture generation.

Critical strengths
  • The paper is clearly written and easy to follow.
  • The visual demos are comprehensive, and the generated motions are with satisfying quality overall.
Critical weaknesses
  • The methodology seems generally overlapped with GenMoStyle, which are both established in a VAE-style representation learning, and techniques involved probabilistic style codes, label-free training, within-sequence style consistency for similar purpose. Though the authors propose different approaches, the claimed improvement in unseen-style transfer is not well supported, as GenMoStyle achieves way better unseen SRA shown in Table 1.
  • Other previous related works, such as VQ-Style [1] and ClusterStyle [2], also tackle with unseen-style transfer and advanced style-content disentanglement. Yet there are no discussion or comparison with these works.
  • The text-to-motion performance looks largely limited and not well generalized shown in Table 2 and also in the video demo.
  • The generated speech-to-gesture personalization seems not well demonstrated. There are huge body penetration and ambiguous motions in the generated gestures, where one can't easily tell the style except the posture.

[1] Fatemeh Zargarbashi, Dhruv Agrawal, Jakob Buhmann, Martin Guay, Stelian Coros, and Robert W. Sumner. VQ-Style: Disentangling Style and Content in Motion with Residual Quantized Representations. Computer Graphics Forum, 45(2):e70377, 2026. [2] Kerui Chen, Jianrong Zhang, Ming Li, Zhonglong Zheng, and Hehe Fan. ClusterStyle: Modeling Intra-Style Diversity with Prototypical Clustering for Stylized Motion Generation. In Proceedings of the European Conference on Computer Vision (ECCV), 2026.

Questions for authors
  • The SNR adaptive quantization mechanism is also unclear to me. Is there any justification and illustration of why that the suppressed low-SNR directions has to carry style leakage specifically?

  • Currently the latent space is proven to be disentangled with the orthogonal basis well decomposed, but not semantically meaningful. Is there a way that we can semantically control the process of generating novel styles with the latent space component?

3 comments
AuthorsOpenReview ↗

Rebuttal mAQD (part 1)

Please note relevant parts of Suppl. Video and Paper are highlighted with [Rev. mAQD]. Following discussion/analyses will be incorporated in paper after author-reviewer discussion.

We thank the reviewer for their valuable feedback and are happy to know that they found our paper clearly written and visual demos comprehensive. We address reviewer’s concerns below:

W1: Comparison with GenMoStyle

We share the motivation and goal of disentanglement and zero-shot style transfer with GenMoStyle. However, rather than post-hoc disentanglement on fixed representations, we learn inherently disentangled style/content representations that a downstream generator uses directly, giving built-in stylization.
Beyond disentanglement, we also target:
1) learning a parametric style space in a label-free manner
2) bi-directional disentanglement, in both the style and content branches (unlike GenMoStyle, which only demonstrates uni-directional disentanglement).
We compare our method against GenMoStyle along three capability dimensions:

MethodLabel RequirementParametric or Interpretable style spaceGenerator trained on the content code
GenMoStyle (unsupervised)NoNo. (Sampleable only through unstructured Gaussian prior)No: content code exists, but T2M demo is a cascade (off-the-shelf T2M motion → their encoder)
DisCodec (ours)NoYes (16-d orthogonal basis, can also be sampled by the DiT or controlled by interpretable axes)Yes: DiT and ActionPlan generate zcz_c from text, no cascade needed

The parametric space that we learn through the style bases can be sampled by a downstream generator or be used to control styles through raw coefficients and semantic axes (see answer to Q2). GenMoStyle generates only via a cascade, stylizing T2M outputs. Our framework provides representations that can readily be used for generation and only need one post-hoc decoder pass for zero-shot stylization.

Metrics. Unseen SRA as a metric is skewed towards generations that are closest to the input style. And high performance by GenMoStyle and MotionPuzzle on this metric is understandable, as these methods copy the style and do not keep the content-related parts of motion.
This is clearly evident by high performance on UnseenSRA but high FID and low CRA (which is the accuracy of content classifier on unseen content motions).
As described in Line 403-412, GenMoStyle is much closer to the style reference motion and struggles with content adherence. This is also evident in the video example (Suppl. Video 12:09-12:26).
GenMoStyle follows the trend of uni-directional disentanglement where style information is taken out of content latents in a proper way (low style ← zcz_c), but the method fails disentangle content from style (content ← zsz_s) and find the right balance between SRA and CRA/FID (see Table 1 in paper).

Note that the trajectory for the motion-to-motion comparison table comes from GT to be consistent with baselines, as noted in Ln. 412. The difference between our method and baselines (especially GenMoStyle) is more apparent when we show the trajectory generated by each method. (Suppl. Video 12:26-12:42).
We can see in the first example (Suppl. Video 12:26-12:36) of unseen Armaching style, GenMoStyle walks in its own place (being close to style trajectory). DisCodec produces motion that follows the trajectory but walks in a slow manner with the arm upheld to show the semantics of the style. A similar pattern can be seen in the next example (Suppl. Video 12:36-12:42), where GenMoStyle drifts in order to match the style trajectory, and DisCodec keeps the content-specific action and trajectory while keeping arms similar to the style.

As content usually resides in lower body motion, we can further demonstrate the difference by comparing the correlation of foot rhythm to content motion for all methods below, across the 100STYLE unseen test set. DisCodec is better than both style copying methods (GenMoStyle/MotionPuzzle).
Correlation can be calculated between the output's and content example’s foot-contact signals (share of feet planted per frame with near-zero velocity).

100STYLE, unseen pairsMotionPuzzleGenMoStyleOurs
Correlation to Content Foot rhythm0.060.330.54

Note that all the styles and identities used in visual results are unseen by our method.
Since metrics like UnseenSRA struggle to provide a holistic answer, we therefore conduct a perceptual evaluation against GenMoStyle, and we find that evaluators prefer stylized motions from our method 71.4% more than those from GenMoStyle in terms of style matching (Fig. 5).

AuthorsOpenReview ↗

Rebuttal mAQD (part 2)

Q1: SNR-adaptive Transform Quantization and Analysis of SNR directions.

Explanation. The content VIB adds noise in training, but at inference the decoder reads the noise-free mean μcμ_c, where style survives in low-SNR directions (masked by noise, yet readable from the mean). To remove it, we:
1) rotate μcμ_c into the eigenbasis of its covariance;
2) round each direction jj with a step equal to its posterior noise, Δj=δnjΔ_j = δ n_j;
3) thus, directions with SNRj=sj/nj\mathrm{SNR}_j = s_j/n_j below a threshold (∝ δδ) round to zero and are erased, while strong (content) directions move ≤ half a step, within the noise the decoder tolerates.
This cuts style←zcz_c probe accuracy from 0.76 to 0.18 at no cost to reconstruction or held-out transfer.

Localisation Experiment.
To motivate SNR quantization, we probe a codec trained without it (continuous content latent).
We find that style sits in near-zero-variance directions, content in high-variance ones (shown by table below): the 91 directions the δδ=0.25 grid erases hold 0.05% of variance yet read style as well as the full code (0.782 vs 0.793). This validates using SNR to decide which directions to round off.

Direction subsetkShare of varianceStyle ← subsetContent ← subset
All directions128100%0.7930.726
Lowest SNR320.001%0.4900.278
Highest SNR3299.9%0.3200.621
Random Selection (mean of 5)3226%0.4760.523
Lowest SNR640.007%0.7200.376
Highest SNR6499.99%0.6160.725
Set erased by default δδ=0.25 grid910.05%0.7820.574
Set kept by the grid3799.95%0.3800.638

Q2: Semantic Axes for Style Control through Basis

We agree this capability strengthens our style basis. Inspired by the controllable motion synthesis literature (Line 156-161), we probe the 16 style coefficients and find interpretable directions, e.g., Hand spread (distance between hands) and Wrist height, which serve as new semantic axes. We show additional sliders for these semantic axes (Suppl. Video 07:11-09:13, 09:13-09:37) and demonstrate the generalizability of our learned style space by editing style along these new dimensions (for which our method was not trained at all). This gives any off-the-shelf generator semantic control with a single decoder pass. Note that other attributes can also be explored in future work.

Probing Setup: We fit a regression probe for each attribute. The probe measures if the attribute is linearly readable from a clip's 16 style coefficients. We show Pearson correlation between ridge prediction and measured attributes on test clips.

Attributer, predicted vs measured
Hand spread0.897
Wrist height0.927

W3: Text-to-Motion Performance
We compare with LoRA-MDM and SMooDi, which finetune on each style's data (not zero-shot), and with zero-shot MoMo.
In Table 1, the finetuned methods excel in text adherence (R@1) and FID, as they train on the GT style motions FID measures against, while our method is zero-shot. However, these methods cannot fully learn style from few samples, so we outperform them in style transfer (New Videos: Suppl. Video 13:46-15:27. Matching finetuned methods zero-shot already shows strong T2M performance.
We also beat MoMo on content adherence, as MoMo copies the style and fails to follow text. This forms a Pareto frontier (Sec 4.2) that DisCodec pushes towards both style transfer and content adherence. Note that all the styles and identities used in visual results are unseen by our method.
Our method only needs decoding to transfer to any style (6.6ms on 1 CPU thread for 10 s of motion), vs. ~71 min per style for LoRA-MDM (RTX 3090).

W4: Speech-to-gesture personalization
We add dedicated results for speech-to-gesture personalization (Suppl. Video 15:29-17:50). Here again, we compare with methods that learn speech-to-gesture personalization with label-based embeddings. These learn person-specific traits in training and cannot transfer zero-shot. We aim to match them zero-shot and push beyond them in future work. Note that identities used in visual results are unseen by our method.
Penetration & Ambiguous Motions: The cross-domain example (Suppl. Video 10:16-10:26) tests extreme personalization by driving gesture generation trained on BEATX & Seamless Interaction using a style sample from PerMo, which is a vastly different distribution and task.
This manifests in posture being transferred, which we believe is the expected behavior for stylization as content motion lies in arm gesticulations. Artifacts stem from this domain gap and do not occur in-domain, e.g., DisCodec gives sensible poses in text-to-motion.

W2: VQ-Style and ClusterStyle
Thank you for the references. We briefly discuss VQ-Style (Ln. 148), and will expand it. ClusterStyle will also be cited. Unfortunately, we cannot perform comparison with the two methods as the codebase for them is not open-sourced.

Dear Rev. mAQD,

We wanted to check if the rebuttal has answered your concerns or if you have more questions. Let us know if you have any further questions before Author-Reviewer Discussion deadline.

Best, Authors.

Reviewer_2T2dOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies, in a generative model of human motion, how to disentangle the motion style and the motion content in the representation space (then the problem converts to a simple latent generation, it can be either diffusion or variational). Given a window of parameterized human motion, the style encoder uses an information bottleneck and a basis factorization to obtain a global latent code. The content encoder also encodes the motion into a sequence of latents. Then, the decoder decodes this content latent with a FiLM insertion of style latent back into the human motion again. For the disentanglement of the content and style, several regularizations are used during the training. By using this decomposition, the paper demos zero shot motion style transfer during inference.

Critical strengths
  • The idea to model the styling in the representation step is clean.
  • the cycle consistency that extract style+content, swap style and decode and re-encode to get the same content code is cute and interesting.
  • the zero shot results shows the decomposition is taking effect and could be useful.
Critical weaknesses
  • Sometimes the style transferred motion may be not plausible anymore, there is no or little such failure cases discussed in the paper.
  • There should be more figures and viusualizations of the generation AND side by side comparison with the baselines since the metrics of human motion generation are corrupted and is not very convincing.
  • Some tables are scaled too small and is hard to read, the method figure needs more polishing.
Questions for authors
  • How sensitive is the decomposition to the tuning of the losses and system, is the disentanglement easy and stable to achieve? or in some case the representation can collapse?
  • Why the author needs to use the orthogonal basis? is this design being ablated and justified?
  • What are failure cases?
3 comments
AuthorsOpenReview ↗

Rebuttal 2T2d (part 1)

Please note relevant parts of Suppl. Video and Paper are highlighted with [Rev. 2T2d]. Following discussion/analyses will be incorporated in paper after author-reviewer discussion.

We thank the reviewer for their encouraging reviews and are glad to know that our idea is clean, loss formulation is interesting and zero-shot results demonstrate the decomposition clearly. We address concerns by the reviewer below:

W1/Q3: Failure Cases

Styles contradicting with content: When a posture style conflicts with the action, it results in an averaged-out reconstruction. An example can be if we stylize a HumanML3D motion with caption “bends at the waist, picks something up” with ArmsaboveHead, as shown in
(Suppl. Video 19:11-19:20). This shows the reconstruction bending and keeping arms above but misses out on the semantic intent of picking something up (shown by the content motion).

Cross-domain posture can sometimes be implausible to achieve e.g. a 100STYLE style code for ArmsaboveHead will not transfer well for a gesture model trained on BEATX, where the posture of gesture is always with arms in a talking position.

We will add the discussion in the paper once you have reviewed the discussion.

Q2: Requirement for Orthogonal Basis

Controllability and style control are one of the major bottlenecks for motion generation. Orthogonal basis gives a compact, low-dimensional coordinate system i.e., a parametric basis, in which style can be sampled, interpolated, and edited along named attributes. This lets us define style with a compact set of coefficients that can (a) regularize and compress style latents and (b) provide a controllable/interpretable space for style control, only requiring a decoder pass for stylization.

Emergent dimensions of control:
New ways and dimensions of control can emerge due to this design (without any explicit training for this control task). We can find new semantic directions of style control in 16-d parametric space, just by fitting simple linear regression probes (Suppl. Video 07:11-09:37). These directions can be used to stylize any downstream generation method with just a single decoder pass. This is a capability beyond the existing architectures and is only possible due to DisCodec’s design.

Ablation: We also ablate the orthogonal basis by removing it and training the method on 100STYLE dataset for 150k iterations. At matched iterations, our default setting compares to the no-basis setting as follows:

CellSeen SRA↑Unseen SRA↑CRA↑FID↓FSR↓Fwd leak style←zcz_c↓Rev leak content←zsz_s↓MPJPE (mm)↓
DisCodec0.4490.2330.3261.220.2450.1900.52635.1
no orthogonal basis0.5390.1880.2801.370.2510.2340.42535.2

We find that having no orthogonal basis moves towards overfitting to seen styles while unseenSRA performance drops. Secondly, content-related metrics also worsen by removing the orthogonal basis.

AuthorsOpenReview ↗

Rebuttal 2T2d (part 2)

Q1: How sensitive and stable is the decomposition?
To measure if disentanglement is easy and stable to achieve, we report additional analysis on the limits of our current architecture:
SNR quantization parameter δδ trades forward leak smoothly (0.348 → 0.193 → 0.064 → 0.054) with MPJPE increasing 32.7 → 35.3 mm.
Higher δδ provides better forward leak (due to stronger rounding in zcz_c) and worse MPJPE. This can be seen in the Suppl. Video 18:34-18:40 where δδ=1.0 (rightmost) shows higher foot sliding. Our default settings balance both style and content.

βsβ_s ∈ {1e-3, 2e-2} (our default: 5e-3): performance stays stable from 1e-3 to 5e-3, but stronger βsβ_s=2e-2 regularizes strongly and makes the style code collapse, shown in the table below (seen SRA: 0.095, unseen SRA: 0.015). Visually, this can be seen in Suppl. Video 18:40-18:46. High βsβ_s loses style information, and reconstructions follow only content motion.

βcβ_c ∈ {3e-3, 3e-2} (our default: βcβ_c 1e-2):
Lower content βcβ_c keeps more detail (MPJPE: 32.0) but leaks style into zcz_c (fwd leak: 0.349) due to less regularization, causing worse SRA; a higher βcβ_c does the reverse (better SRA and leak but has bad reconstruction quality (high MPJPE).
We can see this pattern in Suppl. Video 18:46-18:52, where the result with higher βcβ_c has more foot sliding In this example (Suppl. Video 18:52-18:58), lower βcβ_c has bad style transfer, as it does not lean back, which is similar to the style reference.

K ∈ {4, 8, 32, 64} basis vectors: The conclusions hold for K ≥ 16: transfer is stable or better above 16 (K=64 reaches unseen SRA 0.276, while reducing CRA). This is shown in Suppl. Video 18:58-19:11, where small K=4 also does not follow the lean back style.
Note that all the styles and identities used in visual results are unseen by our method.

Cell @250k iterMPJPE (mm)↓Seen SRA↑Unseen SRA↑CRA↑Fwd leak style←zcz_c↓Rev leak content←zsz_s↓
Default (K 16, δδ 0.25), — ablation table’s "full" row33.50.4790.1840.3740.1930.521
δδ=0.12532.70.4250.1950.2990.3480.491
δδ=0.533.40.4680.1630.2790.0640.564
δδ=1.035.30.4980.2300.3400.0540.631
βsβ_s=1e-332.90.5180.2420.2980.1280.528
βsβ_s=2e-234.90.0950.0150.2170.3570.188
βcβ_c=3e-332.00.3490.1710.3150.3490.395
βcβ_c=3e-236.90.4790.2280.2240.0720.606
K=435.30.1310.0190.3100.3100.234
K=834.50.3380.0780.2840.2540.303
K=3233.00.5440.2390.2900.1850.535
K=6433.60.5590.2760.3130.1720.461

W2 Visual Examples for Generation Tasks: Thanks for pointing it out. Regarding downstream generation, we agree that metrics for motion synthesis can be unreliable therefore, we add more examples for text-to-motion and gesture synthesis in the updated video (Suppl. Video 12:43-17:50). There is already a static figure visualized for the generation task. It was put in the Appendix due to space constraint, but is now moved to the main paper (Fig. 6). Note that all the styles and identities used in visual results are unseen by our method.

W3: Thanks to the reviewer for pointing it out. We will polish the method figure and have enlarged the ablation table since we have more space due to 1 extra page.

Dear Rev. 2T2d,

We wanted to check if the rebuttal has answered your concerns or if you have more questions. Let us know if you have any further questions before Author-Reviewer Discussion deadline.

Best, Authors.

Reviewer_jfSbOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a motion codec that learns separate representations of motion content and style. Such code be used for zero-shot behavioral style transfer. The framework uses two encoding streams with variational information bottlenecks, represents style through a learned low-dimensional basis, and applies SNR-adaptive transform quantization to reduce style leakage.

Critical strengths

Learning separate representations of motion content and style within a codec is a reasonable and interesting approach. Using SNR-adaptive quantization to reduce information leakage between the two representations is quite an interesting design and is informative. Metrics are good and the zero-shot style transfer is also promising. The style representation also provides potentially useful ways to interact with the model.

Critical weaknesses

I found the overall presentation easy to follow and the method sound enough. However, my main concern is the practical controllability of the style representation. Being able to adjust multiple dimensions does not necessarily mean that users can obtain the style they want, making the claims about downstream applications not strong enough. Although reference motion can provide the style, it may be noisy and hard to explain. Considering existing benchmarks, this paper already achieves promising results, but good applications can make this paper better.

Overall, I find the idea interesting, but the experiments specifically visual effects are not yet sufficient to convince me of the downstream tasks. In particular, there are relatively few visual examples with clear comparisons and explanations and I would like to see a few concrete examples. The supplementary video shows that adjusting PCA coefficients changes the motion but still not convincing enough to judge the specifc style.

Questions for authors
  1. Qualitative visual comparison. For instance, the authors could fix the content of a walking motion and interpolate between two reference styles to show what the intermediate results look like, this should be easy to add in ICLR rebuttal.
  2. Hyperparameter sensitivity. Two VIB weights and the SNR-adaptive quantization scale control the trade-off between information compression and motion reconstruction. It would be helpful to report how sensitive the method is to these parameters, particularly in terms of style transfer, content preservation, and reconstruction quality.
  3. Following Q2, Corresponding visual examples, including failure cases under overly strong or weak regularization, would help clarify the method’s robustness and practical operating range.

I will recommand weak acceptance based on the low-level metrics before the requiring esults are provided, but if the corresponding results and experiments are not convicible enough (e.g. still lacking qualitative results, relying heavily on specific parameter tweaking) i'll reconsider the score.

4 comments
AuthorsOpenReview ↗

Rebuttal jfSb (part 1)

Please note relevant parts of Suppl. Video and Paper are highlighted with [Rev. jfSb]. Following discussion/analyses will be incorporated in paper after author-reviewer discussion.

We thank the reviewer for their constructive and encouraging feedback. We are glad to know that the presentation was easy to follow. We also appreciate reviewer’s feedback on style representation and address their concerns below:

W1/Q1: Practical Controllability of Style Representation.
We thank the reviewer for pointing this out, and agree that having practical controllability in the proposed style space matters. To address this, we add the following capabilities to the demo of the style subspace (Suppl. Video 03:23-09:37).

(1) Nearest Style Readout in the Demo: Since stylization and its comprehension are highly subjective tasks, examples shown can be hard to judge, and their differences can become hard to distinctly convey. Therefore, we add a nearest style readout in the demo. As the user moves the 16 coefficient sliders, the demo shows the closest dataset style (seen or held-out) and decodes it on the same content beside the current result, so the user can judge what style the sliders produce. This provides an additional semantically meaningful reference point to judge decoded styles and another practical control dimension, instead of raw coefficients (Suppl. Video 03:23-04:48).

(2) Interpolation between two reference styles with the content fixed (Q1) so users can mix the two desired styles. This can be done for any style, and users can mix any two styles from the dataset in the demo to get the desired style mixture they require. This provides a practical and interactive way for style control. This is also visualized as a demo and standalone videos in Suppl. Video 04:48-07:11.

(3) Named motion Attributes for control: Inspired by the controllable motion synthesis literature (Line 156-161), we perform additional probing analysis on 16 style coefficients and demonstrate that we can find directions of interpretable properties of motion. These properties can include Hand spread(distance between the two hands) and Wrist height (height of both wrists). They can serve as new semantic axes for style control. We add additional sliders for these semantic axes (Suppl. Video 07:11-09:12) and demonstrate the generalizability of our learned style space by editing style along these new dimensions of control (for which our method was not trained at all). This gives users more semantically meaningful directions of control. Also shown as a separate video for directions in (Suppl. Video 09:12-09:37).

Probing Setup: On the default model of DisCodec (100STYLE). We fit a regression probe for each attribute. The probe measures whether the attribute is linearly readable from a clip's 16 style coefficients. A ridge regression from coefficients to an attribute is fitted on the training set and scored on held-out test clips. We show Pearson correlation between ridge prediction and measured attributes on test clips.

Attributer, predicted vs measured
Hand spread0.897
Wrist height0.927
AuthorsOpenReview ↗

Rebuttal jfSb (part 2)

Q2/Q3: Hyperparameter sensitivity Analysis for the two VIB weights and δδ (incl. Visual examples of failure cases)

SNR quantization parameter δδ trades forward leak smoothly (0.348 → 0.193 → 0.064 → 0.054) with MPJPE increasing 32.7 → 35.3 mm.
Higher δδ provides better forward leak (due to stronger rounding in zcz_c) and worse MPJPE. This can be seen in the Suppl. Video 18:34-18:40 where δδ=1.0 (rightmost) shows higher foot sliding. Our default settings balance both style and content.

βsβ_s ∈ {1e-3, 2e-2} (our default: 5e-3): performance stays stable from 1e-3 to 5e-3, but stronger βsβ_s=2e-2 regularizes strongly and makes the style code collapse, shown in the table below (seen SRA: 0.095, unseen SRA: 0.015). Visually, this can be seen in Suppl. Video 18:40-18:46. High βsβ_s loses style information, and reconstructions follow only content motion.

βcβ_c ∈ {3e-3, 3e-2} (our default: βcβ_c 1e-2): Lower content βcβ_c keeps more detail (MPJPE: 32.0) but leaks style into zcz_c (fwd leak: 0.349) due to less regularization, causing worse SRA; a higher βcβ_c does the reverse (better SRA and leak but has bad reconstruction quality (high MPJPE).
We can see this pattern in Suppl. Video 18:46-18:52, where the result with higher βcβ_c has more foot sliding. In the example (Suppl. Video 18:52-18:58), lower βcβ_c has bad style transfer, as it does not lean back, which is similar to the style reference.

K ∈ {4, 8, 32, 64} basis vectors: The conclusions hold for K ≥ 16: transfer is stable or better above 16 (K=64 reaches unseen SRA 0.276, while reducing CRA). This is shown in Suppl. Video 18:58-19:11, where small K=4 also does not follow the lean back style.
Note that all the styles and identities used in visual results are unseen by our method.

Cell @250k iterMPJPE (mm)↓Seen SRA↑Unseen SRA↑CRA↑Fwd leak style←zcz_c↓Rev leak content←zsz_s↓
Default (K 16, δδ 0.25), — ablation table’s "full" row33.50.4790.1840.3740.1930.521
δδ=0.12532.70.4250.1950.2990.3480.491
δδ=0.533.40.4680.1630.2790.0640.564
δδ=1.035.30.4980.2300.3400.0540.631
βsβ_s=1e-332.90.5180.2420.2980.1280.528
βsβ_s=2e-234.90.0950.0150.2170.3570.188
βcβ_c=3e-332.00.3490.1710.3150.3490.395
βcβ_c=3e-236.90.4790.2280.2240.0720.606
K=435.30.1310.0190.3100.3100.234
K=834.50.3380.0780.2840.2540.303
K=3233.00.5440.2390.2900.1850.535
K=6433.60.5590.2760.3130.1720.461

Examples of Failure Cases:

Styles contradicting with content: When a posture style conflicts with the action, it results in an averaged out reconstruction. An example can be if we stylize a HumanML3D motion with caption “bends at the waist, picks something up” with ArmsaboveHead, as shown in Suppl. Video 19:11-19:20. This shows the reconstruction bending and keeping arms above but misses out on the semantic intent of picking something up (shown by the content motion).
Cross-domain posture can sometimes be implausible to achieve e.g., a 100STYLE style code for ArmsaboveHead will not transfer well for a gesture model trained on BEATX, where the posture of gesture is always with arms in a talking position.

We will add the discussion in the paper.

W2: More Visual Comparisons (Text-to-Motion & Speech-to-gesture)
We add dedicated results for text-to-motion stylization & speech-to-gesture personalization (Suppl. Video 12:43-17:50).

For text-to-motion, we compare our method with finetuning-based baselines for style transfer i.e. LoRA-MDM and SMooDi. These methods do not perform zero-shot transfer and finetune on individual style data. Another method we compare against is MoMo, which is a zero-shot transfer method. Due to finetuning, LoRA-MDM & SMooDi perform well in text adherence but our method outperforms them in style-transfer, that too in a zero-shot manner (Suppl. Video 13:46-15:27). MoMo tries to copy the style completely and fails to follow the text.
This quality-vs-transfer comparison shows a pareto frontier (Sec 4.2) where generators trained with DisCodec push the front towards reasonable style-transfer and content adherence. We also like to point outpoint-out that our method only needs decoding for transfer to any style and is very fast (6.6ms on 1CPU thread for 10seconds of motion), compared to LoRA-MDM, which takes ~71 min per style (RTX 3090).

For speech-to-gesture, we compare with methods that learn speech-to-gesture personalization with label-based embeddings. This entails that these methods always learn person-specific traits during training and cannot perform zero-shot transfer. Therefore, our goal is to match the performance of these methods in a zero-shot manner and, hopefully, push the frontier for zero-shot transfer beyond training-based methods through future work.
Note that all the styles and identities used in visual results are unseen by our method.

Dear Rev. jfSb,

We wanted to check if the rebuttal has answered your concerns or if you have more questions. Let us know if you have any further questions before Author-Reviewer Discussion deadline.

Best, Authors.

Reviewer_jfSbOpenReview ↗

Response to Authors

The authors have responded thoroughly to most of my questions with supporting metrics and visual results. I will maintain my rating of 3 (weak acceptance).

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary for Area Chair

We thank the Area Chair and reviewers for conducting the review process that provided us valuable feedback. We are glad to know that reviewers find the paper is well-written [Rev. mAQD] and easy-to-follow [Rev. jfSb] with comprehensive visual demos [Rev. mAQD]. Reviewers agree that the approach of modelling the styling in the motion representation itself is clean[2T2d], interesting[jfSb] and useful for downstream tasks [Rev. WANA].
We briefly explain shared concerns by reviewers below with pointers to individual comments and video timestamps in attached Suppl. Video. Reviewers and Area Chairs are encouraged to watch the new video as we add new visual results and analyses in relation to rebuttal.

Please note relevant parts of Suppl. Video and Paper are highlighted with [Reviwer ID] on top-left. Following discussion/analyses will be incorporated in paper after author-reviewer discussion.

Semantic ways of Style Control.

In relation to style subspace demo (Suppl. Video 03:23-09:37), Rev. jfSb & mAQD point out that the orthogonal style basis is well disentangled but it should have more semantically relevant knobs for users to control style. We additionally demonstrate that the basis can be used to modulate motion attributes like Hand spread or height (Suppl. Video 07:11-09:37). This gives users more semantically meaningful directions of control (without training our model specifically for this type of control) and shows generalizability of our design.

Evidence of direct bi-directional disentanglement

Based on concerns by Rev. vZvn & WANA, we provide a factorial grid across content-clips and unseen styles, to measure whether changing style keeps the action content and changing content keeps the style (See Rebuttal WANA part 1 and Suppl. Video 17:53-18:33). We demonstrate DisCodec is the only method that satisfies ideal disentanglement thresholds, showing bi-directional disentanglement between both style and content. This is unlike baselines that either skew their stylization to content (MoST) or completely follow style (MotionPuzzle/GenMoStyle).

100STYLE (12×10)Style selectivity↑Content selectivity↑
Copy-content control−1+1
Copy-style control+1−1
MoST−0.6+0.7
GenMoStyle (unsupervised)+0.4−0.1
Motion Puzzle+0.3−0.2
DisCodec+0.7+0.2

Comparison with Baselines

Based on concerns by Rev. vZvn and mAQD, we explain that while DisCodec’s architecture shares the goal of zero-shot style transfer with the existing works, it importantly provides one stylization recipe for two important problems together due to its design (which baselines donot address):

  • learning a parametric style space (label-free)
  • performing bi-directional disentanglement, i.e., not only disentangled in the style branch but also in the content branch (unlike GenMoStyle).

Our design also enables controllable and interpretable style control due to its parametric basis (as shown above), which existing methods cannot due to their properties (see table below):

MethodLabelsParametric/Interpretable style spaceGenerator
MoSTYesNoNo
Motion PuzzleNoNoNo
GenMoStyle (unsupervised)NoNo. (only unstructured Gaussian prior)No
DisCodec (ours)NoYes (16-d basis, can also be sampled by the DiT or controlled by interpretable axes)Yes: DiT and ActionPlan generate zcz_c from text, no cascade required

Rev. WANA also pointed out our unseenSRA is lower from MotionPuzzle/GenMoStyle, which is a metric that can be skewed by only keeping style traits. We show visual results (Video 12:09-12:42) and disentanglement analysis (above) and demonstrate that both baselines copy style and miss content-specific actions, while our method balances style and content (See Rebuttal WANA part 2) .

Evidence of style in low-SNR directions to motivate our design

Rev. vZvn and mAQD asked to provide evidence for SNR-adaptive quantization mechanism. Therefore, we provide analysis on measuring whether the style leakage really is in low-SNR directions of content latent. We find that the style sits in the directions that carry almost no variance, the content in the directions that carry almost all of it. The removed directions by default settings hold 0.05% their removal drops leakage from 0.79 to 0.38. This analysis validates our design approach (See Rebuttal vZvn part 2).

Other concerns:

Reviewers requested additional generation results [Rev. 2T2d, mAQD], sensitivity analyses [Rev. WANA, jfSb] and failure cases [Rev. 2T2d].
We provide more generation results for text-to-motion and speech-to-gesture synthesis in Suppl. Video 12:43-17:50.
Sensitivity Analysis is provided in terms of qualitative evaluation in Rebuttal jfSb (part 2) and visual results in Suppl. Video 18:34-19:11.
We discuss failure cases in Rebuttal 2T2d (part 1) and add visual examples to Suppl. Video 19:11-19:20

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.