- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper argues that contrastive pretraining optimizes text encoders for cross-modal alignment, a proxy objective that does not directly optimize their usefulness for motion synthesis. To address this objective mismatch, LeGO jointly trains a motion diffusion generator and adapts a pretrained text encoder through LoRA. Frozen text and motion projectors provide semantic consistency losses alongside motion reconstruction. The resulting LeGO-CLIP encoder can also be transferred to other motion generators. Experiments on HumanML3D, KIT-ML, and SnapMoGen evaluate generation performance, encoder transferability, and sensitivity to fine-grained instructions.
Critical strengths
-
Well-motivated text encoder adaptation: The paper identifies a plausible mismatch between contrastive pretraining and downstream motion generation. Jointly adapting the text encoder with the motion generator is a reasonable approach to learning representations suited to the generation task.
-
Comprehensive empirical evaluation: Experiments cover three datasets and demonstrate encoder transferability across different generation architectures. Physical controllability tests and a user study complement embedding-based metrics, providing broader evidence of improved motion quality and instruction following.
Critical weaknesses
-
Insufficient positioning and comparison with related work: MATE [1] also improves text representations for fine-grained motion generation, but its relationship to LeGO is not discussed in the Introduction or Related Work. Although it appears in the benchmark comparisons, it is omitted from the semantic sensitivity analysis in Section 4.3. Clarifying the methodological differences and including this comparison would better establish LeGO’s contribution.
-
Reliance on contrastively pretrained projectors: The paper argues that contrastive learning optimizes cross-modal alignment but may fail to capture subtle dynamic distinctions needed for motion generation. However, its consistency losses use contrastively pretrained text and motion projectors to guide joint training. These projectors may share the limitations that motivate the paper. The authors should explain why they provide suitable supervision for fine-grained dynamics and how joint training overcomes those limitations. Furthermore, although the paper addresses a meaningful problem, its methodological contribution seems somewhat incremental.
[1] Motion-Aligned Word Embeddings for Text-to-Motion Generation. ICLR, 2026.
Questions for authors
-
Suitability of the supervision projectors: Why are the contrastively pretrained projectors suitable for supervising subtle dynamic distinctions, given the paper’s concern that contrastive learning may align text and motion at a coarse semantic level but overlook fine-grained motion details?
-
Semantic sensitivity comparison: How does LeGO-CLIP compare with MATE [1] under the keyword-substitution protocol in Section 4.3?
-
Loss formulation and performance claims: Minimizing Eq 2 as written would discourage matched-pair alignment. Is this a typesetting error? Also, does the state-of-the-art FID claim in Section 4.1 refer only to diffusion-based methods?
4 comments
Thank you for adding the MATE comparison under the keyword-substitution protocol and clarifying how LeGO differs from MATE. The ablation that blocks reconstruction gradients to the text encoder addresses my main concern about the role of the projectors. I also appreciate the correction to Eq. (2) and the more precise FID claim in Section 4.1.
I have one remaining clarification. For the blocked-gradient ablation, the response text reports an FID of 0.053, while both the response table and Table 5 of the revised manuscript report 0.088 ± 0.003. Could you confirm which value is correct? This discrepancy does not change my interpretation of the ablation.
We thank the reviewer for carefully pointing out this inconsistency.
We confirm that the correct FID is 0.088 ± 0.003. This result was already reported in the revised manuscript uploaded five days ago, specifically in Table 5 under the ablation setting "Block MSE gradients to CLIP." The value of 0.053 in our previous response text was incorrect, while both the response table and the revised manuscript report the correct result.
We apologize for this inconsistency and appreciate the reviewer's careful reading.
We sincerely thank you for your constructive feedback and for acknowledging that your concerns have been resolved.
Given that your concerns have now been addressed, we would be very grateful if you could reconsider your overall rating in light of our responses and additional experimental results.
We greatly appreciate your time and thoughtful consideration.
[Q1]: Comparison with MATE on Semantic Sensitivity Analysis
[Q2]: Suitability of Using Contrastively Pretrained Projectors
[Q3]: Discussion of MATE in Related Work
[Q4]: Clarification of FID and R-Precision Claims
[Q5]: Correction of the Projector Training Objective
=-\frac{1}{2N}\sum_{i=1}^{N} \left( \log\frac{\exp(S_{ii}/\tau)}{\sum_j\exp(S_{ij}/\tau)} + \log\frac{\exp(S_{ii}/\tau)}{\sum_j\exp(S_{ji}/\tau)} \right) +\frac{1}{N}\sum_{i=1}^{N}(1-S_{ii}).$$