- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
- The paper addresses a practically relevant problem, and the experiments show encouraging results. Motion transfer requires not only motion fidelity but also flexibility to change the subject and scene. Table 1 on page 9 shows that CMC improves motion fidelity over the base model and achieves the highest diversity score among the evaluated motion customization methods. These results indicate that the method can learn reference motion while retaining substantial variation in the generated outputs. Table S4(a) in the appendix shows that reducing the cutoff step for motion constraints from 28 to 10 decreases training time from approximately 945 to 349 minutes and reduces GPU memory usage, without worsening the motion metric in this experiment. This is a practical improvement over the method’s own version that uses the full denoising trajectory for training.
Critical strengths
- Motion transfer requires not only motion fidelity but also flexibility to change the subject and scene. Table 1 on page 9 shows that CMC improves motion fidelity over the base model and achieves the highest diversity score among the evaluated motion customization methods. These results indicate that the method can learn reference motion while retaining substantial variation in the generated outputs.
- Table S4(a) in the appendix shows that reducing the cutoff step for motion constraints from 28 to 10 decreases training time from approximately 945 to 349 minutes and reduces GPU memory usage, without worsening the motion metric in this experiment. This is a practical improvement over the method’s own version that uses the full denoising trajectory for training.
Critical weaknesses
- In Table 1, CMC has a motion distance of 0.1093, compared with 0.0781 for MotionDirector, where lower is better. Thus, although CMC produces more diverse outputs, its motion is less faithful to the reference than that of some baselines. Could the proposed method simply be relaxing the motion constraints, thereby allowing greater visual variation? A comparison at similar levels of motion fidelity is needed to determine which method copies less of the reference appearance and background.
- Moreover, the text fidelity metric mainly checks whether the target object appears, while the diversity metric measures pixel-level differences across generated videos. Neither directly indicates whether the reference background or appearance is retained. From a theoretical standpoint, “remaining close to the base model” does not imply “not copying reference content.”
- Equation (20) in the appendix uses a hard argmax to select matching positions and then computes the motion loss from positional differences. However, the training procedure in Eq. (23) requires gradients of this loss with respect to the input. As currently formulated, the discrete position selection cannot provide useful motion gradients through standard backpropagation. The authors should clarify which differentiable approximation or gradient estimator is used in practice. This affects whether the method can be reproduced as described, rather than being merely an issue of mathematical presentation.
Questions for authors
At similar levels of motion fidelity, does CMC still copy less of the reference appearance and background? Could the authors provide a comparison that directly evaluates content leakage, rather than reporting diversity alone?
How is the argmax in Eq. (20) handled during training? Please provide the loss formulation and gradient computation used in the actual implementation.
3 comments
W2 (cont.). Closeness to the base model and copying
To show more clearly why staying close to the base model prevents copying, we make the link explicit. The key is that CMC stays close to the prompt-conditional base distribution , and copying the reference requires appearance or background that the prompt does not describe. Let be the set of such videos and the accumulated motion cost, and assume that (a) for a small , i.e., the prompt alone rarely produces the reference-specific content (e.g., "a panda turning its head" is very unlikely to produce the background of a specific wolf video), and (b) is non-negligible, i.e., the desired motion is reachable from every initial noise. Since , Eq. (7) gives for every , hence CMC can therefore raise the probability of copied content by at most a factor of . When the reference and the prompt share content, the bound weakens, but the shared content is not leakage either. We do not estimate , since it is a worst case over all initial noises; instead, the direct measurements in W1 & W2 show that this amplification does not occur in practice. We have revised Sec. 4.3 accordingly (Properties 1–2) and added the proof to App. B. (Revised PDF: Sec. 4.3 Properties 1–2; proof in App. B)
W3, Q2. Handling of the argmax in Eq. (20) (Eq. (22) in the revised PDF)
The hard argmax in Eq. (22) is used only for the reference: the reference motion field is a fixed target and requires no gradient. For the generated video, whose motion field enters the loss and must be differentiated, we replace the argmax with a soft-argmax over the same local window ( tokens, Table S1): where is the cosine similarity of Eq. (22) between the block-15 features of token in frame and token in frame , is the position of token , and is a temperature. The displacement replaces the argmax displacement in the motion loss (Eq. (23)). The reference field is computed once with the hard argmax and detached, so the gradient flows only through the generated video: from the loss through the softmax weights and the similarities to the block-15 features of the generated latent, and from there to the input , as required by Eq. (25). As , the soft-argmax recovers the argmax of Eq. (22) when the maximum is unique. We use 0.25 (Table S1), and we have added this formulation to App. C.2. (Revised PDF: App. C.2, Table S1)
We hope these clarifications and additional experiments address the reviewer's concerns.
We sincerely thank the reviewer for the thoughtful discussion and for raising the score. Your comments have helped us strengthen our paper.
We thank the reviewer for the constructive feedback, in particular on the comparison at similar levels of motion fidelity, the direct measurement of content leakage, and the implementation of Eq. (22). All revisions are shown in blue in the revised PDF, and those addressing your comments are marked with LY4k in orange.
W1, Q1 & W2. Direct leakage metrics, and comparison at similar levels of motion fidelity
(1) [W2, Q1] Direct leakage metrics. We thank the reviewer for this point, and we agree that neither text fidelity nor diversity directly measures whether the reference background or appearance is retained. We therefore add direct leakage metrics and show that CMC achieves the lowest subject leakage, the highest subject-centric diversity, and background leakage comparable to that of the backbone.
Specifically, we evaluate the 7 object-motion references of Table 1 (105 videos per method) with three metrics: subject leakage (DINOv2 similarity between the subject crops of the generated and reference videos), background leakage (CLIP similarity between the backgrounds after removing the subject with SAM), and subject-centric diversity on Grounding DINO crops; the 3 camera-motion references contain no subject and are excluded.
Table A. Direct measures of content leakage (CogVideoX-2B, 105 videos per method).
The tuning-based methods (MotionDirector, MotionInversion, DeT) show the highest leakage in both the subject and the background, together with the lowest subject-centric diversity. These results support our content-leakage claim more rigorously: CMC enables diverse motion customization by mitigating content leakage.
These metrics are feature-similarity proxies for retained appearance and background rather than complete measures of copying, which is why we report them relative to the backbone for the same prompts. (Revised PDF: Sec. 5.1 Evaluation Protocol, Table 1, Sec. 5.2 Quantitative Comparison; definitions in App. D.3)
(2) [W1, Q1] Comparison at matched motion fidelity. Following the suggestion, we compared content leakage at matched motion fidelity, and CMC copies less of the reference than all three tuning-based methods, which have a lower motion distance than CMC in Table 1 (Table B). For each (reference, prompt), we match the 5 baseline videos 1:1 to the 5 CMC videos with an optimal assignment (Hungarian algorithm) that maximizes the number of pairs whose per-video motion distance differs by at most 0.01 and, among those, minimizes the total difference; videos without a partner are excluded, which yields 43–46 pairs out of 105.
Table B. Content leakage at matched motion fidelity (pairs within the same reference and prompt, |ΔMF| ≤ 0.01).
The difference is large for MotionDirector and MotionInversion, and the 95% bootstrap intervals of the difference (CMC − baseline, resampled over references) lie below zero for all three methods in both subject and background leakage. The lower motion distance of the tuning-based methods in Table 1 therefore comes with copying the reference, i.e., the overfitting we described, now measured over the whole benchmark. (Revised PDF: App. E.1 "Content Leakage at Matched Motion Fidelity", Table S5)
(3) [Q1] CMC does not relax the motion constraint. Tables A and B show that the diversity of CMC does not come from following the motion less closely: at the same motion distance, CMC still copies less. CMC nevertheless has a higher average motion distance than the tuning-based methods because MF measures how closely a video reproduces the reference trajectories, including their spatial extent, whereas CMC transfers the motion to a different layout. On the examples of Fig. S2, the motion distance between the reference and baseline outputs with content leakage (0.0090) is lower than that between the reference and CMC (0.0200), while unrelated videos with different motion score far worse (0.3250); copying the reference layout thus lowers the distance. The videos are shown in the Rebuttal section Motion fidelity metric.