- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work presents TAME, a flow-matching transformer for text-driven motion editing across humanoid skeletons with different topologies. The method propose both topology-constrained skeletal attention (TCSP) module which restricts skeletal attention to one-hop neighbors; and edit-focused representation alignment (EFRA) which is a self-distilled representation alignment strategy. The authors further build TopoMotionFix, which extends MotionFix to multiple skeleton topologies, and report state-of-the-art results on both seen and unseen topologies benchmarks.
Critical strengths
- The paper is the first to propose text-conditioned motion editing across heterogeneous humanoid skeleton topologies.
- The proposed method achieves state-of-the-art performance on both single toplogy or heterogeneous topologies motion editing benchmarks.
- The ablation studies are comprehensive, and the proposed components show clear improvements.
- An extended heterogeneous humanoid skeleton topologies benchmark/dataset on motionfix is proposed.
Critical weaknesses
-
Dataset/Benchmark quality lack evaluation. In the third unseen-topology example in the supplementary video, both the ground-truth motion and TAME's output show noticeable self-penetration, raising concerns about the quality of the retargeted dataset.
-
Lack of user study. The evaluation mainly relies on retrieval metrics, without human evaluation to assess editing quality, instruction faithfulness, and source motion preservation for comparison with baselines.
-
Unclear unseen-topology difference. It is unclear how different these 9 held-out rigs are from the training skeletons in terms of topology and if they can really reflect generalization to new topology.
-
Figure1 visualization unclear. Some editing effects in Figure 1 are difficult to observe, such as "turn more to the right"(the turn of TAME is unclear and the trajectory shown is a staight line) and "shake both hands"(too clustered to see).
Questions for authors
See Weakness
5 comments
Response to Reviewer qCPP (2/3)
W2: User study (continued)
The 30 sample sets of the unseen-topology study cover 15 skeletons with 2 sample sets each: the SMPL skeleton, the 9 held-out rigs, and 5 Mixamo characters. The 5 Mixamo characters are randomly selected and belong to neither the training rigs nor the held-out rigs, and their results are reported separately. Participants rate each candidate on a 3-point scale, and the candidates are shown in random order. The new criterion, source preservation, asks how well the edited motion keeps the parts of the source motion that the instruction does not ask to change (1 = unedited parts are largely changed, 2 = slightly changed, 3 = almost unchanged). Appendix F also explains how this criterion differs from the generated-to-source retrieval in Table 1. Each cell below is the mean over 1200 ratings (Seen), 800 ratings (Unseen), or 400 ratings (Mixamo), with the standard deviation after the ± sign.
| Protocol | Method | Alignment ↑ | Plausibility ↑ | Source Preservation ↑ |
|---|---|---|---|---|
| Seen | GT | 2.65 ±0.52 | 2.61 ±0.57 | 2.93 ±0.27 |
| SimMotionEdit | 2.28 ±0.75 | 2.18 ±0.62 | 2.19 ±0.39 | |
| OmniME | 2.40 ±0.74 | 2.29 ±0.72 | 2.26 ±0.51 | |
| TAME w/o EFRA | 2.36 ±0.75 | 2.24 ±0.69 | 2.25 ±0.50 | |
| TAME (ours) | 2.53 ±0.64 | 2.33 ±0.67 | 2.29 ±0.54 | |
| Unseen | GT | 2.62 ±0.55 | 2.64 ±0.51 | 2.95 ±0.23 |
| AnyTop | 2.20 ±0.72 | 2.12 ±0.69 | 2.16 ±0.47 | |
| TAME w/o TCSP, w/o EFRA | 2.12 ±0.70 | 1.91 ±0.70 | 1.90 ±0.43 | |
| TAME w/o EFRA | 2.28 ±0.59 | 2.14 ±0.70 | 2.19 ±0.56 | |
| TAME (ours) | 2.31 ±0.59 | 2.19 ±0.69 | 2.21 ±0.60 | |
| Mixamo | GT | 2.60 ±0.53 | 2.65 ±0.46 | 2.94 ±0.24 |
| AnyTop | 2.23 ±0.65 | 2.08 ±0.66 | 2.13 ±0.57 | |
| TAME w/o TCSP, w/o EFRA | 2.09 ±0.74 | 1.96 ±0.65 | 1.90 ±0.67 | |
| TAME w/o EFRA | 2.29 ±0.68 | 2.10 ±0.69 | 2.16 ±0.58 | |
| TAME (ours) | 2.30 ±0.69 | 2.14 ±0.70 | 2.20 ±0.63 |
TAME receives the highest ratings among the methods on all three criteria in all three blocks. Under the seen-topology protocol, it is rated above SimMotionEdit (alignment 2.53 vs 2.28, plausibility 2.33 vs 2.18, source preservation 2.29 vs 2.19) and above OmniME (2.53 vs 2.40, 2.33 vs 2.29, 2.29 vs 2.26). Under the unseen-topology protocol, it is rated above AnyTop (2.31 vs 2.20, 2.19 vs 2.12, 2.21 vs 2.16), and the same ordering holds on the Mixamo characters (2.30 vs 2.23, 2.14 vs 2.08, 2.20 vs 2.13). The two ablated variants show that participants also perceive the effect of each component. Removing EFRA lowers alignment from 2.53 to 2.36 under the seen-topology protocol, and further removing TCSP lowers plausibility from 2.14 to 1.91 and source preservation from 2.19 to 1.90 on the unseen skeletons.
The three criteria correspond to the three aspects that the reviewer lists. Alignment measures instruction faithfulness, plausibility measures editing quality, and source preservation measures how well the source motion is preserved.
Response to Reviewer qCPP (3/3)
W3: Difference between the held-out rigs and the training skeletons
We appreciate this insightful point. The 9 held-out rigs are motion-capture rigs that appear only in the test set, with their original topology and bone lengths. To enlarge the difference between the held-out rigs and the training skeletons, we apply skeleton augmentation to the 83 training rigs (Algorithm 1, Appendix A). The augmentation randomly adds or removes joints along the spine and the neck, inserts dummy joints at the hip, the lower back, and the shoulder, and rescales the bone lengths and the height. The training skeletons are therefore randomised variants whose joint counts and topologies are changed, and the held-out rigs are excluded from the character pool that the augmentation samples from (Appendix B). By design, the held-out rigs thus differ from the training skeletons. To show how the held-out rigs look, we have added Figure 5 in Appendix B, which visualises the SMPL skeleton and the 9 held-out rigs in their rest poses at a common scale. We also acknowledge that this difference is not as large as the difference across species. Our work targets humanoid skeletons, and the unseen-topology protocol evaluates generalisation within this scope.
To further examine generalisation beyond the held-out rigs, the user study in Appendix F (W2 above) includes 5 randomly selected Mixamo characters. These characters belong to neither the training rigs nor the held-out rigs. The ratings of TAME on these characters are close to its ratings on the SMPL skeleton and the held-out rigs (alignment 2.30 vs 2.31, plausibility 2.14 vs 2.19, source preservation 2.20 vs 2.21), and TAME is rated above AnyTop on all three criteria. Figure 4 also shows qualitative results on Mixamo skeletons.
W4: Visualisation in Figure 1
We thank the reviewer for pointing this out. We have re-rendered both examples in Figure 1. For "shake both hands", the frames were drawn on top of each other. They are now spread out along the trajectory, so that the hands can be seen in every frame. For "turn more to the right", we render the motion from a different viewpoint and space the frames apart, and the trajectory arrow now follows the path of the character instead of a straight line, so that the turn of the target motion can be seen.
Response to Reviewer qCPP (1/3)
We sincerely thank the reviewer for the thoughtful and constructive review and for recognising TAME as the first work on text-driven motion editing across heterogeneous humanoid skeletons, together with the ablation studies and the TopoMotionFix benchmark. We answer each point below.
W1: Self-penetration in the third unseen-topology example
We appreciate the reviewer's careful look at the supplementary video. We would like to clarify that the self-penetration comes from the heavy armour of the character that we use for display, rather than from the quality of the TopoMotionFix dataset. We examined the third unseen-topology example frame by frame and detected the frames in which the skin surfaces of two body parts intersect. The self-penetration that the reviewer notes does exist in both the ground-truth motion and the output of TAME. It occurs where the forearm guard cuts into the chest plate and where the shoulder plate reaches the helmet. This armour is specific to the character. The character is a heavily armoured soldier whose armour is large and thick. Its bone lengths are close to those of a slim character, but its surface lies 1.4 to 2.3 times further from the bones. A character of this kind is therefore prone to self-penetration once it is rendered.
TopoMotionFix is built by retargeting the motions of MotionFix onto other skeletons, and the retargeting uses the joints only. The vertices of a mesh, such as those of clothes, hats, or armour, take no part in it, and the dataset contains skeletons and joint motions only (Sec. 4.1, Appendix A). The motions were captured on human bodies, and the heavy armour in effect enlarges the body of this character, which leads to the penetration.
In Table 3 (Appendix D), the row GT (retargeted) reports the effect of the retargeting on the retrieval metrics. Table 3 follows the MotionFix evaluation on the full test set, where each motion is ranked among all test motions rather than within a batch of 32 as in Table 1. The absolute values are therefore lower than in Table 1, and what matters here is the difference between the two rows. We copy this row and the GT row below.
Retargeting the MotionFix test targets onto the held-out rigs and back to SMPL has a negligible effect on the retrieval metrics.
W2: User study
We thank the reviewer for this suggestion, and we agree that human evaluation is necessary for motion editing. The submission contained a user study in Appendix F, titled "Perceptual Study", but the experiments section did not refer to it. Following the reviewer's suggestion, we have conducted the study again at a larger scale, with more candidates, and with source motion preservation as a third criterion. We rename the appendix to "User Study" and refer to it in Sec. 4.2. The setting changes as follows.