- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes an unsupervised Transformer autoencoder for motion retargeting between humanoid skeletons with different joint counts and connectivity. Its learnable flattening operation multiplies projected joint features by skeleton-dependent masks generated by GraphSAGE and aggregates them into a fixed-dimensional pose representation. The model represents the root trajectory separately and predicts it in global coordinates. Training combines cycle consistency, adversarial learning, and augmentation consistency without paired retargeting data. Experiments evaluate reconstruction and retargeting on unseen datasets, alongside architectural ablations and a perceptual user study.
Critical strengths
- The model handles unseen humanoid skeletons without paired training data. Although it was not trained on Mixamo motions or skeletons, it outperforms SAME in joint-position, joint-rotation, and root-trajectory reconstruction on all three evaluation datasets (Table 1). For instance, on Bandai-Namco, the joint-position error falls from 12.40 to 2.63 cm. The reported inference cost is also low (Appendix A.4).
- The multiplicative-versus-additive comparison provides useful design evidence. Replacing multiplicative encoding with additive encoding raises intra and cross-structure GJP from 1.45/1.28 to 2.06/2.01 (Table 4). The augmentation and individual-loss ablations further help distinguish the contributions of the architecture and training procedure.
Critical weaknesses
- Table 2 does not provide convincing evidence for the claimed 43–47% improvement under comparable conditions. Appendix A.5 specifies 95 Mixamo clips per character, but the Aberman values in Table 2, 2.76/2.25, exactly match the published SAN results from the original 106-clip protocol. The PALUM values, 2.72/5.67, also match its published height-normalized MSE values multiplied by 1,000, whereas this paper defines GJP using mean Euclidean distance. PALUM additionally averages across different seen/unseen character and motion settings. It is unclear whether these values were recomputed using the current paper’s metric and evaluation subset.
- The paper overstates what the ablations and user study establish. The flattening decomposition motivates multiplicative masking, and Table 4 shows it outperforms the tested additive variant. This is a useful result, but the broader claim that the formulation is what makes Transformers effective for retargeting needs more support. The no-PE and GAT variants have very large errors, with much of the GAT error attributed to root-trajectory prediction, making it difficult to isolate the role of structural encoding. Variability across training runs is not reported either. The abstract also overstates the user-study findings: Table 3 shows no significant alignment improvement over Aberman, and the measured criterion was perceptual quality rather than physical plausibility.
Questions for authors
- Does the multiplicative-versus-additive gap hold across independent training runs? Were the architecture and hyperparameters selected using validation data separate from the Mixamo evaluation set?
- Which evaluated skeleton pairs have different numbers of joints along corresponding chains? For example, how is motion distributed when the target has five spine joints, and the source has two? An example of the resulting rotations and learned masks would help clarify this behavior, including how twist is handled beyond what position-based GJP captures.
- What is known about how Adobe produced the Mixamo ground-truth pairs? I would also appreciate clarification of the practical advantage over conventional retargeters such as MotionBuilder/HumanIK, particularly the required joint correspondences and preparation effort. This asks for context on the intended use case rather than a new benchmark.
2 comments
RE: Variance of multiplicative vs. additive gap across training runs
We ran independent seeds for both our full model and the additive PE variant. The results, including the original run from the paper, are as follows:
| Model | Intra GJP (runs) |
|---|---|
| Additive PE | |
| Ours (multiplicative) |
| Model | Cross GJP (runs) |
|---|---|
| Additive PE | |
| Ours (multiplicative) |
Two of the three new additive PE runs collapsed to poor local minima (GJP ), while all multiplicative runs converged stably. For the collapsed additive runs, we decomposed GJP into local joint position error and root trajectory error, finding that both components are affected (local position error , root trajectory error ). The two additive runs that did converge achieved intra GJP of and (mean ), which on intra alone is comparable to or marginally better than some of our multiplicative runs (mean ). However, on cross-structure retargeting our multiplicative formulation outperforms every individual converged additive run (mean , best and worst ) compared to our multiplicative runs (mean , best , worst ). Together, the multiplicative formulation provides not only better average performance but also greater training stability.
We thank Reviewer aSCr for the thorough review and for highlighting the strong zero-shot generalization results and the value of the multiplicative vs. additive comparison as design evidence. Please find our responses below.
RE: GJP definition and baseline comparability
See general response.
RE: Abstract — "physical plausibility" and user study overstating
See general response.
RE: Local joint position GJP — isolating root-trajectory contribution
See general response.
RE: Variance of multiplicative vs. additive gap across training runs
The training variance across runs has not been reported, and the robustness of the multiplicative vs. additive gap is therefore unknown. Currently the training is still running and will be added afterwards. We additionally note that hyperparameter selection on a validation set separate from the Mixamo evaluation set will be clarified.
RE: Joint-chain distribution example — differing chain lengths across skeletons
We evaluate this directly using a controlled experiment on the 95 motion clips from our standard retargeting evaluation set. We retarget from BigVegas to two target variants: (1) the original BigVegas skeleton with 3 spine joints, and (2) the same skeleton with 2 additional spine joints (5 total). We then compare the spine curves of the two retargeted outputs directly, fitting a cubic spline through each spine normalized by total spine length, and report mean L2 (average point-by-point deviation) and Fréchet distance (worst-case shape deviation), both as fractions of spine length.
The two retargeted outputs differ by less than 1% of total spine length on average, confirming that the model smoothly distributes spinal motion across the additional joints.
RE: Comparison with MotionBuilder / HumanIK — required correspondences and setup effort
HumanIK requires per-character setup that our method avoids. Every source and target skeleton must first be characterized: this means manually mapping its joints onto HumanIK's canonical template (15 mandatory nodes in a fixed hierarchy), posing it in a prescribed T-stance, and entering foot and hand sizes by hand if floor contact is used. Retargeting is then a full-body IK solve driven by reach and pull weights for each body part. By default only the ankles are tightly constrained — this handles foot sliding well, but other contacts (such as a hand on the hip) are lost unless an artist tunes these weights or keys IK effectors by hand.
RE: How was Mixamo dataset created (Retargeting pairs)
The exact method used to produce the Mixamo retargeting pairs is not publicly disclosed by Adobe; the retargeting algorithm is proprietary. We therefore cannot provide details on how the ground-truth pairs were generated, and to our knowledge no publication or documentation from Adobe describes the specific pipeline used.