acceptodds
Under review as a conference paper at ICLR 2027

Motion Retargeting Based on Inter-Character 3D Gaussian Mapping Pairs

Abstract

Motion retargeting aims to model the correlation between source and target characters and transfer the source motion to the target character. Conventional deep learning methods typically take the source and target meshes as a condition to directly generate target motion. These methods require large numbers of training samples, and their retargeting performance drops substantially when the two characters differ greatly in skeleton or morphology. To address these problems, we propose a motion retargeting method based on inter-character 3D Gaussian Mapping Pairs (GMPs). Our method transfers motion through the body-part primitives (BPPs) of the source and target characters. GMPs represent the mapping between their BPPs and enable source-to-target motion retargeting without requiring large numbers of training samples. We design an evolutionary framework consisting of GMP-Selection and GMP-Update to optimize GMPs. GMP-Selection uses a fitness function based on morphological and homeomorphic constraints to stabilize the search for GMPs. GMP-Update uses the same constraints to guide Gaussian adjustment, cloning, and splitting to accelerate GMP refinement. We also design a "Gaussian Binding”–"Motion Correction” strategy. It ensures motion fidelity, prevents interpenetration, and maintains continuity during motion retargeting by using tracking, collision, and smoothness constraints. Our method provides an interpretable inter-character representation for motion retargeting and supports characters with large differences. Extensive experiments on the public Mixamo dataset show that our method delivers leading performance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Motion Retargeting, 3D Gaussian Representation, Optimization-based Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_1KvAOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents an optimization-based motion retargeting framework that establishes source–target correspondences through 3D Gaussian Mapping Pairs (GMPs) representing body-part primitives. Its main contributions are a Gaussian-based correspondence representation, an evolutionary procedure for selecting and refining GMPs under morphological and neighborhood-preservation constraints, and a motion correction stage that optimizes target joint rotations using tracking, collision, and temporal smoothness objectives. Evaluation on 18 Mixamo characters includes quantitative baseline comparisons, qualitative examples, and ablations of GMP selection, motion constraints, and Gaussian counts. The results show reduced interpenetration and contact error relative to the compared methods, with trade-offs in joint-position accuracy and computational cost. A 100-participant user study is also described, although its numerical results are missing.

Critical strengths
  1. Explicit, interpretable geometric correspondence. Gaussian Mapping Pairs provide a concrete representation of source–target body-part correspondences that can accommodate differences in body proportions and support both motion tracking and collision handling. This is a meaningful design contribution because it connects geometric correspondence estimation directly to the retargeting objective without requiring large-scale motion training data.
Critical weaknesses
  1. The substantive novelty of the Gaussian-based correspondence formulation is insufficiently established. The framework largely follows optimization-based retargeting, with Gaussian Mapping Pairs serving as its central representation change. However, the paper does not clearly establish what these correspondences enable beyond prior geometry-aware representations, such as interaction meshes, surface context graphs, or other body-part proxies. The reported ablations assess GMP selection and motion constraints, but do not isolate the benefit of Gaussian correspondences against alternative representations under comparable optimization objectives. Consequently, it remains unclear whether the contribution constitutes a meaningful methodological advance or primarily a different geometric parameterization.

  2. The evidence for perceptual and temporal motion quality is incomplete. The provided material shows static examples but does not include accessible motion videos, making it difficult to assess temporal coherence, motion fidelity, and contact behavior throughout a sequence. Although the paper reports a 100-participant user study and claims higher ratings than the baselines, it does not present the numerical ratings or their variability in the provided manuscript, or the appendix. These gaps are consequential because the quantitative improvements in penetration and contact error are modest and do not consistently extend to joint-position accuracy; stronger temporal and perceptual evidence is therefore needed to establish an overall improvement in retargeting quality.

Questions for authors
  1. Clarification of methodological novelty: What substantive capability do Gaussian Mapping Pairs provide beyond existing geometry-aware correspondences or body-part proxies? Can you identify which aspects of the formulation specifically depend on the Gaussian representation, and provide evidence that its benefits persist when compared with alternative representations under comparable optimization objectives?

  2. Clarification of temporal evaluation: Are videos of the retargeted motion sequences available, including comparisons with the strongest baselines on challenging contact-sensitive motions? Such evidence would help assess whether the reported geometric improvements translate into temporally coherent motion without introducing jitter or distorting the source motion.

  3. Clarification of user-study results: Could you provide the numerical results of the reported 100-participant study, including per-method ratings, uncertainty estimates, and the methods and clips evaluated? These results would materially affect my assessment of whether the quantitative gains correspond to a meaningful perceptual improvement.

Reviewer_M4kaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper builds paired anisotropic Gaussians from rigged source and target T-pose characters, then optimizes a target rotation sequence against Gaussian-displacement tracking, proxy collision, and smoothness terms. The paired construction is an interesting way to connect character geometry and motion correction. Table 1 reports the lowest penetration rate (Pen, 4.113%) and contact error (Con, 0.835) among the evaluated retargeters. I am not yet convinced by the experimental evidence for the quality of the resulting motions or by the attribution of the gains to the proposed construction losses.

Critical strengths
  • S1. The explicit Gaussian Mapping Pair (GMP) representation provides an interesting link between character geometry and per-sequence motion retargeting. It allows geometry-aware motion constraints without training a cross-character retargeting network for this method.
  • S2. The reported collision and contact measures are promising. Table 1 reports the best Pen and Con among the tested retargeters; Table 3 also makes the fidelity/collision trade-off visible rather than claiming that every measure improves at once.
Critical weaknesses
  • W1 — The paper does not yet provide enough evidence to judge the full retargeted motions. The penetration results alone do not show whether motions look natural, keep foot contact, and stay smooth over time.

    • Section 4.4 describes a user study, but supplement C/Table 5 gives its setup without ratings for each method. The submission also does not include a motion video demo.
    • This evidence gap does not show that unnatural poses, foot sliding, or jitter occur.
    • It would help to provide full-speed source, target, and baseline clips, method-level ratings, and contact and motion-smoothness measurements.
    • Comparing the same starting motion before correction, after simple collision correction, and after the full method would show what each step changes.
  • W2 — The paper does not show how each GMP construction loss affects the final motion. I cannot tell which construction losses improve retargeting from the current results.

    • Table 3 changes motion-stage losses, not the losses used to build GMPs.
    • It would be more convincing to remove construction losses one at a time, especially context and orientation, and report both the final motions and the quality of the Gaussian matches.
    • Keeping character pairs, initial Gaussian pairs, construction steps, and the motion solver the same would make these comparisons easier to interpret.
  • W3 — It is not yet clear why the matching losses are enough for characters with different body-part shapes. The paper has not shown why its direction-only matching is enough when limb length and width differ.

    • Eq. (5) compares directions between Gaussian centers, but not their distances or the orientations of paired Gaussians.
    • Eq. (4) checks whether an outside Gaussian becomes closer than an original neighbor. Could the authors give a simple example of how this loss works?
    • Could the authors explain why directions alone are enough? Testing a distance constraint adjusted for body-part size or a Gaussian-orientation constraint under the same motion solver would make the answer more convincing.
  • W4 — It is not yet clear whether tracking Gaussian motion keeps the intended pose and foot contact. I cannot tell whether matching Gaussian displacements preserves joint rotations and contact for characters with different proportions.

    • Eq. (8) defines each Gaussian's normalized displacement uicu_i^c by subtracting its own T-pose center and dividing by its character's body-part scale. Eq. (9) compares the displacements of each pair.
    • Eq. (12) adds collision and smoothing, but no term directly compares source and target joint rotations or keeps a foot in place during contact. This does not show that the resulting poses are wrong.
    • Could the authors explain how the body-part scale and coordinate axes are chosen? Joint-rotation and foot-contact results for characters with different limb lengths would make the case for this loss more convincing.
    • A rotation or contact control under the same solver could show what the current loss captures.
  • W5 — The paper does not yet show what paired Gaussians add over older geometry-based optimization methods. Earlier methods also use simple geometric body models to adjust poses and reduce collisions, so I cannot yet tell what GMP pairing adds.

    • SMPLify uses capsules, spheres, and Gaussian collision penalties to fit a body pose and shape to one image. It is a useful precedent, but not a motion-retargeting baseline.
    • Molla et al.'s Egocentric Mapping of Body Surface Constraints uses capsules and body-surface constraints to adapt motion, though its character setup differs. Table 1 does not provide a comparable test of the benefit of GMP pairing.
    • It would help to cite SMPLify and explain what the paired Gaussians add beyond these earlier ideas. A comparison with a traditional retargeting method would be useful where the settings allow.
    • If a direct comparison is not possible, it would help to test a simpler shape proxy using the same starting motion and solver, and report motion quality, Pen/Con, and total time.

Minor Issues

Experimental Setup and Evaluation

  • Could the authors report the number of character pairs/clips and how valid frames and metrics are aggregated across them (§4.1; Table 1)?
  • Results on how part-label errors or boundary perturbations affect final retargeting quality would make the evaluation more convincing (§4.1; Eqs. (8)–(10)).
  • Please report complete per-sequence optimization seconds per output frame and clarify whether Table 4's FPS includes the full motion solver (Table 4).
  • Could the authors clarify whether T-pose GMP construction is reused across clips and how its cost is attributed to each retargeted sequence (Lines 311–312; Tables 2 and 4)?
  • A limited tracking/collision weight sweep would help interpret the fidelity/penetration trade-off in Table 3.
  • Table 1 reports a higher MSElc for GMP (0.061) than for STaR (0.058), despite lower Pen/Con. It would help to note this narrower result.

Method

  • Could the authors explain how the initial source/target GMP pairs are formed, their counts aligned, and whether reassignment occurs (§3.1–§3.3)?
  • Could the authors specify how the shared reference-neighbor set used by Eq. (4) is formed and updated after paired clone/split operations (§3.2; §3.3.2)?
  • Brief pseudocode for the shared GMP-selection mask and candidate fitness/update order would make §3.3.1/Figure 5 easier to follow.
  • Please state the paired clone/split offspring counts and any GMP-count cap or stopping limit (§3.3.2); the ranges in Table 4 do not establish a hard limit.
  • If construction-stage Gaussian contribution weights can approach zero, could the authors clarify what prevents the weighted geometry terms in Eqs. (1)–(3) from decreasing without improved coverage? This is a conditional concern, not an observed failure.
  • Could the authors clarify whether Eq. (6) allows one-pair GMP-selection masks or scores candidates after updating them? Under a fixed-state, mask-independent reading, a one-pair choice may score best; this has not been verified in the implementation.

Method Inputs

  • Please distinguish the rigged T-pose inputs for GMP construction from the source rotation sequence used in motion optimization (§3.4, Lines 311–312); the former does not appear to consume the source motion sequence.
  • Could the authors specify the initial target rotation sequence and whether it relies on another retargeter before per-sequence optimization (§3.4; supplement B.3)?
  • Could the authors explain how §4.1's body-part labels are assigned to (mesh) vertices and Gaussian primitives?

Definitions

  • Please define the finite contour meant by Gaussian “support” and the formula for the directional effective radius ric(u)r_i^c(u) in Eq. (1).
  • Please define the (mesh) inside/outside distance convention used for the nearest-surface term in Eq. (1).
  • Could the authors explain why Eq. (3) penalizes potential Gaussian overlap only along the center-to-center direction, and what off-axis overlap this proxy might not capture?
  • Could the authors clarify whether “local accumulation” means construction-stage Gaussian crowding or overlap rather than posed-body collision (L187)?

Writing and References

  • It would help to cite relevant earlier skeletal Gaussian binding work and distinguish it from inter-character pairing (§3.4).
  • It may be clearer to name the two stages “character modeling” and “per-sequence motion retargeting” (Lines 311–312).
Questions for authors

Please refer to previous sections.

Reviewer_HXXVOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents a geometry-aware motion retargeting approach designed to transfer skeletal motion between characters with significant morphological and skeletal differences. Conventional deep learning frameworks often struggle with large morphological variations, producing artifacts such as hand-body interpenetration or trajectory distortions. To tackle this problem, the authors propose modeling inter-character relationships using Gaussian Mapping Pairs (GMPs). Source and target characters' body-part primitives (BPPs) are represented using anisotropic 3D Gaussians initialized on T-pose meshes. The whole framework include three parts: 1. gaussian pairs association: the proposed method use GMP-Selection and GMP-Update with morphological and homeomorphic constraints to construction the gaussian ball pair association between source and target characters; 2. GMP-based motion retargeting: gaussian binding and motion correction is proposed. During motion transfer, each character's Gaussians are bound to their corresponding skeleton using linear blend skinning (LBS). Target joint rotations are optimized across frame sequences under tracking, collision, and smoothness constraints. The paper claims that evaluated on 2,752 motion clips from 18 Mixamo characters, the proposed method outperforms existing baseline methods by yielding lower limb interpenetration (Pen = 4.113%) and lower contact distance error (Con = 0.835) among compared retargeting algorithms.

Critical strengths
  1. The proposed method is well-organized and well-written.
  2. The proposed gaussian-mapping-pairs is novel and interesting. It is a suitable solution for the motion retargeting task.
  3. The experiment part is comprehensive. The proposed method achieves the state-of-the-art performance under the penetration and contact metrics. Table 4 provides an insightful quantitative analysis balancing construction time, rendering/processing FPS, error metrics, and penetration rates across different Gaussian resolution ranges (100–500, 500–2,000, and 2,000–5,000 GMPs)
Critical weaknesses
  1. Compared to feed-forward neural networks, the proposed GMP-based method is computationally slower due to its two-stage optimization paradigm. First, constructing GMPs in the T-pose via the evolutionary framework and gradient descent requires up to 20,000 iterations. On an NVIDIA RTX 5090, building 500–2,000 GMPs for a character pair takes 7,532 seconds (approx. 2.1 hours), though GMP-Selection already reduces this time by 20.9% (down from 9,520 seconds). Fortunately, this is a one-time preprocessing cost per character pair, and the optimized GMPs can be reused across arbitrary motions. Second, during motion retargeting, iteratively optimizing target joint rotations over all frames under tracking, collision, and smoothness constraints yields 13.7 FPS for 500–2,000 GMPs (ranging from 6.3 to 17.8 FPS across different GMP counts). Despite lower runtime efficiency, this trade-off eliminates the need for large training datasets, generalizes well to extreme morphological differences, and significantly reduces interpenetration.
  2. While the method performs best on geometry-focused metrics (Pen and Con), its joint positional errors (MSE = 0.197, MSElc = 0.061) are higher than several baselines, such as direct Kinematic Copy (MSE = 0.068, MSElc = 0.045), SAN (MSE = 0.069, MSElc = 0.045), and R2ET (MSE = 0.085, MSElc = 0.047) (Section 4.2, Table 1). The authors should explicitly analyze this trade-off in Section 4.2. They should explain why strict joint rotation/position tracking must be relaxed in order to satisfy non-penetration (Ecol\mathcal{E}_{\text{col}}) and homeomorphic constraints when target character proportions differ significantly from the source.
  3. Multiple hyperparameter sets are predefined without justification or sensitivity analysis, such as (λbnd,λatt,λrep,λctx,λori)=(2,5,1,2,3)(\lambda_{\text{bnd}}, \lambda_{\text{att}}, \lambda_{\text{rep}}, \lambda_{\text{ctx}}, \lambda_{\text{ori}}) = (2, 5, 1, 2, 3) and retargeting weights (λtrack,λcol,λsmo)=(4,2,1)(\lambda_{\text{track}}, \lambda_{\text{col}}, \lambda_{\text{smo}}) = (4, 2, 1), along with K=8K = 8 neighbors, population size P=32P = 32, crossover pc=0.9p_c = 0.9, and mutation pm=0.1p_m = 0.1. It is unclear how sensitive the alignment and trajectory outputs are to variations in these parameters. The authors are suggested to provide a hyperparameter sensitivity evaluation or discussion (e.g., varying λcol\lambda_{\text{col}} relative to λtrack\lambda_{\text{track}}) to demonstrate how robust the framework is to tuning changes.
  4. The motivation of the proposed method is similar to a skeleton-free work [1,2]. The authors are suggested to discuss the difference between the GMP and the skeleton-free method.

[1] Liao, Z., Yang, J., Saito, J., Pons-Moll, G., & Zhou, Y. (2022, October). Skeleton-free pose transfer for stylized 3d characters. In European Conference on Computer Vision (pp. 640-656). Cham: Springer Nature Switzerland.

[2] Wang, H., Huang, S., Zhao, F., Yuan, C., & Shan, Y. (2023). Hmc: Hierarchical mesh coarsening for skeleton-free motion retargeting. arXiv preprint arXiv:2303.10941.

Questions for authors

Please see the weakness part.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.