acceptodds
Under review as a conference paper at ICLR 2027

MotionMaestro: Masked Tokenization for Unified Motion Generation

Abstract

Human motion generation plays an important role in applications such as character animation, virtual environments, and embodied interaction. While existing approaches have achieved remarkable progress, many of them are developed for individual tasks, including text-to-motion, pose-conditioned generation, and trajectory control. Although these tasks involve different types of conditions, a unified framework capable of handling them within a common representation would greatly simplify motion generation systems. We observe that diverse motion conditions can be naturally formulated as different observation patterns over motion sequences, where each task corresponds to a specific masking strategy. Based on this insight, we introduce **MotionMaestro**, a unified motion generation framework that learns a shared representation for complete motions and heterogeneous partial observations through masked motion tokenization. MotionMaestro employs a three-stage training strategy that first learns a masked motion tokenizer, then refines its reconstruction ability on clean motions, and finally trains a conditional flow-matching generator in the learned latent space. Furthermore, we introduce an observation map and an observation loss to explicitly preserve provided motion conditions during generation. With this unified representation and conditioning mechanism, MotionMaestro supports text-guided and unconditional synthesis, pose conditioning and partial completion, temporal interpolation, trajectory control, and motion continuation. Experiments on the large-scale RoMo and MotionMillion datasets show state-of-the-art performance across diverse motion generation tasks. Code and weights will be released upon acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Human Motion Generation, Masked Representation Learning, Controllable Generation, Flow Matching, Multi-Task Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_8PHqOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes MotionMaestro, a unified framework that handles multiple motion generation and completion tasks with a single model. The core idea is to formulate diverse motion conditions as different masking patterns over motion sequences, so that masking becomes the task specification rather than just a representation-learning trick. The method trains a masked motion tokenizer whose encoder sees both complete motions and task-aligned partial observations, then trains a conditional flow-matching generator in the frozen latent space with an explicit observation map and an observation loss for condition preservation. Experiments on the large-scale RoMo and MotionMillion datasets report state-of-the-art results on most tasks against baselines.

Critical strengths

[Originality]

  • I found the central conceptual move genuinely appealing: treating the mask as the task specification rather than just a representation-learning trick. Exposing the tokenizer encoder to the exact observation patterns it will meet at generation time, so that targets and heterogeneous partial conditions live in one latent space, is a clean idea. Ilike this formulation.

[Quality]

  • The empirical work is thorough with ten tasks on two large-scale datasets, plus well-designed ablations.
Critical weaknesses

n/a

Questions for authors

Overall I don't have specific questions for authors. After reading the manuscript, I feel this is a solid, well-executed contribution and I lean towards accept.

1 comment
AuthorsOpenReview ↗

Response to Reviewer 8PHq

We sincerely thank the reviewer for the careful review and positive assessment. We appreciate the reviewer’s recognition of our task-aligned masking design and thorough experimental evaluation. We will incorporate the additional experiments and clarifications from the discussion into the revised manuscript.

Reviewer_cHNUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

MotionMaestro unifies 3D human motion generation by encoding complete targets and heterogeneous partial observations in shared joint–time latents. It combines task-aligned masked tokenizer training, decoder refinement, and conditional flow matching with an observation map and decoded observation loss.

Critical strengths
  • The shared representation is well motivated and provides a coherent interface for heterogeneous motion conditions within a unified framework.
  • The visual quality of generated results is overall good from the demo.
Critical weaknesses
  • The baseline comparison is not sufficient. The authors only compare with 2 previous "unified frameworks" and 1 general prior. It would be more of a fair comparison to compare the results with other state-of-the-art controllable generation works, e.g. [1][2][3][4], as most of them are also capable of performing almost all or at least most the sub-tasks demonstrated here.
  • The frame interpolation & first-last frame sub-tasks defined in the paper are not commonly used terms, as they are special cases of motion in-betweening. In motion in-betweening, it can be arbitrary frame(s) provided and it doesn't have to be consistent/periodic intervals.
  • The masking strategy is thus a bit overly complicated and redundant to me. The anatomical tube masking already entangle the spatial and temporal masking, and the temporal intervals are even more flexible than the temporal stride masks. Why do we still need the temporal stride masks in this case? It may even harm the generalization to general motion in-betweening performance. Similary, the short-window masking is also not well motivated.
  • In the experiment part, the ablation for the masking strategy only turns the overall masking on&off, but doesn't include separate study of the effectiveness of each types of masks.
  • Headline baseline gains conflate method and model scale: MotionMaestro has 1.285B parameters versus MotionLab's 0.367B, with unmatched compute (Table 2, Appendix E). The clean-only variant already achieves RoMo T2M FID 18.3 versus MotionLab's 101.5, so structured masking explains an incremental benefit rather than the entire gap (Tables 1, 5).

[1]“FrankenMotion: Part-level Human Motion Generation and Composition.” CVPR, 2026.

[2]“MaskControl: Spatio-Temporal Control for Masked Motion Synthesis.” ICCV, 2025.

[3]“SALAD: Skeleton-aware Latent Diffusion for Text-driven Motion Generation and Editing.” CVPR, 2025.

[4]“Motion Synthesis with Sparse and Flexible Keyjoint Control.” ICCV, 2025.

Questions for authors

For the motion in-betweening tasks, some the generated results look too similar with the reference motion. Is it an overfitting problem? Would it be more effective for comparison if text is also provided?

6 comments
AuthorsOpenReview ↗

Response to Reviewer cHNU (1/5)

We thank the reviewer for the constructive feedback and positive assessment of our work. The suggestions helped us clarify the baseline comparisons, masking design, and interpretation of the gains. We will incorporate the additional results and clarifications into the revised manuscript.

W1. Additional baseline comparisons

The table below compares the conditioning interfaces of our original baselines with those of the methods suggested by the reviewers.

Conditioning interfaces. original identifies baselines in our submission; cHNU and vKm6 identify the reviewers who suggested each additional method. ✓: directly demonstrated; △: narrower or related setting; ✗: not directly reported. References [1]–[4] follow the reviewer’s numbering.

MethodTextPose / keyframesJoint / EE controlPartial completionRoot trajectoryMotion prefix
MDM (ICLR'23; original)✓✓✓✓✗✓
OmniControl (ICLR'24; original)✓△✓△✓✗
MotionLab (ICCV'25; original)✓✓✓✓✓✓
MaskControl [2] (ICCV'25; cHNU)✓△✓△✓✗
SALAD [3] (CVPR'25; cHNU)✓✗✗edit✗✗
Keyjoint [4] (ICCV'25; cHNU)✓△✓✗✓✗
FrankenMotion [1] (CVPR'26; cHNU)parts✗✗text✗✗
Kimodo (arXiv'26; vKm6)✓✓✓△✓✓
MotionMaestro✓✓✓✓✓✓

Pose/keyframes covers P2M, TP2M, FLF, and FI; root trajectory and motion prefix correspond to trajectory-to-motion and extrapolation. Joint/EE control constrains selected joints or end effectors, while partial completion specifies observed body-part motion over time. parts, edit, and text refer to part-level text conditioning, attention-based editing, and text-driven body-part control, respectively. Kimodo’s △ reflects sparse end-effector control, a related form of partial-body control.

As shown above, MDM, OmniControl, and MotionLab cover motion-observation settings closely related to ours, motivating their inclusion in the original comparisons. FrankenMotion [1] combines part-, action-, and sequence-level text control, while SALAD [3] supports text-driven generation and editing. MaskControl [2] and Keyjoint [4] provide spatial control, with Keyjoint also demonstrating initial-pose conditioning.

Given time constraints and limited GPU resources, we prioritized SALAD [3] for text-to-motion generation and MaskControl [2] for multi-task evaluation to provide timely results for the discussion.

Text-to-motion comparison with SALAD. We compare T2M on MotionMillion and RoMo using one unified 1.285B MotionMaestro model per dataset with CFG w=3w=3. Arrows indicate preferred metric directions. In performance comparisons, bold numbers indicate the best result for each metric, including ties; — denotes an unavailable result.

DatasetMethodR@1 ↑R@3 ↑FID ↓MM Dist ↓
MotionMillionSALAD [3] (CVPR'25)0.6140.84358.6829.26
MotionMillionMotionMaestro0.6630.89220.7228.19
RoMoSALAD [3] (CVPR'25)0.5230.78032.5630.21
RoMoMotionMaestro0.5410.79016.4329.98

As a unified framework, MotionMaestro outperforms SALAD, a specialized text-driven generation and editing model, on all four quality and text-alignment metrics across both datasets. MotionMaestro achieves FID 20.72 vs. 58.68 on MotionMillion and 16.43 vs. 32.56 on RoMo.

Multi-task comparison with MaskControl on RoMo. We retrained MaskControl on RoMo and evaluated it under the ten-task protocol in Table 1 of the main paper. FI reports mean per-joint position error (MPJPE, mm ↓) over unobserved frames; partial completion reports MPJPE over unobserved joints. All other tasks report FID (↓). MotionMaestro-S is our smaller 0.319B variant (see W5).

MethodParamsUncondT2MP2MTP2MFLFFIExtrap.PartialTraj.Long
MaskControl [2], retrained0.066B586.092.0165.497.5110.439.2123.4117.3140.086.4
MotionMaestro-S0.319B17.014.336.230.120.86.821.167.154.823.1
MotionMaestro1.285B20.414.934.828.720.86.719.461.059.226.2

Both MotionMaestro variants outperform MaskControl on all ten reported task metrics. Lower MPJPE on the FI and partial completion tasks indicates more accurate reconstruction of unobserved motion. Comparisons with Kimodo fine-tuned on RoMo under the same evaluation protocol are reported in W5.

AuthorsOpenReview ↗

Response to Reviewer cHNU (2/5)

W2. In-betweening terminology and arbitrary keyframes

Thank you for the terminology suggestion. We will revise the manuscript to describe FI and FLF as periodic-keyframe and endpoint-conditioned in-betweening, respectively.

To examine the broader in-betweening setting highlighted by the reviewer, we provide additional experiments with non-periodic keyframes on RoMo. In our original Stage-3 training, keyframe-conditioned samples used periodic keyframes (50%), only the first and last frames (25%), or these endpoints plus 0–6 randomly selected intermediate frames (25%). The five test settings below vary the number and arrangement of observed frames. None requires the first or last frame to be observed.

Non-periodic in-betweening on RoMo. Each cell reports MPJPE (mm ↓) / FID (↓). We evaluate 512 clips using identical seeded masks across methods, with MPJPE computed over unobserved frames. The first three settings provide observations for 5%, 10%, or 20% of the frames. The Two 4-frame blocks setting provides two observed segments of four consecutive frames each; the Sparse (2–8 keyframes) setting provides 2–8 observed frames per clip.

Method5% keyframes10% keyframes20% keyframesTwo 4-frame blocksSparse (2–8 keyframes)
MDM398.7 / 439.4308.6 / 375.5181.1 / 243.1328.1 / 431.6345.7 / 405.8
MotionLab54.9 / 26.234.2 / 19.719.8 / 13.962.3 / 27.144.6 / 24.5
Kimodo (vKm6)49.5 / 23.531.7 / 14.418.2 / 7.459.0 / 24.041.0 / 17.9
MotionMaestro-S (0.319B)49.8 / 19.429.9 / 11.516.5 / 6.163.1 / 21.842.8 / 15.5
MotionMaestro (1.285B)48.2 / 17.728.4 / 10.715.0 / 5.559.9 / 21.040.8 / 14.5

MotionMaestro (1.285B) achieves the best FID in all five settings and the lowest MPJPE over unobserved frames in four of five settings. For the two 4-frame blocks setting, MotionMaestro achieves a lower FID than Kimodo (21.0 vs. 24.0), while Kimodo has a slightly lower MPJPE (59.0 vs. 59.9 mm). These results support motion in-betweening beyond the periodic-keyframe and endpoint-conditioned settings in our original evaluation.

AuthorsOpenReview ↗

Response to Reviewer cHNU (3/5)

W3–W4. Masking design and component ablations

Given time constraints and limited GPU resources, we focus our additional ablations on the two components questioned by the reviewer: temporal stride masking and short-window samples. We first report a generation ablation using MotionMaestro-S, our smaller 0.319B variant (see W5).

Motion generation results with MotionMaestro-S. Generation metrics on RoMo follow Table 1 of the main paper. FI reports MPJPE on unobserved frames, and partial completion reports MPJPE on unobserved joints, both in mm (↓). The other tasks report FID (↓). The main FI column uses stride 4; the last two columns additionally report FI at strides 16 and 32.

MotionMaestro-SUncondT2MP2MTP2MFLFFIExtrap.PartialTraj.LongFI (s=16)FI (s=32)
Full mask mixture17.014.336.230.120.86.821.167.154.823.132.357.0
w/o temporal stride masking17.013.940.432.124.421.320.765.854.422.937.764.0
w/o short-window samples20.015.438.330.721.47.521.368.354.324.932.658.9

These results support the task-aligned design in Figure 4: temporal stride masking prepares the encoder for complete keyframes separated by missing frames, directly matching the FI setting. Even with anatomical tube masking retained, removing temporal stride masking worsens FI MPJPE from 6.8 to 21.3 mm at stride 4. The degradation persists at strides 16 (32.3 → 37.7 mm) and 32 (57.0 → 64.0 mm), while other tasks show mixed changes, supporting the distinct contribution of temporal stride masking beyond anatomical tube masking. Removing short-window samples worsens nine of the ten main task metrics, including P2M FID from 36.2 to 38.3 and TP2M FID from 30.1 to 30.7. FI errors also increase at strides 16 and 32, while trajectory FID improves slightly. These results further support training the tokenizer on short inputs for pose-conditioned generation.

We also evaluate the tokenizer directly, without the generator, to examine reconstruction from partial observations and short motion inputs.

Motion reconstruction after Stages 1 and 2 (MPJPE, mm ↓). Each ablation removes one strategy from Stage 1 and replaces its training samples with samples from the remaining strategies. All tokenizers receive 50k Stage-1 updates, followed by Stage-2 decoder refinement, and are evaluated on the same 2,048 test clips and masks. The first three columns measure error over unobserved frames; the last two measure reconstruction from short inputs.

Tokenizer trainingPeriodic, stride 4Non-periodic, 5%Non-periodic, 20%1-frame input4-frame input
Full mask mixture5.561.421.63.924.43
w/o temporal stride masking108.999.577.54.133.86
w/o short-window samples5.255.922.77.316.64

Although the full mixture is not best in every column, removing temporal stride masking worsens all three keyframe settings, while removing short-window samples worsens 1-frame and 4-frame reconstruction. Together, the generation and reconstruction results support retaining both components to handle sparse keyframes and short motion inputs.

Reviewer_vKm6OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes MotionMaestro, a unified framework for motion generation that supports many tasks (text-to-motion, keyframes, interpolation, extrapolation, partial-body completion, trajectory control, long sequences) through one shared representation and conditioning interface. A motion tokenizer is trained with task-aligned masking so that one encoder embeds both complete motions and partial observations. A flow-matching generator is conditioned on the encoded observations, together with an observation map and a motion-space observation loss that encourage consistency with the provided conditions.

Critical strengths
  • The paper addresses an important and challenging problem of controllable motion generation.
  • The idea of task-aligned masking is intuitive and clearly improves over random masking for reconstructing masked regions.
  • The experiments are large-scale, on two recent million-scale datasets, though both datasets contain low-quality pseudo-labeled motion annotations.
Critical weaknesses
  • Key related works are not cited. Kimodo (Rempe, Petrovich et al., 2026) and ARDY (Zhao et al., TOG 2026) are not cited at all. Both are state-of-the-art text-to-motion models that support a wide range of control signals. Relatedly, the paper does not use Bones-SEED, a public motion capture dataset that would allow a direct comparison with Kimodo, and is much higher quality compared to the used pseudo-labeled datasets. The main claim overlaps with these methods, and there are no comparison. The central claim is a unified framework that supports many motion generation tasks through one shared representation and conditioning interface. That is very similar to Kimodo and ARDY, which also handle varied conditioning signals (keyframes, sparse joints, end-effectors, root trajectories, in-betweening) in a single model. Unfortunately, the paper does not compare against both, so its position relative to the current SOTA is unclear. I consider the missing Kimodo comparison a significant shortcoming.

  • Encoding constraints through the tokenizer limits how precisely they can be met. Because constraints pass through the encoder and decoder, they cannot be met exactly. The paper itself needs an explicit observation map and an additional observation loss (L_obs) to encourage consistency with the given conditions. Kimodo instead overwrites constrained values directly in motion space during denoising, which is a more direct way to enforce constraints and offers better controllability. A major open question is therefore how Kimodo’s overwriting-based design compares with MotionMaestro’s encoder-based conditioning.

  • No exact root control. ARDY keeps the root explicit, so root constraints can be overwritten exactly. In MotionMaestro the root is also encoded in the latent, so none of the constraints, including the root trajectory, can be guaranteed to be met exactly. This is a significant limitation for applications that need precise control.

  • Novelty over MAETok is incremental. The tokenizer recipe closely follows MAETok: masked autoencoder pretraining, then decoder-only fine-tuning with the encoder frozen. The main differences are the task-aligned mask patterns and reusing the masked encoder for conditions.

Overall, the paper presents a clean, practical framework with strong large-scale results, and task-aligned masking is well supported against random masking. However, the closest relevant methods (Kimodo, and ARDY as concurrent work) are not cited or compared, and the key design choice, encoding conditions with a masked tokenizer, is not compared against simpler explicit-feature conditioning.

I would be open to raising my score if the authors add the Kimodo comparison and the explicit-conditioning ablation.

Questions for authors
  • My main concern is how MotionMaestro compares with Kimodo on the shared constraint tasks, e.g. by retraining Kimodo’s public code on RoMo or MotionMillion, or by evaluating MotionMaestro on Bones-SEED?
  • Can you add an ablation where the tokenizer is trained on clean motion only, and conditions are given to the generator as explicit features plus the observation map (no encoding of the conditions)?
5 comments
AuthorsOpenReview ↗

Response to Reviewer vKm6 (1/5)

We thank the reviewer for the careful review and helpful suggestions. We have added the requested comparisons with Kimodo and an explicit-feature conditioning ablation. These additional experiments and the clarifications below will be incorporated into the revised manuscript.

W1/Q1. Comparison with Kimodo and related work (1/2)

Kimodo and ARDY will be cited and discussed in the revised manuscript. Given the limited time and GPU resources during rebuttal, the additional experiments focus on Kimodo. Following the reviewer’s suggestions, we compare the methods in two settings: fine-tuning Kimodo on RoMo and training MotionMaestro on BONES-SEED.

Fine-tuning Kimodo on RoMo. We select Kimodo-SMPLX-RP-v1 as the baseline for our RoMo comparison because its 22-joint SMPL-X body topology is compatible with MotionMaestro’s 22-joint representation. Since Kimodo provides pretrained checkpoints and inference code but no official training pipeline, we use the released checkpoint as the starting point for fine-tuning on RoMo. Evaluation follows Table 1, with the same test clips, observation masks, and metrics used for MotionLab and MotionMaestro.

The comparison also includes MotionMaestro-S (0.319B), introduced in response to Reviewer cHNU’s concern about model scale (W5). It uses the same tokenizer, training data, task mixture, and 200k-update generator training schedule as the full model, allowing comparison with Kimodo at similar parameter counts. MotionLab and full-model MotionMaestro results are reproduced from Table 1.

FI reports mean per-joint position error (MPJPE, mm ↓) over unobserved frames; partial completion reports MPJPE over unobserved joints. All other tasks report FID (↓). Throughout this response, bold numbers indicate the best result among the compared models, including ties; — denotes an unavailable result.

MethodParamsUncondT2MP2MTP2MFLFFIExtrap.PartialTraj.Long
MotionLab (Table 1)0.367B76.7101.553.546.530.012.728.171.889.3123.5
Kimodo, fine-tuned (this response)0.283B104.534.264.845.530.39.124.2—110.042.1
MotionMaestro-S (cHNU, W5)0.319B17.014.336.230.120.86.821.167.154.823.1
MotionMaestro (Table 1)1.285B20.414.934.828.720.86.719.461.059.226.2

Both MotionMaestro variants outperform fine-tuned Kimodo on all nine tasks with available results, with gains across text, pose, keyframe, and trajectory conditioning. MotionMaestro-S outperforms fine-tuned Kimodo on both T2M FID (14.3 vs. 34.2) and FI MPJPE (6.8 vs. 9.1 mm). These results demonstrate consistent improvements across different motion conditions, including when the models have comparable parameter counts.

AuthorsOpenReview ↗

Response to Reviewer vKm6 (2/5)

W1/Q1. Comparison with Kimodo and related work (2/2)

Training MotionMaestro-S on BONES-SEED. Following the reviewer’s second suggestion, we compare with Kimodo-SOMA-SEED-v1.1, the released SOMA model trained on BONES-SEED. MotionMaestro-S is used to keep the model size comparable to Kimodo and make training feasible within the rebuttal budget. Both its tokenizer and generator are trained from scratch on the official training split, following our three-stage procedure with 50k updates in Stage 1, 10k in Stage 2, and 100k in Stage 3.

To adapt our 22-joint representation to BONES-SEED’s 77-joint SOMA skeleton, we map the corresponding body joints and convert generated motions to the 77-joint format required by the benchmark. The tokenizer, masking strategies, tasks, and losses remain unchanged. The conditioning inputs differ: MotionMaestro does not receive end-effector rotations, but it receives joint rotations at full-body keyframes that Kimodo does not use.

Evaluation uses the official 22,474-case motion-generation test suite and Kimodo’s scoring code as released. The table below reports constraint following for full-body keyframes, end-effector targets, root paths and waypoints, and their combinations, with and without text. Content tests unseen motion categories; Repetition tests new clips from categories seen during training.

All position errors are in cm (↓). FB Pos. and EE Pos. measure full-body keyframe and end-effector position errors. 2D Root measures mean root-position error on the ground plane; Pelvis@95 reports its 95th percentile. GT is included as a reference.

SettingMethodFB Pos. ↓EE Pos. ↓2D Root ↓Pelvis@95 ↓
Content, textGT (reference)0.0000.0003.8375.36
Content, textKimodo3.4193.8174.9769.17
Content, textMotionMaestro-S5.6026.5995.45311.92
Content, no textGT (reference)0.0000.0003.9135.41
Content, no textKimodo3.3203.6654.8019.03
Content, no textMotionMaestro-S5.9236.4065.70713.25
Repetition, textGT (reference)0.0000.0003.6075.44
Repetition, textKimodo3.1863.8494.7339.18
Repetition, textMotionMaestro-S4.1685.1394.3929.37
Repetition, no textGT (reference)0.0000.0003.5675.42
Repetition, no textKimodo3.1193.5104.2657.89
Repetition, no textMotionMaestro-S4.1905.4374.3589.98

GT has nonzero root errors because the benchmark’s root constraints target a smoothed trajectory, which differs from the original pelvis trajectory.

Kimodo achieves better constraint following in nearly all settings; MotionMaestro-S performs slightly better on 2D Root in Repetition with text (4.392 vs. 4.733 cm). MotionMaestro-S retains the training task settings used in our RoMo experiments, whereas Kimodo’s training explicitly includes the sparse constraint patterns evaluated in this benchmark. Differences in task settings, training schedules, and conditioning inputs may contribute to the gap. Further work will examine training with these sparse constraints under matched conditioning inputs.

These comparisons highlight different strengths: MotionMaestro achieves better generation results under our RoMo protocol, while Kimodo provides more accurate constraint following on BONES-SEED.

AuthorsOpenReview ↗

Response to Reviewer vKm6 (3/5)

W2. Preserving motion conditions

In MotionMaestro, the observation map identifies the given motion features, and the observation loss penalizes errors in those features after decoding. To compare Kimodo’s overwriting-based design with MotionMaestro’s encoder-based conditioning, we conduct two evaluations: the first measures how closely the output matches the observed motion, while the second evaluates the accuracy and quality of the generated, unobserved motion.

The comparison includes MDM, MotionLab, and Kimodo fine-tuned on RoMo (W1). All methods are evaluated on the same RoMo clips and observation masks, and both evaluations use the same generated outputs. As in Kimodo’s quantitative experiments, we evaluate the generated motions directly, without further corrections to match the given conditions.

Preserving observed motion. For all tasks except trajectory control, we report MPJPE on the observed joints, measured in local coordinates aligned with the character’s heading. Traj. reports error on the observed root trajectory. All errors are in mm (↓).

MethodFLFFIExtrap.PartialP2MTP2MTraj.
MDM0.000.000.000.000.000.00—
MotionLab10.1010.329.9017.469.509.6825.93
Kimodo, fine-tuned7.687.257.33—7.127.1922.02
MotionMaestro4.853.702.913.495.064.637.51

MDM achieves zero error by overwriting the observed values during inpainting. For FI, MotionMaestro achieves an MPJPE of 3.70 mm, improving on 7.25 mm for Kimodo and 10.32 mm for MotionLab. Although MotionMaestro does not preserve the given conditions exactly, it preserves them more accurately than MotionLab and Kimodo in every available comparison.

Generating unobserved motion. Generation metrics on RoMo follow Table 1 of the main paper. FI reports MPJPE on unobserved frames, and partial completion reports MPJPE on unobserved joints, both in mm (↓). The other tasks report FID (↓).

MethodFLFFIExtrap.PartialP2MTP2MTraj.
MDM523.3117.6342.8278.9518.4456.1—
MotionLab30.012.728.171.853.546.589.3
Kimodo, fine-tuned30.39.124.2—64.845.5110.0
MotionMaestro20.86.719.461.034.828.759.2

MotionMaestro achieves the best generation results in all seven settings. For FI, its MPJPE is 6.7 mm, compared with 9.1 mm for Kimodo and 117.6 mm for MDM. Together, the two evaluations show that MotionMaestro combines low error on the observed conditions with strong performance when generating the missing motion.

W3. Exact root control

In MotionMaestro, the root is encoded together with the other motion features, so exact root preservation is not guaranteed. Our design uses the observation loss to encourage the decoded root positions to match the given conditions. To assess this design, we conduct a dedicated evaluation of root-position accuracy, keeping the existing trajectory control, FI, extrapolation, and FLF task settings unchanged.

Root control accuracy on RoMo. We compare MotionMaestro with MotionLab, OmniControl, and Kimodo fine-tuned on RoMo (W1–W2). We report mean root-position error in the xz plane at the observed frames, measured in mm (↓).

MethodTraj.FIExtrap.FLF
MotionLab25.924.76.1132.0
OmniControl34.785.646.823.1
Kimodo, fine-tuned22.021.736.945.8
MotionMaestro7.55.70.944.1

MotionMaestro achieves the lowest root-position error in three of the four settings: trajectory control, FI, and extrapolation. For trajectory control, MotionMaestro achieves 7.5 mm, compared with 22.0 mm for Kimodo. The low errors in FI and extrapolation also show that root positions remain closely matched when supplied together with full-body motion conditions.

Although MotionMaestro does not guarantee zero error at the given root positions, these results show that an encoded root representation can still provide accurate control across multiple motion-conditioning settings.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Rebuttal Update and Manuscript Revision

We sincerely thank the reviewers for their time and careful consideration, and appreciate the reviewers’ recognition of our work and constructive feedback. We apologize for the delay in our responses, as we needed additional time to complete the experiments. We will revise the manuscript to address all feedback and include the additional results, and upload the revised version before the rebuttal deadline.

AuthorsOpenReview ↗

Revised Manuscript Uploaded

We have uploaded the revised manuscript with additional experiments and clarifications addressing the reviewers’ feedback. Revised text is highlighted in blue. We would appreciate your consideration of the updated manuscript and our responses. We thank the reviewers and Area Chair for their time and thoughtful consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.