acceptodds
Withdrawn from ICLR 2027

Editing Human Motion through Structured Motion Descriptions

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Text-driven human motion editing aims to modify a source motion sequence according to natural language instructions, enabling users to iteratively refine generated motions. However, current diffusion-based methods denoise the entire motion sequence from random noise within an opaque latent space. This makes it difficult to preserve regions that should remain unchanged and limits the interpretability of the editing process. In this paper, we leverage Structured Motion Descriptions (SMD) to represent skeletal motions as readable text and perform edits directly via a Large Language Model (LLM) guided by instructions. By converting motion into SMD, the LLM selectively modifies only the instruction-relevant regions to achieve precise, localized edits; meanwhile, the readability of SMD enables direct inspection of which joints are altered, over which temporal intervals, and by what magnitude, offering interpretability. We implement this framework through a three-stage training pipeline: supervised fine-tuning to teach the LLM instruction-guided SMD editing, a flow-matching model to enable bidirectional conversion between motion and SMD, and a Group Relative Policy Optimization (GRPO) stage to improve the LLM's numerical accuracy for higher motion fidelity. Extensive experiments on MotionFix demonstrate that our approach achieves state-of-the-art performance while providing interpretability. Code is available in the supplementary materials.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Human Motion Editing, Human Motion Generation, Large Language Models, Flow-Matching, Group Relative Policy Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_SPrXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a human motion editing framework based on Structured Motion Descriptions (SMD). An LLM edits the joint-level descriptions according to natural language instructions, and a flow-matching model reconstructs the edited motion. The authors further use supervised fine-tuning (SFT) and GRPO to improve editing accuracy. Experiments on MotionFix show improvements over several existing motion-editing methods. However, the methodological novelty over existing structured motion-editing approaches remains unclear, and the current experiments do not fully establish the necessity of the proposed design.

Critical strengths

1.The paper addresses an important problem in motion editing: modifying the requested parts while preserving the rest of the motion. 2.SMD provides an interpretable representation of joint-level changes and their temporal locations. 3.The method achieves competitive results on MotionFix and includes experiments on motion preservation and reconstruction accuracy.

Critical weaknesses

1.Limited methodological novelty. CoMo already explores LLM-based editing of structured pose codes, while Iterative Motion Editing with Natural Language uses structured editing programs to modify motions. The paper should clarify what new capability is provided by SMD-based editing beyond these approaches. 2.Insufficient comparison with related methods. Although the paper discusses related work, it does not directly compare with CoMo or Iterative Motion Editing with Natural Language. A more detailed comparison would help establish the contribution of the proposed framework. 3.Preservation after decoding is not fully established. Unchanged SMD entries do not necessarily guarantee unchanged joint trajectories after flow-matching decoding. The reported mean leakage of 2.99° in the single-joint perturbation experiment also suggests that changes may affect other joints.

Questions for authors

1.What is the main technical difference between the proposed method and CoMo or Iterative Motion Editing with Natural Language? 2.Can the authors provide a direct comparison with these structured motion-editing approaches, or a more systematic analysis if direct evaluation is not feasible? 3.How well does the flow-matching decoder preserve unedited joint trajectories and temporal intervals? 4.How much of the performance gain comes from GRPO rather than SMD or SFT? 5.Does the implementation of Preservation Error use absolute angular differences or signed differences?

Reviewer_pHYxOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper formulates text-driven motion editing as rewriting Structured Motion Descriptions (SMD), which encode joint-angle extrema and their timing in readable text. A fine-tuned LLM edits the descriptions, a flow-matching decoder converts the parsed angles into motion, and GRPO trains the editor using motion-embedding similarity rewards. The intended benefit is an inspectable editing process with localized changes.

Critical strengths
  1. The editing interface is concrete and easy to inspect. Comparing source and edited SMD exposes the requested changes in joint angles and timing, which is useful for understanding and revising an edit.

  2. Making SMD decodable is a useful extension of a representation originally designed for motion understanding. Numerical conditioning and feedback from decoded motions provide a practical connection between readable descriptions and motion synthesis.

Critical weaknesses
  1. The motivation is only partially addressed. The paper criticizes prior methods for regenerating the whole motion from noise, yet its own decoder does the same, conditioned on edited angles. These angles do not uniquely determine the source trajectory. Making an edit readable therefore does not itself resolve the central difficulty of preserving unrelated motion.

  2. The methodological advance appears incremental. CoMo already uses an LLM to edit interpretable pose codes. Continuous-angle decoding is a useful extension, but the claimed contribution to numerical precision remains unclear: GRPO rewards motion-embedding similarity, which does not directly enforce the requested angle or timing accuracy.

Questions for authors
  1. Relative to CoMo, what is the key technical advance enabling faithful numerical editing? Appendix G considers numerical token supervision, but how does the final embedding-similarity reward specifically address angle and timing accuracy?

  2. I could not find video demonstrations in the supplied materials. Could you provide side-by-side source and edited motions for localized edits, so readers can inspect temporal continuity, foot contacts, and preservation beyond the static examples?

Reviewer_QctEOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces a pipeline to improve text-driven human motion editing, using Structured Motion Descriptions (SMD) to represent motion and then perform editing by using an LLM to directly edit SMD. The edited SMD is then converted back to human motion with a flow matching model. The proposed pipeline uses GRPO to refine the LLM output to align it to the TMR embedding space. The paper reports marginal improvement over baselines on the MotionFix dataset.

Critical strengths
  • The presented pipeline makes it easier to understand what is being edited than the latent diffusion model, as the edited motion is represented with SMD text.
  • The paper provides a detailed analysis of each component.
Critical weaknesses
  • Overall, the contribution is incremental. The pipeline is constructed by connecting recent prior works with minimal modification.
  • The only change to SMD is unrolling the ‘repeat’ into explicit lines. As the SMD discards global root translation, the edit ignores global trajectory by design, which prevents any precise locomotion edit.
  • The RL alignment is directly taken from HY-Motion and other prior work.
  • The performance is nearly identical to the baselines and is only demonstrated in a single dataset. This point, combined with an expensive GRPO tuning stage, raises questions about the necessity of the whole pipeline.
  • No qualitative motion comparison is provided.
Questions for authors
  • How can the proposed method handle global root translation in SMD?
  • Can you provide a qualitative motion comparison to baselines?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.