acceptodds
Withdrawn from ICLR 2027

Editable Action Words for Generative Motion Editing

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Text-based generative motion editing revises a generated clip from language, but mostly within an action, changing how it is performed. Editing the set of actions itself (adding, switching, moving, or removing one) has no direct handle: a generator returns a uniform grid of frames or tokens in which no action exists as an element, so the edit is approximated—a torn boundary, a disturbed rest, a sliding foot. We instead generate the units editing needs: a motion is a set of action words, one per action per body-part track. Each is a Gaussian on the clip's clock (a learned center, duration, and opacity) carrying a text-aligned content vector (what) and a manner residual (how); a named action is the group of action words that share it. The action is now a first-class element: the caption reads back as a legible timeline and an edit is a set operation that re-renders in distribution. Across insert, switch, move, and delete on an action-level editing benchmark, action words alone realize the operation while preserving the untouched motion and keeping foot contact physically plausible, where masked-infilling, attention, and instruction-tuned editors each give up one of these.

Primary area:
generative models
Keywords:
generative motion editing, action-level editing, text-to-motion generation, editable motion representation, human motion synthesis

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_QcWxOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

Human motions are represented as a set of editable action words, each describing an action's content, style, body part, and timing. These words can be extracted from an existing motion clip, or generated from text. The editing adds, replaces, shifts, or removes action words before decoding the modified set back into motion. The evaluation experiments are mostly about whether the requested edits succeed while preserving other actions and maintaining motion quality.

Critical strengths

This paper gives individual actions an explicit, reusable representations, providing a useful interface for editing motion at the action level. The same representation supports both captured motion and generated motion through a shared decoder. The evaluation measures editing success together with preservation and physics quality. The insertion and replacement results support the befinits and practical usefulness of the action editing.

Critical weaknesses

I'm confused on whether editing regenerateds the whole clip, or preserves the original frames outside a local window. To my understanding, the default editing rules can affect more than the selected action. Move may shift other concurrent actions, while Delete may remove matching actions elsewhere in the clip. Currently I cannot find clear descriptions of whole-clip regeneration and preservation of the original frames outside an edit window. For deletion restricted to the selected time span, the paper reports lower success but does not provide the full preservation and motion quality results.

Questions for authors

I want to know which quantitative results and visual examples regenerate the entire clip, and which copy the original frames outside an edit window. For the existing span-only deletion variant, I want to see the report about editing success, preservation of other actions, boundary smoothness, or foot skating on clips with repeated actions. Such evaluations can better strengthen the practical usefulness of this method.

Reviewer_UegtOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on action-level editing. The authors first propose the concept of action words together with a converter to translate raw motion sequences into sets of action words. Then a generator is trained from scratch, based on the caption of action words and the corresponding motion sequences. To evaluate the performance of this new task, the authors also establish a new benchmark.

Critical strengths
  1. The related work part is intriguing. The authors propose a taxonomy to identify the different characteristics for action-level editing, and use it to categorize previous related works, which clearly shows the contribution of this paper.
  2. The overall writing is satisfying, ensuring that its content is easily understandable for readers.
  3. The authors have fully discussed the limitation of this work, which is beneficial for the future research.
  4. The proposed method quantitatively outperforms existing work with a significant margin, as shown in Table 1.
Critical weaknesses

My major concern is about qualitative results:

  1. In the main text, there is no user study to compare the proposed methods with our existing work. Several demonstrations rely on TMR read-back to confirm that an action has changed. Since TMR also defines the content space, independent visual or human assessment is needed to establish that the intended edit is perceptually convincing.
  2. The demo video also lacks visualization for the comparison, making it hard for readers to clearly know how the proposed method.
  3. The qualitative examples cover a narrow range of actions and short clips. They provide limited evidence for repeated action instances, concurrent actions on the same body part, or complex multi-action sequences.
  4. Selected poses and motion strips make it difficult to assess whether timing, trajectory, and performance details remain unchanged outside the edited region. Matching source–output trajectories and frame-aligned comparisons would make this claim easier to verify.
  5. Although the failure cases are discussed, it would be better to shoe some visualization results about them.
Questions for authors

Please refer to the weakness section.

Reviewer_PRVAOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Existing text-based motion editors modify how actions are performed but struggle to edit whole actions because their representations lack explicit action units. The paper introduces “action words,” Gaussian units on body-part timelines that separate action content from manner and make actions directly editable through set operations. Across insertion, switching, moving, and deletion, this representation preserves untouched motion and plausible foot contact, satisfying all three goals where competing editors sacrifice at least one.

Critical strengths

The paper is well organized in reasonable pages.

The problem is essential and interesting.

Critical weaknesses

This paper is highly potential written by an LLM.

The author list of the citation of MG-MotionLLM is not aligned with the real author list.

For "Kinematic Phrases," Zeyi Zhou should be Zizheng Zhou; for "Act As You Wish," Yang Wei should be Wei Yang; and the order of the last two authors for "MotionLab" is reversed.

This paper juxtaposes STMC with FlowMDM, characterizing the latter’s approach—in the context of the DG4 comparison—as "concatenating entire segments and smoothing the boundaries rather than generating them." However, the abstract, the caption for Figure 1, and the methodology section of the original FlowMDM paper explicitly state that the entire sequence is generated simultaneously, with transitions formed via mixed positional encodings during the diffusion process—without requiring any additional post-processing. Consequently, it is inaccurate to broadly describe the FlowMDM mechanism as a process of "concatenation followed by smoothing." While one could separately argue whether or not FlowMDM meets the authors' specific criterion of "re-renderability of action-word sets," such a conclusion cannot be predicated on an imprecise description of the method.

The code for Motion-Agent has been made public. Therefore, it is inaccurate to state, in general terms, that "no executable code has been released." (L1257-L1258). See: https://github.com/szqwu/Motion-Agent/tree/8a5c7b669751dfc21b9348f6bf2193c1732e8643.

The full title for "Rethinking Diffusion for Text-Driven Human Motion Generation" should be "Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression".

The method removes all instances of an action, whereas baselines mainly target a specified window. This raises success from 0.77 to 0.94 but produces 8.7 cm of outside-window motion error, versus CondMDI’s 0.9 cm. Deleting every kick does not demonstrate selective deletion of the second kick.

MotionLab uses an older checkpoint with acknowledged training-data issues, SimMotionEdit produces nearly static outputs unrelated to the source, and MotionCLR is evaluated through regeneration. These results cannot establish fundamental limitations of competing method families without validating checkpoints, conversions, and adaptations and rerunning comparisons.

Questions for authors

TMR supplies both the representation’s action content and the metrics for editing success and semantic preservation, potentially favoring the proposed representation. Physical-quality metrics do not independently establish whether the intended action was correctly edited; an independent semantic evaluator or blinded human evaluation is needed.

Claims that no frames are frozen and the entire clip is decoded conflict with statements that re-rendering remains restricted to the editing window. Code inspection and controlled ablations must clarify whether preservation comes from the action representation or from locking, copying, or conditioning on outside-window motion.

Retrieval is described as best despite scoring 0.843 against SALAD’s 0.865. The highlighted FID of 0.040 measures source-motion reconstruction; text generation scores 0.092, worse than MoMask’s 0.057. Reconstruction quality supports representational capacity but cannot substitute for a fair text-generation comparison.

Reviewer_SJ5KOpenReview ↗
Rating
1
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a novel idea: making the editable units directly correspond to the actions users wish to modify. It represents motion as a set of action words, each encoding semantic content, a body track, and temporal Gaussian parameters, and supports insertion, replacement, repositioning, and deletion through set operations.

Critical strengths
  1. The representation design closely reflects the motivation. M2W, T2W, and W2M enable the same action-word representation to support both captured motion and text-driven motion generation.
  2. Beyond reporting generation quality, the paper examines action grouping, geometry preservation, failure cases of short-window baselines, and potential evaluator coupling, demonstrating a reasonable depth of experimental analysis.
Critical weaknesses
  1. The main results in Table 1 rely on an author-designed benchmark containing 140 HumanML3D clips, with 88 clips for insertion. This does not establish the generalizability of the editing capabilities.
  2. The action-word representation exposes body-part tracks, but preservation of non-target body parts during an edit is not sufficiently evaluated. Although the paper acknowledges limitations in part-level independence and reports cross-track composition results, these do not establish how faithfully untouched body parts are preserved.
  3. The readability could be improved. E.g. Fig 6. The temporal Gaussian plots and body-part track labels are densely arranged, making it difficult to identify which tracks and temporal regions are modified.
Questions for authors
  1. The descriptions of the editing protocol appear inconsistent. Section 3.5 explicitly states that the clip is re-decoded with “no frames frozen,” and the caption of Table 5 also describes whole-clip re-decoding. However, Appendix E states that edit-time re-rendering is windowed, while Appendix I and Figure 12 state that frames outside the window remain the original capture.
  2. In Table 4, the action-words-only reconstruction achieves R@3 = 0.798, compared with 0.806 for the original motions. Could the authors report uncertainty estimates or a paired analysis to establish whether this difference is meaningful and whether particular action semantics are lost during reconstruction? Comparisons with caption-conditioned generation should account for the different conditioning information. 3.T here is a numerical inconsistency: Section 4.4 and Appendix E describe the caption pipeline as outperforming SALAD on retrieval or achieving the highest R@3, whereas Table 4 reports 0.843 for the pipeline and 0.865 for SALAD. Please reconcile these statements with the table.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.