- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
TRACE presents a framework for text2motion generation and editing for the Unitree G1 humanoid. For generation, it extracts confidence-weighted support, root, and hand events from training motions through forward kinematics and supervises a language model to predict corresponding event plans and duration from text. A diffusion Transformer combines numerical, temporally aligned event conditions with the original caption. The system also uses a gravity-referenced motion representation and geometric/contact objectives.
For editing, a separate language adapter and deterministic compiler convert supported requests into body-time scopes and numerical source-relative goals. A separately trained diffusion editor conditions on the source motion, goals, and revised events, while reference projection preserves protected encoded coordinates.
Evaluation includes generation comparisons on a processed BONES-SEED split, matched generation ablations, a human study, qualitative robot examples, and controlled hand-height editing.
Critical strengths
-
Realization-aligned supervision for intermediate plans is a clean, articulable idea, and the paper tests it honestly. The claim that an intermediate description should be grounded in the specific paired demonstration (because a caption may have many valid realizations) is a real conceptual point that distinguishes this from Motion-R1 / MotionScript / FineMoGen-style caption-derived decomposition. The authors build exactly the matched control that could falsify it and report that it helps on every metric. I value the design of this control more than its magnitude.
-
Compiling an edit request into a measurable kinematic scope and a metric, source-relative goal, then scoring success by FK rather than by distance to a single "ground-truth" edited motion, is the right evaluation philosophy for motion editing. The multi-turn protocol is more rigorous than what MotionFix-style benchmarks use. If the community adopted "goal-verifiable relative edits" as an evaluation paradigm, that would outlive any particular model, which is why I count it as consequential even though the specific task tested is narrow.
Critical weaknesses
1. The lasting conceptual contribution is insufficiently distinguished from prior work and insufficiently isolated by the generation analysis.
MotionScript already identifies the risk of caption expansion becoming inconsistent with the paired motion and constructs motion-grounded descriptions. CoMo[1], which is not discussed in this submission, already uses motion-derived semantic codes for body-time localization and iterative language-guided editing. TRACE’s numerical goals, continuous diffusion realization, and coordinate projection are meaningful distinctions, but the paper needs to establish what new capability or understanding follows from them. The matched generation ablations support the the proposed conditions, but they do not directly measure the motivating plan–motion conflict, generated-event adherence, or a failure regime that aligned plans resolve. Moreover, Table 5 fixes the same inference planner while changing part of diffusion-training supervision. It therefore does not distinguish the proposed realization-alignment explanation from other changes in training-condition compatibility. My concern is the limited explanatory evidence for the central insight.
2. The controlled editing study does not establish what the proposed learned editor adds beyond existing kinematic tools.
The paper emphasize the ability of whole-body editing, and demonstrates examples like "Jump higher, rest unchanged" (Fig 1). Such tasks requires full-body joints editing or manipulation. However, the actual quantitative benchmark only evaluate a single metric, i.e. single-hand vertical wrist-height offsets on a fixed sine-squared temporal envelope, and explicit time intervals in the request. While other baselines receives different information. Consequently, the success gap does not isolate the contribution of the event representation. The broader whole-body editing example remains qualitative. Together, these limitations prevent the editing results from establishing a substantial advance beyond the restricted task and interface used for training.
References:
[1] Huang, Y., Wan, W., Yang, Y., Callison-Burch, C., Yatskar, M., & Liu, L. (2024, September). Como: Controllable motion generation through language guided pose code editing. In European Conference on Computer Vision (pp. 180-196). Cham: Springer Nature Switzerland.
Questions for authors
- Can the authors clarify the precise new insight relative to MotionScript’s motion-grounding argument and CoMo’s semantic-code-based iterative editing? Which reported result most directly establishes the benefit of TRACE’s formulation?
- OMG/ECHO/TextOP/TeXedo use official checkpoints trained on different data and are scored by an evaluator trained on your split. Were they retrained on your split, or can you report them on a shared evaluator / shared data? Otherwise the Table 1 rows for these methods mostly measure distribution shift.
- How much of the improvement remains when revised source-event conditioning is removed from TRACE? Conversely, how does an adapted learned baseline perform when given the same numerical offset representation? These controls would help identify whether event reuse provides a consequential benefit beyond explicit goal conditioning.
- Can the authors quantify the difference between caption-derived plans and paired motions, and show whether generated motions follow their supplied events?
2 comments
Thank you for the additional results. The new experiments showed that generated motions follow the event plans and support the benefit of motion-derived supervision. These provide stronger evidence of my concerns.
I am not an expert in the area but I still find the new insight beyond prior work unclear. The palm experiment does not show how much event conditioning contributes to the original editing results, and the MotionFix comparisons do not establish an advantage over existing editors. I question whether the complex framework could provide substantial impact to the community. I would retain my original score for now.
Response to Reviewer 8mvh
Thank you for asking which experiment shows that the generator executes a plan, and what a learned editor adds once every method receives the same goal.
Do generated motions follow the timing supplied by the plan?
Yes. On segments where foot support changes, supplying event intervals improves overlap with the requested timing by 17.39 percentage points for the left foot and 18.24 for the right, relative to a clip-wide summary. MotionScript grounds descriptions in demonstrations, and CoMo localizes edits with body-part pose codes. TRACE builds on those ideas by predicting when an event should hold and checking the interval against the one extracted from the generated trajectory. A caption can permit several timings. The supplied plan specifies which one to test. Interval IoU measures the overlap between those two intervals.
With the inference planner fixed, changing the generator’s training labels improved retrieval and lowered FID.
What does generation add when an edit specifies no joint trajectory?
In the jumping jack, “jump higher, keep the rest unchanged” leaves the trajectory of the pelvis and both legs unspecified. TRACE generated a deeper crouch followed by a higher apex. The user supplied no target height, knee angles or keyframes. At 0.45 s, the pelvis was 9.37 cm lower than in the source and both knees were more flexed. The pelvis then rose 38.29 cm to its apex, compared with 19.35 cm in the source.
A constant upward shift of 9.57 cm would match the edited apex but would put the pelvis at 79.67 cm at the low point, versus the observed 60.73 cm. In this example, generation provides one coordinated trajectory from the instruction and source motion.
Event features help when the wrist trajectory alone leaves the edit underdetermined. Holding the wrist centre fixed leaves wrist rotation available to move the palm across a requested height boundary.
The whole-body refinement followed instructions on MotionFix. On 1,013 held-out source–target pairs, it came closer to the target on all three measures when it received the paired human-written instruction than when the same instructions were shuffled across sources. The first two columns of scores report gains over copying the source. For paired instructions, the 95% CIs for those gains were 0.00668–0.01806 in target-embedding distance and 0.635–0.853° in mean joint-angle error.