acceptodds
Withdrawn from ICLR 2027

PartiGen: Part-Aware Representations for Language-Conditioned Humanoid Motion Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Translating language into humanoid motion requires coordinating body parts, determining motion duration, and producing references suitable for robot execution. However, existing text-to-motion methods have limitations in whole-body coordination and duration adaptation, while the physical constraints of humanoid robots are not always explicitly considered. We introduce PartiGen, a part-aware latent diffusion framework that incorporates anatomical structure into a shared whole-body motion representation. Part-specific attention processes anatomically masked frame features, and its outputs are fused to combine local limb information with whole-body context. Conditioned on text and motion history, the model generates motion primitives autoregressively. Forward-kinematics supervision encourages kinematic consistency, while a learned end-of-sequence predictor selects a frame-level endpoint after generation. On HumanML3D retargeted to the Unitree G1, PartiGen reduces segment-level FID by 16.8% over TextOp and improves duration accuracy over a fixed-length cutoff. With a pretrained, frozen tracking controller, we evaluate generated references in MuJoCo and demonstrate execution of selected motions on the real G1 after simulation screening and manual selection.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Motion Generation, Embodied AI, Humanoid Robotics, Structured Representations

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_BX8sOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents a part-aware latent diffusion framework for language-conditioned humanoid motion generation. It introduces anatomically masked attention to learn a shared whole-body motion representation, and combines autoregressive motion generation with forward-kinematics supervision and frame-level endpoint prediction. Experiments on HumanML3D-G1 evaluate motion quality, while simulation and real-world demonstrations on the Unitree G1 examine the executability of generated motions.

Critical strengths

The paper evaluates generated motions in simulation and demonstrates their execution on a real Unitree G1 robot, providing practical evidence of the proposed framework's applicability.

Critical weaknesses

I have significant concerns about the novelty and contributions of this paper. Humanoid motion generation is closely related to human motion generation. On the one hand, this paper directly uses existing human motion generation models followed by retargeting as baselines. On the other hand, the conversion from human motion to humanoid motion mainly relies on established methods.

Given that the proposed architecture does not appear to be inherently restricted to humanoid robots, I believe it should also be evaluated on standard text-to-motion benchmarks such as HumanML3D. Furthermore, the lack of comparisons with more recent and competitive text-to-motion methods makes it difficult to assess the novelty and performance advantages of the proposed approach over existing work.

Questions for authors
  1. There are already many part-wise motion generation methods in the T2M literature. I suggest including more of these in the Related Work section and providing a clearer discussion of how the proposed approach differs from them.
  2. Were the TextOp results obtained using a model retrained on HumanML3D-G1, or using its original pretrained checkpoint? Similarly, which training datasets and checkpoints were used for DART, MDM, and AttT2M? Please clarify these in the paper.
  3. I suggest including more recent and closely related text-to-motion methods in the experimental comparisons.
  4. It seems that the existing baselines use pretrained checkpoints rather than being retrained on HumanML3D-G1. I understand that retraining all baselines would be time-consuming, but evaluating two or three representative methods trained directly on HumanML3D-G1, alongside their original human-domain models followed by retargeting, would help assess the fairness of the comparison. Alternatively, the authors could train their proposed method directly on HumanML3D and report its performance both in the original human motion domain and after retargeting to G1.
2 comments
AuthorsOpenReview ↗

Response to Reviewer BX8s:Relation to Prior Part-Based Methods and Direct G1-Domain Evaluation

We thank the reviewer for requesting a clearer account of the relationship to prior part-based methods, checkpoint provenance, and recent comparisons.

Q1. Relationship to prior part-based generation. Several prior works use body-part structure, but they target different interfaces and representations. AttT2M (ICCV 2023) inserts a body-part spatio-temporal encoder into a VQ-VAE and uses global-local text attention, producing human motion in a discrete latent space. SINC (ICCV 2023) composes separately generated motions by selecting body-part trajectories at test time, while STMC (CVPRW 2024) extends spatial composition with multi-track timeline conditioning and denoising-time aggregation. TextIM (2024) uses an LLM to identify interacting parts, decoupled conditional diffusion, and a part-graph spatial-coherence module for interactive human motion. KinMo (ICCV 2025) represents six kinematic groups and uses hierarchical global/action/group descriptions with part-level generation and editing. FineMoGen (NeurIPS 2023) models fine-grained spatio-temporal composition with a mixture-of-experts attention module and dedicated annotations. FrankenMotion (2026) adds temporally aligned part captions and hierarchical part/action/sequence conditioning, which we evaluate below.

PartiGen differs along three axes. First, it takes sequence-level language without part-level text annotations. Second, anatomical masks are applied to G1 joint features before the attention projections, while all body regions remain in one shared whole-body latent; the model does not stitch independently generated part trajectories or require part-specific textual controls. Third, it is trained directly in G1 joint space with forward-kinematics supervision and evaluated for robot execution through a frozen G1 controller. The cited human-domain methods primarily report human-motion quality or controllability on SMPL/HumanML3D-style benchmarks; they do not target executable G1 references. PartiGen additionally evaluates controller tracking in MuJoCo, where 409/500 generated references pass the posture screen, and physical G1 execution, where 26/27 manually selected trials are judged successful. Thus, these works study fine-grained human-motion control through part-level prompts, annotations, or test-time composition, whereas PartiGen studies anatomical structure as an internal representation for executable robot motion.

Q2. Training data and checkpoints. TextOp was trained on the same HumanML3D-G1 split as PartiGen. The original DART, MDM, and AttT2M rows use their released human-domain checkpoints followed by GMR. FrankenMotion+GMR uses the released Coral79/frankenmotion generation checkpoint. For ARDY, we use nvidia/ARDY-G1-RP-25FPS-Horizon52. Its native G1 outputs are mapped to the shared 23-DOF representation and resampled from 25 to 30 fps before applying the same feature extraction and evaluator as PartiGen.

Q3. Recent comparisons. We evaluated the additional methods on the common HumanML3D-G1 test split:

MethodR@1 ↑R@3 ↑FID ↓Diversity
PartiGen0.3010.6113.1657.736
FrankenMotion+GMR0.2940.5743.5197.901
ARDY-G1 (Horizon52)0.3130.6583.3577.964
DART-G10.2190.4656.3317.813

PartiGen's values are the original multi-run results, while the new baselines are single-run evaluations. PartiGen has higher R@1 and R@3 and a 10.1% lower FID than FrankenMotion+GMR. Against ARDY-G1, PartiGen achieves 96.2% and 92.9% of its R@1 and R@3 scores, respectively, while obtaining a 5.7% lower FID. ARDY-G1 uses the 700-hour Bones Rigplay corpus, consisting of 630 hours for training and 70 hours held out for testing. HumanML3D contains 28.59 hours of distinct source motion before mirroring, and PartiGen uses its training split.

Q4. Same-domain comparison. We additionally trained DART-G1 directly on the same HumanML3D-G1 split. Its encoder and decoder architecture was retained while its motion input and output were adapted to G1. We removed the SMPL-dependent consistency loss and used the same G1 FK loss definition and coefficient as PartiGen; the remaining loss weights and original VAE/diffusion settings were retained. The resulting DART-G1 baseline is evaluated without post-generation retargeting. Together, TextOp and DART-G1 provide two directly trained G1-domain baselines under the shared dataset and evaluation protocol.

Thank you for the response and additional experiments. Overall, I appreciate the real-robot validation, but two concerns remain:

  1. The response identifies design differences from prior work, but does not sufficiently explain which specific limitations these changes address. The ablations also do not adequately establish the incremental value over the closest existing methods. Targeting G1 execution alone also does not establish methodological novelty.
  2. I share MH8F’s concerns about the additional comparisons, particularly whether the DART-G1 adaptation was sufficiently validated and whether the results establish a clear advantage over ARDY-G1. I will not repeat those points here.

My main concerns therefore remain unresolved, and I maintain my original rating.

Reviewer_BppHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2(+1)
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper studies language-conditioned motion generation for a humanoid robot, where the generated reference must coordinate body parts, have an appropriate duration, and be executable on the robot. It proposes PartiGen, a latent diffusion model trained on HumanML3D retargeted to the Unitree G1. A VAE with part-aware attention, which groups joint angles into four fixed body regions before the attention projections, encodes motion primitives into a shared latent, and a text- and history-conditioned diffusion model generates primitives autoregressively with forward-kinematics supervision. A per-frame EOS head then selects the endpoint after the full candidate is generated. References are executed by a frozen, pretrained GRIT controller in MuJoCo and on the real G1. On HumanML3D-G1 the method lowers segment-level FID from 3.80 to 3.17 relative to TextOp, leads a self-built 100-prompt compositional test, reduces duration error over a fixed cutoff, and 26 of 27 manually selected motions are judged successful on hardware by the authors.

Critical strengths
  • Proposes a part-aware attention design that groups joint angles by body region before the attention projections while keeping a single shared latent, trained directly in the robot’s joint space with forward-kinematics supervision.
  • Adds a frame-level EOS head that selects the endpoint at any frame rather than at primitive boundaries, reducing duration error from 12.5 to 3.1 frames over a fixed cutoff.
  • Reports improvements over the same-domain baseline TextOp on HumanML3D-G1 together with ablations, and closes the loop to execution with a frozen controller in simulation and on a real G1, with a candid limitations section.
Critical weaknesses
  • Part-aware attention is the core contribution, but its necessity is not shown and its design is barely examined. On necessity, the paper never demonstrates that a monolithic model fails at coordination. The “kinematic interference” asserted in the appendix comes with no evidence, the only support is a citation to AttT2M, which is the worst baseline after retargeting, and the authors state that the design does not enforce independent body-part representations, so no mechanism is offered for why grouping should help. On design, the grouping is hand-specified without justification. The legs are split left and right while both arms share one group, even though the targeted prompts, such as waving the left hand while kicking the right leg, require separating the arms. The only control is three random partitions with one run each, one of which nearly matches the full model, and the authors concede this does not establish superiority over arbitrary partitions. The limitations section acknowledges that alternative groupings were not studied, but this does not settle the matter. The method is named after its part structure, and an analysis of that structure is the minimum the paper owes. Listing it as future work leaves the central claim unsupported.

  • The duration component is a degraded version of a standard technique, evaluated only against the weakest baseline. The EOS head is a per-frame stop classifier, the continuous-output analogue of an end token, which has been standard in sequence generation for years. Yet it does not stop anything. Every sample is generated to the fixed 320-frame horizon and then truncated, so a two-second action still costs a full ten seconds of diffusion, and the authors note that this saves no computation, even though the head’s inputs are available at every step and online stopping would be a one-line change. The only comparison is a mean-length cutoff. Text-based length prediction and end tokens from prior motion work are not cited, so the reader cannot tell whether the design adds anything. The released code contains a trained text-based length predictor and duration-conditioned denoisers, which are exactly the missing comparisons, and their results are not reported.

  • The constraint motivation is inflated and the method does not deliver it. The abstract lists joint limits, dynamics, actuator capabilities and contact constraints as the gap. By the second paragraph of the introduction this has shrunk to kinematic consistency, and in the method it is a forward-kinematics reconstruction loss with no limit of any kind. Feasibility is inherited rather than achieved. Joint limits come from the retargeted training data and dynamic feasibility from the frozen third-party controller, exactly as for TextOp, so nothing in the paper’s contribution addresses the constraints it names. The paper’s own evidence points the other way. FK supervision, the only constraint-related component, lowers the simulation stability pass rate from 92.8% to 81.8%, a number that appears only in the appendix while the main text says merely “a lower pass rate”. The hardware result is a manually selected set judged by the authors, which the appendix itself states is not a success rate. A motivation this prominent needs either a method that enforces the constraints or evidence that the references are more executable than the baselines, and the paper provides neither.

Questions for authors
  • Can at least one human-domain baseline, for example DART, be trained in the G1 domain in the same way as TextOp, so that architecture and training-domain effects can be separated?
  • How are the approximate references of the compositional test assembled from part-level trajectories, and how were its 100 prompts selected?
  • The simulation screen computes joint velocity and torque saturation for every trial. Can these be reported per method, and can the upright-but-not-following failures observed for w/o FK be quantified across all methods?
  • Table 1 uses 8-frame primitives and Table 5 uses 16-frame primitives. Which configuration runs on the robot, and does deployment also generate the full 320-frame candidate before truncation?
  • Would online EOS stopping change the duration results, and what do the text-based length predictor and the duration-conditioned denoisers in the released code achieve under the Table 5 protocol?
  • The code snapshot removes every experiment setting.
2 comments
AuthorsOpenReview ↗

Response to Reviewer BppH:Part-Aware Design, Duration Modeling, and Robot Executability

We thank the reviewer for the detailed questions. The following results clarify the part-aware design, same-domain baseline, composition test, execution measurements, and deployment settings.

Part-aware design. Anatomical masks restrict the joint features used to form attention projections, while the shared latent represents the whole body. We separate the legs to preserve leg-specific support and contact information and group the arms to model coordinated upper-body motion. In Table 4, Full achieves R@3/FID/contact accuracy of 0.611/3.165/0.914, compared with 0.575/3.811/0.861 for standard attention and 0.586/3.533/0.880 for size-matched random masks. The closest individual random partition obtains R@3 0.601 and FID 3.287.

Q1. Same-domain DART. We trained DART-G1 on the same HumanML3D-G1 split and used the same evaluator as PartiGen. DART's encoder and decoder architecture was retained with G1 motion inputs and outputs. The SMPL-dependent consistency loss was removed, and G1 FK supervision uses PartiGen's definition and coefficient. The other loss weights and the original DART VAE/diffusion settings were retained. Its single-run R@1/R@3 are 0.219/0.465 and its FID is 6.331, compared with PartiGen's original multi-run values of 0.301/0.611 and 3.165.

Q2. Composition test. We constructed a larger candidate pool and retained 100 reference motions after MuJoCo tracking checks with the frozen controller. The final set contains 25 motions from the held-out test set, 50 motions assembled from body-part trajectories, and 25 motions obtained from Hunyuan-generated videos and retargeted to G1. For example, “walk while waving” combines the upper-body waving DOFs with the lower-body walking DOFs. A composite reference is retained only when it can be tracked in MuJoCo. On this 100-instruction test, PartiGen obtains R@3 0.413 and FID 6.531, compared with 0.241/9.951 for TextOp and 0.220/9.340 for DART+GMR.

Q3. Execution measurements. The same frozen GRIT controller and execution protocol are used for all methods. Full lowers mean global MPJPE from 275.61 mm without FK supervision to 252.40 mm with FK supervision. The posture-screen rates are 92.8% without FK and 81.8% with FK over 500 requests. Compared with TextOp, Full has lower global MPJPE (252.40 vs. 295.85 mm) and a higher posture-screen rate (81.8% vs. 79.2%). The execution protocol reports global tracking error and posture-screen rate; the latter evaluates pelvis height and body inclination under the same thresholds. The hardware evaluation contains 27 attempted motions, of which 26 were judged successful.

Q4. Primitive length and hardware. Table 1 uses 8-frame primitives. All Table 5 variants and physical G1 demonstrations use 16-frame primitives. Hardware deployment generates the full 320-frame candidate and then applies frame-level EOS endpoint selection.

Q5. Duration alternatives. Oracle Length supplies the annotated duration as an upper-bound comparator. The text-based length predictor was an exploratory variant with performance close to the EOS model. We use the jointly trained EOS head because it estimates the endpoint from the same text- and history-conditioned generation process. Against the mean-length cutoff, Table 5 shows duration error decreasing from 12.471 to 3.080 frames, R@3 increasing from 0.584 to 0.599, and FID decreasing from 4.317 to 3.207.

Q6. Experimental settings. All reported training runs follow the settings specified in the paper and supplement. Appendix S2 specifies the architectures, losses and weights, diffusion setup, and EOS configuration. Appendix S3 specifies the dataset split and evaluation protocol.

Thank you for the rebuttal. It resolves part of my concerns, but several important issues remain.

  1. There are still no comparisons with other plausible body-part grouping way, which is insufficiently justified.

  2. Although the rebuttal explain the choice of the EOS head, it doesn't provide quantitative comparisons with the text-based length predictor.

  3. There are still no specific data about robot-level measurements, such as, joints or torque, to further support the benefits of the proposed method.

Therefore, I maintain my original score.

Reviewer_MH8FOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

PartiGen generates language-conditioned Unitree G1 motion using a two-stage model: a VAE with anatomically masked attention heads and a shared latent, followed by history/text-conditioned latent diffusion with FK supervision and post-generation EOS endpoint selection. It is evaluated on retargeted HumanML3D, a custom compositional test, 500-motion MuJoCo evaluations, and a screened set of 27 physical-robot trials. Results show that it achieves best performance on almost all metrics.

Critical strengths
  • Well-isolated internal evidence for anatomical grouping. Full PartiGen consistently improves over both standard attention and capacity-matched random masks in retrieval, FID, jerk, and contact accuracy (L447–466). It also outperforms every individual random partition reported in L773–796. This directly supports the paper’s central design choice.
Critical weaknesses
  • The baseline suite is neither sufficiently controlled nor current. Only TextOp is a 2026 same-domain baseline, DART is from 2025, while MDM and AttT2M are from 2023, and the latter three are evaluated only after human-to-G1 retargeting (L324–336). The submission omits FrankenMotion (CVPR 2026), the closest work on part-level composition, ActionPlan (ECCV 2026), a strong recent model for streaming and long sequential text-conditioned motion, and ARDY (SIGGRAPH 2026), a real time auto-regressive motion model. All of them were public before submission and could be evaluated through the same GMR pipeline. Consequently, the experiments are not fully convincing.

  • Table 1 confounds model quality with retargeting error. PartiGen is trained and evaluated directly in the G1 domain, whereas DART, MDM, and AttT2M generate human motion that undergoes an additional SMPL to G1 retargeting conversion pipeline (L324–329, L1059–1081). Because no before/after-GMR degradation analysis or same-domain retraining is provided, their performance gaps cannot be attributed to PartiGen’s part-aware representation alone. This is especially consequential for AttT2M, the closest architectural baseline. Only TextOp offers a same-domain comparison.

Questions for authors
  • Were DART, MDM, and AttT2M retrained directly on the retargeted HumanML3D-G1 training set, or were human-domain outputs simply passed through GMR? L324–329 and L1059–1066 indicate the latter. If so, how can Table 1 distinguish improvements from the proposed model from degradation introduced by representation conversion and retargeting? A same-domain retraining comparison, or minimally, an evaluation quantifying each baseline before and after GMR is necessary to support the architectural comparison.

  • supp video 4:21: the robot steps back while the text input is standing, does this suggest the generation motion is not stable?

2 comments
AuthorsOpenReview ↗

Response to Reviewer MH8F: Baseline Fairness and Additional G1-Domain Comparisons

We thank the reviewer for recognizing the anatomical-grouping controls and for suggesting recent and same-domain comparisons.

Q1. Baseline settings and additional experiments. TextOp was trained on the same HumanML3D-G1 split as PartiGen. The original DART, MDM, and AttT2M rows used their released human-domain checkpoints followed by GMR. We additionally evaluated FrankenMotion+GMR and ARDY-G1-RP-25FPS-Horizon52, and trained DART-G1 directly on HumanML3D-G1. All results below use the same test split and G1 evaluator:

MethodR@1 ↑R@3 ↑FID ↓Diversity
PartiGen0.3010.6113.1657.736
FrankenMotion+GMR0.2940.5743.5197.901
ARDY-G1 (Horizon52)0.3130.6583.3577.964
DART-G10.2190.4656.3317.813

PartiGen's values are the original multi-run Table 1 results, while each added baseline is a single-run evaluation. Compared with FrankenMotion+GMR, the closest recent part-compositional baseline, PartiGen achieves higher R@1 and R@3 and a 10.1% lower FID. Compared with ARDY-G1, PartiGen reaches 96.2% and 92.9% of its R@1 and R@3 scores, respectively, while obtaining a 5.7% lower FID.

ARDY-G1 uses the 700-hour Bones Rigplay corpus, consisting of 630 hours for training and 70 hours held out for testing. HumanML3D contains 28.59 hours of distinct source motion before mirroring, and PartiGen uses its training split. Thus, PartiGen obtains competitive retrieval quality and lower FID using a substantially smaller source-motion corpus.

ARDY-G1 generates motion directly in the G1 domain. Its converter maps 25-fps local rotations and root positions to a root pose and 29 joint angles. We retain indices [:19] and [22:26] for the common 23-DOF representation, reorder root quaternions from wxyz to xyzw, and resample from 25 to 30 fps. Positions and joint angles use linear interpolation, root rotations use SLERP, and foot contacts are recomputed with the same displacement-based rule as PartiGen before applying the same 57-dimensional feature extraction.

DART-G1 directly addresses the training-domain and post-generation-retargeting differences. We retain DART's encoder and decoder architecture while adapting their motion input and reconstruction output to G1. We remove the SMPL-dependent consistency loss and use PartiGen's G1 FK loss definition and coefficient. The remaining loss weights and DART's original VAE and diffusion settings are retained. DART-G1 uses the same HumanML3D-G1 training split and is evaluated without post-generation GMR. Together with TextOp, it provides a direct G1-domain comparison in the same executable robot representation.

Q2. Supplementary video. The backward movement occurred after the generated action finished. We replayed the original reference in simulation, generated and tested a new trajectory for the same prompt in simulation, and replayed the original reference in two additional hardware trials. The behavior did not recur in any of these checks. The deployment immediately preceding the recording was also normal.

Thank you for the response and additional experiments. They however do not address my concerns. In particular:

  1. DART-G1 vs. DART+GMR. The same-domain baseline DART-G1 (R@1 0.219, R@3 0.465, FID 6.331) is worse on the reported metric than the original DART+GMR pipeline (0.230 / 0.486 / 4.837). This result does not make sense to me. Unless another model shows that retraining is actually worse than human gen + retargeting.
  2. ARDY-G1 result is now better than PartiGen. I do not find the 5.7% FID gap or the corpus-size argument persuasive. Given this result, it is not convincing that the proposed part aware representation is actually necessary and effective.

therefore, i maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.