acceptodds
Under review as a conference paper at ICLR 2027

G2E-T2M: Global-to-Event Hierarchical Conditioning for Compositional Text-to-Motion Generation

Abstract

Text-to-motion (T2M) models have achieved strong caption-level alignment, yet they often fail to realize individual actions in complex multi-event descriptions. Through extractive event-level analysis of standard T2M benchmarks, we observe that descriptions containing more constituent events are generally associated with longer motions and degraded generation performance. Motivated by these observations, we propose , a Global-to-Event hierarchical conditioning framework for compositional T2M generation. G2E-T2M first conditions motion features on the full-caption tokens through global cross-attention and then performs independent token-level cross-attention with each constituent event. The resulting event-wise feature updates are adaptively aggregated at each motion position according to motion–event relevance and a weak order prior, separating fine-grained interaction within each event from competition across events. In addition to the standard generation objective, we introduce boundary-free event-wise semantic supervision that aligns each constituent event with semantically compatible local motion regions through multi-scale temporal search without requiring event-level temporal annotations. For event-level evaluation, we introduce Event MM-Dist with average- and worst-event aggregation to measure constituent-event alignment, derive event-order consistency from localized event–motion matches, and employ a VLM-based score to assess event realization. Experiments across HumanML3D, KIT-ML, and SnapMoGen demonstrate consistent improvements in both caption- and event-level semantic alignment while maintaining competitive motion quality across different levels of event complexity.

Primary area:
generative models
Keywords:
Text-to-Motion Generation, Compositional Generation, Multi-Event Modeling, Motion-Language Alignment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vqsROpenReview ↗
Rating
2(−1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper addresses an emerging issue in the T2M field and attempts to resolve it effectively. While extensive experiments demonstrate the superiority of the proposed methodology, the experimental setup has some shortcomings. Overall, the paper is not particularly easy to read.

Critical strengths
  1. The problem formulation is good.
  • Although the T2M field has become quite standardized and is currently seeing a flood of research papers, its practical application in the industry remains limited. This is because, while state-of-the-art models excel at generating relatively simple motions, there has been comparatively little research into generating the complex, long-duration motions required by the industry. This paper directly addresses this gap and makes a valuable contribution to the motion research community.
  1. The boundary-free event-wise semantic supervision methodology is intuitive.
  • It effectively addresses an area within the T2M field that has not yet been extensively researched. The idea of ​​identifying individual events without the need for labels—and then using them for supervision—is, in my view, highly practical.
  1. Including experiments on SnapMoGen is an good addition.
  • While it is standard practice in the T2M field to evaluate models solely on HumanML3D and KIT-ML, expanding the benchmark suite is a positive step. This effectively demonstrates the model's generalization performance.
  1. Create additional event-specific metrics to effectively capture aspects that existing metrics fail to measure.
Critical weaknesses
  1. Absence of videos and user studies. In the field of human motion, human evaluation is essential to determine whether the motion is truly "human-like." As it stands, no motion videos have been provided, making an accurate assessment difficult.

  2. The paper is very difficult to read.

  • It appears that all citations have been formatted using \citep, which makes the text feel quite awkward. Please use \citet appropriately where needed.
  • Some terms appear suddenly without any explanation (e.g., "R@1"). Since readers unfamiliar with this specific field might find this confusing, please provide a brief explanation when a term is first introduced.
  • Section 2.1 ("Related Work") is written in a rather perfunctory manner. It essentially just lists works in the T2M (Text-to-Motion) field, and the paper could be understood perfectly well even without this section. Either expand on it with more detail or merge it with Section 2.2.
  • The presentation of mathematical formulas is not reader-friendly. Symbols are often introduced without clear definitions, forcing the reader to infer their meanings. (For instance, in Section 3.2, a symbol previously denoted as "T" suddenly appears as "L," and it is not at all clear what the distinction is—this issue persists throughout the paper.)
  1. Somewhat Incomplete Experimental Setup
  • SnapMoGen does not include an experiment analyzing performance variations based on the number of events—a key claim of the paper. While the intention behind using diverse benchmarks with a higher number of multi-event sequences is clear, conducting a simple additional experiment—which would not require further training—would strengthen the validity of the paper. Furthermore, the study lacks a multi-modality component. It also omits results for the full KIT-ML dataset, as well as for subsets with more than 3 (E>3) or 4 (E>4) events.
Questions for authors
  1. Could you conduct a user study with at least 20 participants and provide statistical validation?
  2. Please include experiments regarding "events" (as in SnapMoGen).
  3. Please add the "MModality" metric.
  4. Please include performance results for "full," "E>3," and "E>4" on the KIT-ML dataset.
  5. It is my understanding that the feature extractor developed by Guo is biased. Could you re-evaluate the performance using the feature extractor from MARDM instead?
  6. Please include a comparison of latency and the number of model parameters.
3 comments
AuthorsOpenReview ↗

Response to Reviewer vqsR

We thank the reviewer for the constructive feedback.

Videos and Human Evaluation

We added continuous motion comparison videos of MDM, MoMask, Event-T2M, SALAD, and G2E-T2M to the supplementary material, with annotations showing which constituent events are realized.

Following Event-T2M's perceptual evaluation criteria, we conducted a study with 20 participants (10 male, 10 female) and 512 motion samples. We used a 7-point Likert scale for Fidelity, Order Alignment, and Naturalness and report means and standard deviations.

MethodsFidelityOrder alignmentNaturalness
GT6.013 ± 0.1525.889 ± 0.1805.115 ± 0.167
Event-T2M5.322 ± 0.0975.304 ± 0.1624.771 ± 0.092
CMDM5.561 ± 0.1215.211 ± 0.1174.852 ± 0.157
Ours5.712 ± 0.1075.565 ± 0.0894.966 ± 0.138

G2E-T2M achieves the highest mean ratings among the evaluated generators. Compared with CMDM, it improves Fidelity, Order Alignment, and Naturalness by 0.151, 0.354, and 0.114, respectively, supporting improved event realization and temporal consistency.

Citation Formatting

We revised citations throughout the manuscript, using \citet{} for narrative citations and \citep{} for parenthetical citations to improve readability in Related Work and the method section.

SnapMoGen

G2E-T2M improves R@1 and FID over CMDM (Base) on every event-count subset.

MethodE≥2 R@1 ↑E≥2 FID ↓E≥3 R@1 ↑E≥3 FID ↓E≥4 R@1 ↑E≥4 FID ↓E≥5 R@1 ↑E≥5 FID ↓
Base0.81115.1420.79715.8100.81016.0560.79716.519
Ours0.83414.5560.81614.7350.82915.4150.81915.868

KIT-ML

We report MM-Dist and FID for the full cohort and event-count subsets. E≥4 has 10 samples, below the Guo evaluator's 32-candidate requirement for R-Precision. No samples have E≥5.

SetSamplesOurs MM-Dist ↓Ours FID ↓Event-T2M MM-Dist ↓Event-T2M FID ↓CMDM MM-Dist ↓CMDM FID ↓TriC-Motion MM-Dist ↓TriC-Motion FID ↓
Full7042.44040.30852.69290.13412.49910.18362.50340.1873
E≥21682.74420.45293.08740.55072.83030.52862.86490.5552
E≥3802.71471.15192.69110.69332.54830.86262.63030.8683
E≥4102.70884.74374.01062.66273.81922.82284.05433.9757

G2E-T2M achieves the lowest MM-Dist on the full cohort and the lowest MM-Dist and FID at E≥2. Both metrics trail the baselines at E≥3. At E≥4, G2E-T2M has the lowest MM-Dist and highest FID. The E≥4 distributional metrics should be interpreted with caution given the small sample.

Multimodality

MModality measures motion diversity for the same text description.

DatasetCMDMSALADEvent-T2MTriC-MotionMoScaleMoMaskOurs
HumanML3D1.5751.8881.0321.3410.9231.3301.534
KIT-ML1.3351.1990.8980.9691.3371.4181.347
SnapMoGen11.08512.2026.17310.7466.3157.49210.819

G2E-T2M remains competitive on all three benchmarks, suggesting that improved event alignment can coexist with diverse motions per prompt.

MARDM Evaluation

We re-evaluated CMDM and G2E-T2M with MARDM's 67-dimensional essential-feature evaluator (CVPR 2025), using its motion representation and normalization statistics. We kept the generated motions, captions, and retrieval pools fixed for the re-evaluation.

SubsetModelR@1 (%) ↑FID ↓MM-Dist ↓Avg-E ↓Worst-E ↓
E≥2Base53.8281.2463.0154.5126.162
E≥2Ours54.4971.0212.9754.4956.151

G2E-T2M gains 0.669 percentage points in R@1, lowers FID from 1.246 to 1.021, and slightly improves MM-Dist, Avg-E, and Worst-E, supporting gains beyond the Guo evaluator.

Efficiency

We tested CMDM and G2E-T2M under matched inference settings on an NVIDIA RTX 5090, using 196-frame motions with three constituent events.

MethodGenerator Params (M)Total Params (M)Batch 1 mean (s)Batch 1 variance (s²)Batch 16 mean (s)Batch 16 variance (s²)
CMDM38.697181.37245.5422.30245.9212.065
G2E-T2M42.908185.58346.2392.43546.6642.219

G2E-T2M adds 4.21M trainable parameters (+10.9% generator, +2.3% total). Mean latency increases by about 1.5% and 1.6% at batch sizes 1 and 16, respectively.

Reviewer_vqsROpenReview ↗

Response to Author

Thank you for your diligent efforts in the rebuttal process. I have reviewed the revised manuscript; however, as there were no indications or notes specifying where or how the changes were incorporated, it was quite difficult to locate each one. My responses are as follows:

  1. Videos
  • There are discrepancies in the ✓/✗ classifications between the videos and the captions provided in the paper’s figures. For instance, in Fig. 3(a), the event "Does not touch toes" is labeled for Event-T2M, yet the corresponding video is marked with a ✓.
  1. User Study
  • The requested statistical validation is missing. Consequently, it is impossible to determine whether the reported results are statistically significant.
  1. Readability
  • The citation format and mathematical notation issues have been resolved.
  • The term "R@1" appears early on without any explanation; adding a brief definition would improve readability.
  • My comments regarding the "Related Works" section have not been addressed. The paragraph on text-to-motion still reads more like a simple list of methods.
  1. SnapMoGen Experiments
  • The subset E≥2E \ge 2 accounts for 99.78% (3,991/4,000) of the test set, yet the figures differ significantly from those in Table 2 (Full). An explanation is required.
  1. MModality
  • MModality has addressed my concerns regarding diversity. Thank you.
  1. KIT-ML Experiments
  • The values ​​for the E≥2E \ge 2 row differ between the table provided in your response (CMDM: 2.8303/0.5286) and Table 12 in the revised manuscript (2.940/0.601).
  • Compared to Event-T2M, G2E-T2M shows significantly lower performance in terms of FID for the "Full," E≥3E \ge 3, and E≥4E \ge 4 categories.
  1. MARDM Experiments
  • Results from other comparative papers are missing.
  • Only E≥2 is available.
  1. Efficiency
  • I am curious whether the cost of LLM event decomposition during inference has been accounted for.

Although most of the requested experiments were conducted, several instances of discrepancies between the reported numerical values ​​and the corresponding conclusions were identified in the additional results. This has, in fact, raised further concerns regarding the overall reliability of the experiments. Consequently, I am adjusting the score to "weak reject."

AuthorsOpenReview ↗

Response to Reviewer vqsR

Thank you for the careful review. We apologize for the annotation and reporting inconsistencies. We summarize the revisions below.

1. Videos and figure annotations.

Fig. 3 (p. 8) uses sparsely sampled mesh frames that may miss brief movements. For Event-T2M in Fig. 3(a), we replaced the 6.40 s frame with the 6.00 s pose from the same sequence and changed “Does not touch toes” to “Brief toe-directed reach,” without claiming toe contact. We also checked other main-paper visualizations against their videos and corrected affected frames and annotations.

2. Statistical validation.

From the available means and SDs of participant-average ratings (n = 20), we derived conservative paired-t p-value bounds; the original paired records were unavailable. After Holm correction, the upper bounds are below 0.01 for all three criteria against Event-T2M, and 0.0203 (Fidelity), 2.46 × 10⁻⁶ (Order Alignment), and 0.1126 (Naturalness) against CMDM. These results depend on paired-t assumptions and do not establish a Naturalness advantage over CMDM (p. 31, lines 1640–1647 and footnote 1).

3. Readability and Related Work.

We defined R@1 at its first occurrence (p. 1, lines 48–49) and revised Sec. 2 (p. 3, lines 121–161) to compare generation strategies, event representations, and semantic supervision.

4. SnapMoGen.

Our previous response inadvertently included results from an experimental setting with randomly combined prompts and extended target motion lengths. The corrected results follow the evaluation protocol used in Table 2 (Table 11, p. 29). Full and E ≥ 2 differ by at most 0.1 percentage points in R@1 and 0.005 in FID.

SubsetCMDM R@1 (%)Ours R@1 (%)CMDM FIDOurs FID
Full82.485.315.00813.972
E ≥ 282.485.215.00913.967
E ≥ 382.185.115.20614.136
E ≥ 482.985.515.94314.803
E ≥ 582.285.316.66815.714

5. KIT-ML.

Due to time and computational constraints, the additional KIT-ML evaluations used 10 repetitions, matching CMDM’s repetition count (Yu et al., CVPR 2026). CMDM’s previously reported MM-Dist/FID of 2.8303/0.5286 came from this evaluation. Table 12’s E ≥ 2 row reuses Table 1’s 20-repetition results (2.940/0.601); the other rows use 10 repetitions. The caption now specifies these counts (p. 29, lines 1548–1550).

We acknowledge higher FID than Event-T2M on Full, E ≥ 3, and E ≥ 4. Our best-window event loss does not explicitly supervise durations or transitions, but its contribution to the FID gap remains unverified. The E ≥ 3 and E ≥ 4 subsets contain only 80 and 10 samples, respectively, limiting FID interpretation without explaining the Full result. G2E-T2M has the lowest MM-Dist and FID on E ≥ 2 in both evaluations, but improvements are not uniform across subsets.

6. MARDM evaluation.

We added comparisons with ten baselines on HumanML3D and KIT-ML, reporting retrieval and event-level metrics, FID, MM-Dist, and Diversity with pointwise 95% CIs (Tables 16–17, p. 32). This MARDM-based analysis covers the complete E ≥ 2 subsets only.

7. Inference cost.

Table 18 reports motion-generation latency with precomputed event descriptions. Offline LLM decomposition is excluded; new prompts incur additional preprocessing cost (p. 33, lines 1753–1754).

We appreciate your consideration.

Reviewer_GgkvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents G2E-T2M, a hierarchical conditioning mechanism that moves from the overall description to individual events to address omitted actions in complex text descriptions. The method first extracts ordered events from the original description, then applies token-level cross-attention separately to the full description and within individual events, and aggregates event information using content relevance and a weak ordering prior. During training, a frozen text-motion retrieval model finds the best semantic match for each event across temporal windows at multiple scales in the predicted motion, providing local supervision without annotated event boundaries. The authors report overall and event-level evaluations on HumanML3D, KIT-ML, and SnapMoGen, with multiple ablations examining the conditioning structure, event representations, and auxiliary supervision.

Critical strengths
  1. The research problem is clearly defined, and the method is designed around realizing individual events.
  2. The experiments cover additional conditioning modules, hierarchical versus concatenated attention, event representation granularity, different auxiliary supervision objectives, and the choice of supervision model, helping explain the role of each design.
  3. This combination of semantic decomposition and flexible temporal allocation offers an promising approach to multi-event motion generation.
Critical weaknesses
  1. The paper's main conclusions rely on quantitative comparisons with existing methods, but the evaluation protocol and sources of baseline results are not described clearly enough. For example, the official CVPR version of SALAD report a Top-1 retrieval accuracy of 0.581 on HumanML3D. This value does not appear to fully correspond to the 0.579 listed for the full test set in the appendix.
  2. The review materials do not include supplementary generated videos or animations. Static keyframes alone make it difficult to determine whether all events are realized, whether their temporal relationships are correct, and whether transitions are natural.
  3. The weak ordering prior is a specific design that affects event timing, and its role has not been established through an experiment that removes it in isolation.
Questions for authors
  1. Were all baseline results on the full test set and the multi-event subset recomputed using a unified evaluation pipeline? Which model or evaluation settings mainly account for the differences between the reported results for baselines such as SALAD and TriC-Motion and those in their public papers?
  2. Please specify whether each result comes from a published paper or a local reproduction under a unified setup, so that comparability across methods can be assessed.
  3. I recommend providing continuous animations from the proposed method and major baselines under the same text conditions, labeled with the descriptions and event decompositions, to allow direct inspection of event completion, temporal relationships, and transition naturalness.
3 comments
AuthorsOpenReview ↗

Response to Reviewer Ggkv

Clarification of Baseline Evaluation Protocol and Result Sources

We thank the reviewer for raising this concern. All baseline results on HumanML3D and KIT-ML were obtained through local evaluation using the official implementations under a unified evaluation protocol, rather than directly taken from the original publications. Results are reported as mean ± 95% confidence interval over 20 generation seeds. Differences from published results (e.g., SALAD's R@1) may arise from stochastic sampling and variations in evaluation settings.

For SnapMoGen, we evaluate the full test set of 4,000 captions, of which 3,991 (99.78%) contain multiple events (Appendix A.1). Retrieval-based evaluation uses 3,900 samples forming complete 100-way candidate pools. Results for MoScale (CVPR 2026), SALAD (CVPR 2025), Event-T2M (ICLR 2026), TriC-Motion (ICLR 2026), and CMDM (CVPR 2026) were likewise obtained through local evaluation under a unified protocol. We will clarify these evaluation details and result sources in the revised manuscript.

Continuous Motion Visualization and Qualitative Comparisons

We thank the reviewer for this suggestion. We have added continuous motion comparison videos of MDM, MoMask, Event-T2M, SALAD, and G2E-T2M to the supplementary material. Each video includes the extracted constituent-event descriptions and annotations highlighting successfully realized or missing events, enabling direct inspection of event completeness and temporal progression beyond static visualizations.

Effect of the Gaussian Order Prior

We thank the reviewer for this suggestion. We conducted additional ablations on HumanML3D (E≥2 and E≥3) by removing the Gaussian order prior (λp=0\lambda_p=0) while keeping all other components unchanged. Results are reported as mean ± 95% confidence interval over 20 generation seeds.

SubsetVariantR@1 ↑FID ↓MM-Dist ↓Order (%) ↑
E≥2w/o Prior0.582 ± 0.0020.067 ± 0.0012.678 ± 0.00772.65 ± 0.58
E≥2Ours0.586 ± 0.0030.065 ± 0.0032.653 ± 0.01173.12 ± 0.49
E≥3w/o Prior0.531 ± 0.0040.104 ± 0.0062.736 ± 0.01470.03 ± 0.61
E≥3Ours0.553 ± 0.0060.096 ± 0.0072.703 ± 0.01271.18 ± 0.31

Removing the prior reduces Order by 0.47 and 1.15 percentage points on E≥2 and E≥3, respectively, alongside degradation in R@1, FID, and MM-Dist. These results indicate that the Gaussian order prior provides complementary temporal guidance beyond learned event relevance, contributing to both event ordering and overall text–motion alignment.

The authors have addressed the concerns on the source of reported results. According to the ablation studies on prior, it has some benefits, but the improvements brought by the weak order prior are not that obvious. According to the response, I still have questions about the "± 95% confidence", and does it imply being within two sigma (2$\sigma$)?

AuthorsOpenReview ↗

Response to Follow-up Questions on Gaussian Order Prior and 95% CIs

We thank the reviewer for the follow-up questions. We provide the following clarifications.

1. Effect and Intended Role of the Gaussian Order Prior.

We agree that the overall quantitative improvements brought by the Gaussian order prior are relatively modest. We would like to clarify that this component is primarily designed to address challenging recurrent or alternating event configurations, such as A→B→A, A→B→A→B, and A→A→A, rather than uniformly improve all multi-event sequences. In these cases, semantically similar event occurrences may be difficult to distinguish based on content relevance alone. The Gaussian prior introduces a weak temporal positional bias into event-to-motion relevance estimation, helping distinguish their relative temporal locations without imposing explicit temporal boundaries or fixed durations.

To further characterize these configurations, we analyzed the HumanML3D test split. Among the 11,516 successfully decomposed captions out of 13,114 test captions, we identified 358 captions containing ABA patterns (3.11%), 42 containing ABAB patterns (0.36%), and 113 containing AAA patterns (0.98%). These patterns represent relatively uncommon temporal configurations in the overall test corpus. Their targeted benefits may therefore be less apparent in aggregate metrics.

Consistent with its intended role, removing the prior decreases the Order score by 0.47 and 1.15 percentage points on the E≥2 and E≥3 subsets, respectively. We therefore view the Gaussian order prior as a complementary temporal inductive bias for challenging recurrent and alternating event configurations, rather than a primary source of overall improvement.

2. Clarification of the 95% Confidence Intervals.

We thank the reviewer for pointing out the need for clarification. To maintain consistency with the statistical reporting format used in the main paper, we report mean performance with 95% confidence intervals (mean ± 95% CI).

Specifically, for the 20-run results discussed here, the reported ± values represent the half-widths of two-sided 95% Student-t confidence intervals, rather than ±2 standard deviations (SD). We calculate:

xˉ±t0.975,19s20,\bar{x}\pm t_{0.975,19}\frac{s}{\sqrt{20}},

where xˉ\bar{x} and ss denote the sample mean and sample standard deviation across the 20 runs, respectively. Since t0.975,19≈2.093t_{0.975,19}\approx 2.093, the confidence-interval half-width is approximately 2.093 standard errors (SE), or 0.468 times the sample SD, rather than 2 SD.

We will explicitly clarify this statistical definition in the revised manuscript.

We thank the reviewer again for these helpful comments.

Reviewer_iCjdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes G2E-T2M for compositional text-to-motion generation with multi-event descriptions. It decomposes a caption into constituent events, applies global caption conditioning followed by event-wise token-level cross-attention, and introduces boundary-free local semantic supervision through multi-scale temporal matching. The paper also proposes event-level metrics to evaluate constituent-event realization and temporal ordering.

Critical strengths
  1. The paper targets a meaningful failure mode of current T2M models: strong caption-level alignment can still miss individual actions in complex multi-event descriptions.
  2. The method is technically coherent and well motivated. The hierarchical conditioning and event-wise supervision are directly tied to the identified problem of incomplete event realization.
  3. The experimental evaluation is relatively comprehensive. The paper includes multiple datasets, event-complexity subsets, component ablations, supervision-model robustness tests, and both caption-level and event-level evaluation.
  4. The paper makes a useful effort to evaluate constituent events explicitly, rather than relying only on global text–motion metrics that may hide omitted actions.
Critical weaknesses
  1. Several key ingredients have close precedents in recent T2M work, which limits the conceptual novelty. Event-T2M already decomposes complex prompts into constituent events and injects event-level conditions through cross-attention for multi-event motion generation. AToM has explored event-level alignment and explicitly evaluates motion integrity and temporal relationships, while ATM introduces local semantic alignment without requiring localized text annotations. Given this overlap, it is unclear whether the proposed hierarchical conditioning and local matching constitute a sufficiently distinct methodological advance.
  2. The source of the reported temporal-order improvement is not fully disentangled. Event aggregation explicitly incorporates a Gaussian order prior, yet no ablation appears to isolate the contribution of learned motion–event relevance from this handcrafted positional bias.
  3. The reliability of the event decomposition and temporal evaluation remains insufficiently established. The LLM-generated decomposition is verified for extractive validity but not against human semantic annotations or alternative decomposition granularities. Meanwhile, the Order metric is derived from representation-dependent best-window matches, and the paper’s own VLM temporal diagnostic is shown to be unstable.
Questions for authors
  1. Could you report an ablation without the Gaussian order prior ((\lambda_p=0)), or relevance-only/prior-only variants, to clarify how much temporal organization is actually learned?
  2. How sensitive is the method to the semantic quality and granularity of the LLM-generated event decomposition? Have you compared against human annotations, alternative LLM decompositions, or perturbed decompositions?
  3. Given that best-window matches are representation-dependent and the VLM-based temporal-order diagnostic is unstable, how confidently can the reported Order improvements be interpreted as better temporal organization?
  4. How does the softmax-based event competition handle concurrent or overlapping events that should be realized simultaneously?
2 comments
AuthorsOpenReview ↗

Response to Reviewer iCjd

Methodological Novelty

Hierarchical event conditioning. Unlike Event-T2M (ICLR 2026), which represents each event with a pooled embedding, G2E-T2M preserves token-level semantics through independent event-specific cross-attention before adaptively aggregating event contributions at each motion position. This explicitly decouples intra-event semantic interactions from inter-event relevance estimation. Supplementary Table 8 demonstrates its benefit, reducing Worst-E from 5.624 to 5.564 compared with pooled-event conditioning.

Boundary-free event-wise supervision. While AToM employs VLM-based preference optimization and ATM relies on inferred local semantic proxies, G2E-T2M introduces direct event-wise semantic supervision without requiring temporal boundary annotations. This is particularly important because motion realization is temporally non-unique: the same event, such as walking forward, may span 5 or 100 frames while remaining semantically valid. Consequently, prescribing fixed event boundaries or durations would impose arbitrary temporal constraints. Instead, we search for the most semantically compatible motion region for each event across multiple temporal locations and scales, providing flexible event-specific supervision without predefined event-to-frame assignments. Table 4 shows that incorporating this objective improves R@1 from 0.579 to 0.586.

Order Prior and Relevance

HumanML3D E≥2 results (mean ± 95% CI over 20 generation seeds). See Table 14 in Appendix A.9.2 for E≥3.

VariantR@1 ↑FID ↓MM-Dist ↓Order (%) ↑
Base0.576±0.0070.098±0.0052.706±0.00871.13±0.46
Prior Only0.581±0.0030.068±0.0022.683±0.09072.75±0.42
Relevance Only0.582±0.0020.067±0.0012.678±0.00772.65±0.58
G2E-T2M0.586±0.0030.065±0.0032.653±0.01173.12±0.49

The full model outperforms both single-component variants. Removing the prior lowers Order by 0.47 and 1.15 percentage points on E≥2 and E≥3, respectively.

Event Decomposition

Following the event definition in Event-T2M, we adopt a strictly extractive decomposition strategy: each event must be a verbatim, contiguous, and non-overlapping span of the original caption, preserving its textual order without introducing additional content.

To assess reliability, we repeated the Qwen-based decomposition three times for all 2,308 HumanML3D multi-event test captions, with 2,156 (93.4%) yielding identical decompositions across all runs. We further conducted a manual review of 300 randomly sampled captions, of which 292 (97.3%) were judged consistent with our event criteria. The remaining discrepancies primarily involved ambiguous event granularity, particularly repeated actions.

Order Metric

We emphasize that Order is a coarse diagnostic of relative event sequencing, rather than a definitive measure of temporal correctness.

Specifically, each event is independently matched to its most semantically compatible motion window using the frozen Guo evaluator, without imposing ordering constraints. Order is then computed from the relative positions of the matched window centers. To assess its reliability, three human annotators independently evaluated 300 randomly sampled multi-event cases. Automatic Order judgments agreed with human assessments in an average of 248 out of 300 cases (82.7%), providing preliminary evidence of agreement.

We further examined VLM-based temporal evaluation through an A/B-swap consistency test. Qwen3.5-9B produced contradictory judgments in 26/63 (41.27%) comparisons, while Qwen3.8-27B yielded contradictions in 22/49 (44.90%) comparisons with valid binary responses in both queries (14 additional comparisons contained null responses). Although this test measures internal consistency rather than temporal accuracy, it reveals substantial sensitivity to query presentation. Accordingly, VLM-E measures event observability, whereas Order independently assesses relative temporal consistency through localized motion matching.

Concurrent Events

Our event aggregation employs soft weighting rather than mutually exclusive event selection. See Eq. (3) in Sec. 3.2. Thus, multiple events can contribute to the same motion position without requiring hard assignments or disjoint temporal segments.

For concurrent actions, our extractive decomposition preserves semantically coherent simultaneous actions within a single event whenever possible, allowing their joint semantics to be modeled through token-level cross-attention. When represented as separate events, their responses can still be combined through adaptive aggregation. The Gaussian order prior provides only a soft temporal bias rather than enforcing strict separation.

We acknowledge that softmax normalization introduces competition among event weights and does not guarantee correct realization of overlapping actions.

Thank you for the thorough rebuttal and additional experiments. The order-prior ablations and validation of event decomposition and temporal evaluation address several of my concerns. However, my main reservation regarding conceptual novelty relative to existing event-based T2M methods remains. While the approach is technically sound, I am not fully convinced that the methodological advances are sufficiently substantial. I will maintain my rating of 2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.