- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes EAPM for text-to-3D human motion retrieval. Considering a human motion sequence often contains multiple distinct action events, EAPM introduces an event-aware prototype aggregation module to aggregates text tokens and motion tokens into multiple event prototypes, which is well motivated. The authors also introduce a variance loss and prototype-based event soft alignment to encourage different prototypes to attend to different content, thereby improving semantic diversity among prototypes. The authors conduct text-to-motion and motion-to-text retrieval experiments on HumanML3D and KIT-ML, where EAPM achieves relatively consistent improvements over existing methods.
Critical strengths
- The motivation is clear. This paper points out that a motion sequence often contains multiple distinct action events. This observation provides reasonable motivation for the event-aware formulation.
- The experimental improvements are consistent across datasets. EAPM improves performance on both text-to-motion and motion-to-text retrieval tasks on HumanML3D and KIT-ML, suggesting that EAPM contribute positively to the final performance.
- EAPM requires less inference time, memory, FLOPs, and fewer parameters while achieving better retrieval performance. The trade-off between computational cost and performance improvement is reasonable.
Critical weaknesses
- The definition of action event and event count is unclear. Since event count are used to support the core motivation of the paper, the authors should provide a much clearer description of the action event.
- The paper uses K=3, because the average number of events in HumanML3D and KIT-ML is approximately 3.14 and 2.88, respectively. However, the actual number of actions can vary substantially across motion sequences. Using a fixed number of event prototypes remains heuristic.
- There is insufficient quantitative evidence showing that the learned prototypes truly correspond to semantically meaningful action events.
Questions for authors
- According to the paper, the event count is learned by the model through a weakly supervised approach. Could the authors clarify whether the learned event count is consistent with human perception? Furthermore, how does the model handle ambiguous cases where the boundaries between different events are not clearly defined?
- In practice, could there be cases where a motion sequence is considered to contain three events from a human perspective, but EAPM decomposes it into only two or four events; or a motion sequence is supposed to contain 2 or 4 event, but EAPM decomposes it into 3 events? If such cases exist, how does the proposed method handle this discrepancy? The authors are encouraged to provide further discussion on this issue.
If the authors can address the issues I raised in the weaknesses and questions, I would consider increasing my score.
3 comments
Thank you to the author for the response. After carefully reviewing the additional information and experiment results, most of my questions have been addressed, so I’ve decided to raised my score from 2 to 3..
We sincerely thank the reviewer for the positive score and encouraging assessment, as well as the valuable suggestions for improving our paper.
Response to Reviewer 6jjd
We thank the reviewer for the thoughtful comments and for recognizing the motivation and effectiveness of EAPM. We provide detailed responses to the concerns below.
Q1: Definition of action events and ambiguity in event boundaries.
A1: In our work, an action event refers to a semantically coherent action unit within a motion sequence, such as "walking forward", "picking something up", or "pouring something". We clarify that EAPM does not explicitly predict a discrete ground-truth event count or perform hard temporal segmentation. Instead, it learns K event prototypes through adaptive aggregation of local text and motion features.
Importantly, the learned prototypes are not required to have a strict one-to-one correspondence with manually defined action events. For ambiguous or continuous actions, their local features can contribute to multiple prototypes through the learned aggregation weights. Therefore, even when a motion is perceived as containing three events, EAPM may represent their semantics using two or more prototypes without requiring exact event-boundary agreement.
Q2: Rationale for using a fixed number of prototypes (K=3).
A2: We select K=3 based on the average action-event counts in HumanML3D (3.14) and KIT-ML (2.88). More importantly, our ablation study in Table 3 empirically demonstrates that K=3 achieves the best retrieval performance among the evaluated configurations.
The detailed results on HumanML3D are shown below.
We emphasize that K represents the number of latent prototype slots rather than the exact number of ground-truth action events in every motion. The adaptive aggregation mechanism and global representation jointly enable the model to capture motion sequences with varying action complexities.
Q3: Quantitative and qualitative evidence for the semantic meaning of learned prototypes.
A3: We provide evidence from both prototype visualization and retrieval evaluation.
First, Fig. 5 visualizes the temporal activation patterns of motion prototypes for a sequence containing "walking forward", "picking something up", and "pouring into a container". Different prototypes exhibit distinct activation patterns over the motion sequence.
Furthermore, Fig. 7 visualizes both textual and motion prototype activations, showing that different prototypes emphasize different action-related words and motion regions.
Second, as reported in Appendix Table 11, we construct a Diverse Set containing 10 motion sequences for each action-event count from 1 to 5. We compare its retrieval performance with that of a randomly sampled Origin Set:
The comparable performance on the two sets demonstrates that EAPM maintains effective retrieval performance across datasets with different distributions of action-event counts.
The visualizations provide qualitative evidence that different prototypes capture distinct action-related semantics, while the Diverse Set results demonstrate that EAPM maintains effective retrieval performance across varying event-count distributions.