- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper investigates how to preserve the temporal structure by propose SemTAC, which is built upon a Semantic Encoder and is trained using an InfoNCE objective. To validate, the authors use MDM to synthesize a human motion dataset and construct 1,420 sequences. For evaluation, the authors introduce metrics including smoothness, order recoverability, silhouette score, and kNN consistency, aiming to measure temporal organization and branching properties in the latent space. The experimental results show that contrastive learning based on local temporal adjacency can induce a latent geometry that reflects global temporal structure.
Critical strengths
- The idea of inducing global geometric structure from local temporal relationships is interesting and potentially insightful.
- The authors introduce metrics such as smoothness and order recoverability to evaluate the representation space. These analyses provide additional interpretability to the experiments.
Critical weaknesses
- The paper is not well written, and the overall presentation lacks clarity, particularly in the method section. The paper frequently uses terms such as latent geometry, global structure and trajectory structure, but these concepts are not clearly defined. More importantly, the paper lacks a complete method overview figure, making the proposed approach difficult to understand.
- The proposed method is relatively incremental, with limited technical novelty. The author acknowledges that the method is closely related to Temporal Neighborhood Coding.
- The final dataset contains only 1,420 sequences, which is limited in scale and does not provide sufficiently strong evidence.
- The experiments are conducted on a dataset synthesized by MDM, the generalization ability to real-world data has not been validated.
Questions for authors
please see weakness.
2 comments
Thank you to the author for the response. After carefully reviewing the additional information and experiment results, I find that my concerns have been partially addressed. However, I still have reservations about the paper’s novelty and writing quality. I have therefore decided to raise my score from 1 to 2.
Weakness 1: clarity and method overview
We thank Reviewer tySw for the review. We agree that the method section and the terminology need to be clearer. Figures 1 and 2 convey the concept and the data, but there is no pipeline figure. In the revision we will review and define those terms on first use, and add a method overview figure showing pair sampling, the encoder and InfoNCE objective, prototype construction, and DTW/Fréchet inference.
Weakness 2: novelty
We thank the reviewer for recognizing the central idea as interesting and potentially insightful, and we agree that its positioning can be made clearer in the introduction, which we will revise. As acknowledged in the paper, the closest prior work is TNC; the difference is that we define positive pairs across stages (different motions), and our contribution is the finding, the ablation analysis and the evaluation metrics rather than a new objective (see also our reply to Reviewer UXNo).
Weakness 3: dataset size
The dataset is designed as a controlled probe of whether temporal structure is recovered, not as a benchmark for absolute accuracy. For that purpose, we consider its scale adequate: the test split alone contains about 280 sequences (20% of 1,420), results are aggregated over multiple runs per model (10 seeds for the baselines, about 20 grid-search configurations for SemTAC), and the effects are large (rank-biserial correlations up to 1.0, with Holm-Bonferroni correction). The number of sequences is bounded by the quality filtering step. We agree that sensitivity to dataset size is a legitimate question and we would welcome the reviewer’s view on what scale would be convincing.
Weakness 4: real-world data
We agree that generalization to real-world data has not been validated; Sec. 5 identifies this as the next step. Evaluating on synthetic data was a deliberate design choice: measuring how much of a temporal structure is recovered requires ground truth about that structure, and we are not aware of public real-world datasets annotated in a way that allows this. Generating the motions with MDM, followed by quality filtering, let us control the structure (shared steps, gradual lateralization, label noise). We will make this scope explicit in the abstract and conclusion.