- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents COSMI, a pipeline for composing multi-object human interactions from single-object motion data, and a text-conditioned model for generating such interactions. The pipeline extracts and labels atomic motion clips from several existing datasets, then uses action and object annotations, composition constraints, and geometric filters to construct feasible combinations. The resulting dataset contains 221,677 sequences involving 36 objects and up to five objects per sequence. The proposed diffusion Transformer uses shared object slots, part assignment, object-relative motion, and contact-aware inference guidance. The authors evaluate the dataset and model using physical and semantic metrics, comparisons with adapted baselines, held-out object pairings, and ablations. The results indicate improvements on several reported measures, although the evidence for semantic plausibility and the interpretation of the contact-accuracy results would benefit from further validation.
Critical strengths
-
The paper offers a practical approach to scaling multi-object interaction data. Composing interactions from existing single-object motion data addresses a meaningful data bottleneck, and the reported dataset scale and range of object counts make the resource potentially useful for studying compositional generalization. This is a central reason I view the contribution positively.
-
The generation model is designed around the structure of the task. Shared object slots, part assignment, object-relative motion, and contact-aware guidance are appropriate choices for variable-object interactions. The comparisons and ablations provide useful evidence about how these components contribute, supporting the paper’s methodological value beyond the dataset alone.
Critical weaknesses
-
The semantic plausibility of the composed dataset is not independently validated. The pipeline uses corrected action labels, per-action object and head/gaze flags, composition rules, and geometric filters. These checks can establish compatibility under the chosen rules and help reject physically implausible combinations, but they do not show that people judge the accepted sequences to be semantically plausible interactions. This matters because the scale and usefulness of the composed dataset are central claims of the paper. A modest human audit of accepted sequences would materially strengthen that claim.
-
The reported per-marker “B-Acc” contact metric is difficult to interpret without class-sensitive statistics. B-Acc evaluates marker contact labels across 138 markers, while contact states may be sparse. Without reporting the positive-label prevalence or positive-class metrics, it is unclear how much of the score reflects correctly identifying actual contacts versus correctly predicting non-contact states. The paper also reports T-Acc, which is a useful complementary measure, but the missing statistics limit how strongly the B-Acc results support the claimed contact advantage.
Questions for authors
Q1: Could you report a human audit of a representative sample of sequences accepted by the current composition pipeline, including the fraction judged semantically plausible? This would help assess the semantic-quality claim for the existing dataset.
Q2: What fraction of marker–object labels in the test set are positive? Could you report a majority-negative baseline and positive-class precision/recall or balanced accuracy, and clarify whether COSMI’s advantage persists on positive contact labels?
3 comments
Thanks for the detailed response. I will keep my positive rating.
We thank the reviewer for the constructive review and the continued support.
R3 – W1/Q1 (user study of semantic plausibility)
R3 – W2/Q2 (class statistics of the contact metrics)
Revised manuscript
We hope that our reply addresses the reviewer's concerns on the semantic plausibility of the composed sequences and on the interpretation of the contact metrics. We invite the reviewer to check the following updates in the PDF:
L246–L247andp21–22(App. A.2):Table 7,Figure 7.L441andp23–24(App. A.6):Table 10.We thank the reviewer for their valuable remarks and await any further feedback.