- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents MorphCL, a new self-supervised pretraining method that first finds recurring motion primitives called motifs in the data and then clusters windows containing these motifs based on handcrafted features. Then group-based contrastive learning is done where windows that belong to the same cluster are positive pairs and all others are negative. The authors show that MorphCL improves downstream performance when used alongside existing SSL methods.
Critical strengths
When MorphCL is used as a plug-in to existing SSL training objectives, it shows substantial improvements. The improvements on SimCLR, however, are much smaller.
Critical weaknesses
- The main improvements come from adding MorphCL to existing SSL methods. However, when MorphCL is used alone, it often doesn’t show consistent improvements over baselines and frequently underperforms SimCLR. The reasoning for this is understudied. Whether it’s an effect of combining multiple SSL methods (e.g., how does it compare to MTL + SimCLR?) isn’t examined. Also, an explanation as to why MorphCL alone doesn’t work as well is missing. What about MorphCL makes it more suitable as an additional pretraining objective? While MorphCL uses a much smaller subset of data for its objective, I see its main use as an additional objective alongside a base SSL method that trains on the full dataset. So it is unclear whether the added complexity is justified, especially if combining simpler SSL objectives could achieve similar improvements.
- The authors claim that MorphCL can “inject explicit modeling of global structure” into existing SSL approaches. While MorphCL uses dataset-wide clustering, it is unclear how this leads to better modeling of global structure than existing approaches such as REBAR or RelCon, which also capture similarities between different motion patterns. It also isn’t clear what the authors expect the morphological clusters to represent. From Table 12, it seems like there were ultimately only 8 clusters for WISDM, with nearly 79% of motif-filtered windows rejected as noise. This seems like a rather coarse grouping of motion patterns. Also, since batches are sampled evenly across clusters, an item will be positively paired with roughly 12.5% of other items. It is unclear whether such broad grouping provides sufficiently meaningful supervision, especially when different activities may share the same morphological cluster.
Questions for authors
It’s not clear why the clustering/HDBSCAN was done on features extracted from the “context window” and not the “input window” or the “motif”. Considering that the length of a motif is significantly smaller than the “context window”, it seems like the motif may have very little effect on the feature values. In that case, why should the clustering be done only on windows that contain motifs, and not just all windows in general?
3 comments
Thank you for adding the MTL + SimCLR baseline and responding to my other concerns. Could the authors also report the fine-tuning results? Given the small difference in linear probing, it would be useful to see whether the improvement persists after fine-tuning.
I can see that adding MorphCL brings better performance than adding another SSL method. Adding SimCLR to MTL improves by 2.53%, which is 0.59% less than adding MorphCL to MTL. Also, MTL + SimCLR actually performs slightly better than MTL + MorphCL with direct clustering (49.94 vs. 49.61, from the response to BFTa). This makes me question how much of the improvement comes from the proposed pipeline rather than simply adding another SSL objective.
If we could see the full results for each downstream task, not just the average, on both WISDM and CAPTURE-24, along with a few more combinations of SSL methods, it would be much more convincing. Basically, like the results in the main paper, but rather than just showing that adding MorphCL improves existing SSL methods, that it improves them more than combining other SSL methods. Of course, I understand that running the full set of experiments within this short rebuttal window is unrealistic.
Another point I'm still not sure about is why MorphCL improves REBAR and RelCon so much, given that they already learn motion similarity between different windows, including across participants. I understand that MorphCL defines similarity differently, but I would imagine combining REBAR or RelCon with MTL or SimCLR could also bring similar improvements. However, I've worked with both methods before, and their reported performance can be quite difficult to reproduce, especially RelCon because of its two-stage training. Given how often they underperform simpler baselines here, I'm not sure whether they are performing at their best. This also makes me cautious about comparisons involving these methods, even if additional SSL combinations are tested, unless we can be confident that the baselines are reasonably well-tuned.
The technique proposed in this work seems valuable, but specifically as an add-on rather than a standalone technique. My main concern is that the current experiments and comparisons don't clearly show what makes MorphCL a better add-on than simply combining existing SSL methods, especially considering the additional preprocessing it requires. If this were shown more clearly, I could see this technique being widely used.
Thank you for the follow-up. We chose MTL and SimCLR because they are among our strongest standalone baselines and use different pretext tasks. Tables 4 and 5 report per-dataset results for a WISDM-pretrained ViT-S.
Adding SimCLR to MTL raises mean linear-probing macro-F1 from 47.41% to 49.94%. After fine-tuning, MTL + SimCLR scores below MTL alone on all five datasets, with mean macro-F1 decreasing from 57.51% to 55.71%. MTL + MorphCL reaches 58.23%, exceeding MTL alone by 0.72 points and MTL + SimCLR by 2.52 points on average, with gains over MTL + SimCLR on all five datasets. Adding MorphCL to MTL + SimCLR also improves macro-F1 in linear probing and fine-tuning (Tables 6 and 7). We compare an unweighted sum of the losses with a sum that gives equal weight to the instance- and corpus-level SSL objectives. We hope these results show that the gains stem from MorphCL’s novel approach to corpus-level supervision.
As you point out, MTL + SimCLR (49.94%) slightly outperforms the direct-clustering variant of MTL + MorphCL (49.61%). We included this variant to assess the contribution of motif discovery. The ablation removes the entire motif-based window selection stage while keeping MTL and the MorphCL loss fixed. The comparison shows that motif-based window selection can provide useful supervision for MorphCL.
For REBAR and RelCon, we used the authors’ released implementations and anchor–candidate batch structure. We trained the similarity metrics on the corresponding WISDM or CAPTURE-24 training splits and monitored both stages of training. Under our cross-dataset protocol, both methods score below MTL and SimCLR on average. The original REBAR study uses within-dataset splits, and the original RelCon model was pretrained on a substantially larger corpus. REBAR and RelCon use learned similarities to select or rank candidate windows. MorphCL adds supervision from fixed groups formed across recordings.
We would be happy to prioritize whichever additional SSL combination you find most informative. The CAPTURE-24 runs require more pretraining time than the rebuttal window allows. We will report per-task results for both protocols in the camera-ready version. With the rebuttal deadline approaching, we would appreciate your consideration of a higher overall score if the new fine-tuning results address your remaining concerns.
Table 4. Linear probing results on combining pretraining objectives
| Dataset | SimCLR | MTL | MTL + SimCLR | SimCLR + MorphCL | MTL + MorphCL |
|---|---|---|---|---|---|
| WEAR | 53.08±0.62 | 50.05±0.59 | 53.29±0.55 | 56.30±0.21 | 53.55±0.23 |
| Wetlab | 25.94±0.41 | 22.39±0.24 | 26.19±0.25 | 25.98±0.54 | 26.53±0.30 |
| PAMAP2 | 64.00±1.25 | 63.84±0.42 | 65.27±1.00 | 66.00±0.20 | 67.17±1.20 |
| Hang-Time | 35.73±0.44 | 33.71±0.98 | 37.49±0.46 | 37.36±1.08 | 35.82±0.51 |
| RWHAR | 65.14±1.44 | 67.03±0.80 | 67.47±0.90 | 68.46±2.54 | 69.60±1.20 |
| Avg. | 48.78±0.57 | 47.41±0.22 | 49.94±0.63 | 50.82±0.50 | 50.53±0.45 |
Table 5. Fine-tuning results on combining pretraining objectives
| Dataset | SimCLR | MTL | MTL + SimCLR | SimCLR + MorphCL | MTL + MorphCL |
|---|---|---|---|---|---|
| WEAR | 63.12±0.41 | 66.02±0.33 | 63.03±0.38 | 64.23±0.05 | 66.62±0.24 |
| Wetlab | 32.71±0.24 | 33.25±0.30 | 32.69±0.18 | 33.69±0.03 | 34.46±0.24 |
| PAMAP2 | 67.35±0.87 | 70.05±0.70 | 68.82±1.53 | 68.62±0.34 | 70.82±0.16 |
| Hang-Time | 47.96±0.30 | 49.67±0.15 | 48.19±0.11 | 48.78±0.47 | 49.30±0.09 |
| RWHAR | 65.86±0.43 | 68.58±0.45 | 65.83±0.43 | 65.93±1.15 | 69.93±0.22 |
| Avg. | 55.72±0.25 | 57.51±0.27 | 55.71±0.53 | 56.25±0.19 | 58.23±0.16 |
Table 6. Adding MorphCL to MTL + SimCLR (linear probing).
| Dataset | MTL + SimCLR | + MorphCL (weight 1) | + MorphCL (weight 2) |
|---|---|---|---|
| WEAR | 53.29±0.55 | 55.32±0.12 | 56.01±0.28 |
| Wetlab | 26.19±0.25 | 27.39±0.36 | 27.14±0.36 |
| PAMAP2 | 65.27±1.00 | 65.19±0.25 | 67.21±0.87 |
| Hang-Time | 37.49±0.46 | 37.69±1.07 | 37.41±0.36 |
| RWHAR | 67.47±0.90 | 68.78±0.77 | 69.63±0.58 |
| Avg. | 49.94±0.63 | 50.87±0.52 | 51.48±0.49 |
Table 7. Adding MorphCL to MTL + SimCLR (fine-tuning).
| Dataset | MTL + SimCLR | + MorphCL (weight 1) | + MorphCL (weight 2) |
|---|---|---|---|
| WEAR | 63.03±0.38 | 64.73±0.38 | 65.32±0.16 |
| Wetlab | 32.69±0.18 | 33.95±0.22 | 34.52±0.09 |
| PAMAP2 | 68.82±1.53 | 69.41±0.87 | 71.26±0.67 |
| Hang-Time | 48.19±0.11 | 48.31±0.08 | 49.13±0.29 |
| RWHAR | 65.86±0.43 | 66.20±0.90 | 66.78±0.82 |
| Avg. | 55.72±0.25 | 56.52±0.49 | 57.40±0.41 |
Rebuttal to Reviewer 3uos
We thank the reviewer for their thoughtful and constructive feedback. Below, we address each point in the order raised. We remain happy to provide any additional experiments the reviewer may request.
Analysis of MorphCL as individual loss
Based on the reviewers suggestion, we evaluated the suggested MTL + SimCLR baseline and compared it to SimCLR + MorphCL and MTL + MorphCL (see Table 1). Both baseline objectives score higher when combined with MorphCL than when combined with each other.
Table 1: Results on combining MTL and SimCLR losses individually as well as in combination with MorphCL
Comparison to REBAR and RelCon:
Compared to instance-based base objectives such as REBAR and RelCon, MorphCL adds corpus-level supervision by bringing together distinct windows that exhibit similar motion patterns across recordings and participants. Adding MorphCL to REBAR and RelCon improves WISDM ViT-S linear probing by 8.3 and 15.4 F1 points, respectively, showing that this supervision complements learned pairwise similarity. Standalone MorphCL reaches 49.56% here, and its reliance on the smaller, filtered subset may explain its less consistent performance across settings. Joint training lets the encoder learn these cross-window relationships alongside the broader range of motion available in the full corpus.
Rationale for Context Window Clustering
We chose to cluster context windows as we expect broader context as well as overlap among windows to better help cluster windows in a meaningful manner regarding both their local and global context. Following the reviewer’s suggestion, we compared input- and context-window clustering on WISDM for SimCLR and MTL separately (see Table 2). Experiments showed that performance is higher when using context windows for both SimCLR (49.92% vs. 50.82%) and MTL (48.94% vs. 50.53%). As motifs represent atomic sub-movements whose unique combination characterize repeated motion, we do not compute clustering features directly based on them.
Table 2: Downstream comparison of using Input Window vs. Context Window for morphological clustering
Analysis of Impact of Motif Discovery:
To accomodate character restrictions in our rebuttal, we refer to our initial response to Reviewer BFTa in which we provide an extended analysis of impact of the motif discovery.