acceptodds
Under review as a conference paper at ICLR 2027

MorphCL: Morphological Contrastive Learning for Inertial-based Human Activity Recognition

Abstract

Despite the ubiquity of sensors in wearable and mobile devices and the abundance of human movement data they generate, translating unlabeled recordings into foundational motion models remains an open challenge. Self-supervised learning (SSL) has alleviated the need for costly annotations, yet existing approaches leave the global structure of large-scale motion data largely untapped, relying on randomly sampled batches and local comparisons that become particularly problematic for in-the-wild inertial data dominated by stationary, low-variance behaviors. Here we introduce Morphological Contrastive Learning (MorphCL), a self-supervised pretraining framework that uses structure-aware grouping to inject explicit modeling of global structure into inertial-based SSL approaches. Building on two well-established pillars of motion analysis, the discovery of motion primitives, or motifs, and domain-specific feature descriptors, we show that MorphCL substantially improves linear probing and finetuning results of learned encoders by up to 15 percentage points in F1-score. In a comparison with existing foundation models, we demonstrate that MorphCL-pretrained encoders match or surpass them models in linear probing performance while trained on less data. Qualitative analysis of the resulting embedding spaces further reveals morphologically meaningful cluster structure, with improved separation of kinematically similar activity classes. Code and model weights are publicly available via: https://anonymous.4open.science/r/mcl-AC4B.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
inertial sensors, wearable activity recognition, self-supervised learning

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_3uosOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents MorphCL, a new self-supervised pretraining method that first finds recurring motion primitives called motifs in the data and then clusters windows containing these motifs based on handcrafted features. Then group-based contrastive learning is done where windows that belong to the same cluster are positive pairs and all others are negative. The authors show that MorphCL improves downstream performance when used alongside existing SSL methods.

Critical strengths

When MorphCL is used as a plug-in to existing SSL training objectives, it shows substantial improvements. The improvements on SimCLR, however, are much smaller.

Critical weaknesses
  • The main improvements come from adding MorphCL to existing SSL methods. However, when MorphCL is used alone, it often doesn’t show consistent improvements over baselines and frequently underperforms SimCLR. The reasoning for this is understudied. Whether it’s an effect of combining multiple SSL methods (e.g., how does it compare to MTL + SimCLR?) isn’t examined. Also, an explanation as to why MorphCL alone doesn’t work as well is missing. What about MorphCL makes it more suitable as an additional pretraining objective? While MorphCL uses a much smaller subset of data for its objective, I see its main use as an additional objective alongside a base SSL method that trains on the full dataset. So it is unclear whether the added complexity is justified, especially if combining simpler SSL objectives could achieve similar improvements.
  • The authors claim that MorphCL can “inject explicit modeling of global structure” into existing SSL approaches. While MorphCL uses dataset-wide clustering, it is unclear how this leads to better modeling of global structure than existing approaches such as REBAR or RelCon, which also capture similarities between different motion patterns. It also isn’t clear what the authors expect the morphological clusters to represent. From Table 12, it seems like there were ultimately only 8 clusters for WISDM, with nearly 79% of motif-filtered windows rejected as noise. This seems like a rather coarse grouping of motion patterns. Also, since batches are sampled evenly across clusters, an item will be positively paired with roughly 12.5% of other items. It is unclear whether such broad grouping provides sufficiently meaningful supervision, especially when different activities may share the same morphological cluster.
Questions for authors

It’s not clear why the clustering/HDBSCAN was done on features extracted from the “context window” and not the “input window” or the “motif”. Considering that the length of a motif is significantly smaller than the “context window”, it seems like the motif may have very little effect on the feature values. In that case, why should the clustering be done only on windows that contain motifs, and not just all windows in general?

3 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer 3uos

We thank the reviewer for their thoughtful and constructive feedback. Below, we address each point in the order raised. We remain happy to provide any additional experiments the reviewer may request.

Analysis of MorphCL as individual loss

Based on the reviewers suggestion, we evaluated the suggested MTL + SimCLR baseline and compared it to SimCLR + MorphCL and MTL + MorphCL (see Table 1). Both baseline objectives score higher when combined with MorphCL than when combined with each other.

Table 1: Results on combining MTL and SimCLR losses individually as well as in combination with MorphCL

SimCLRMTLSimCLR + MTLSimCLR + MorphCLMTL + MorphCL
Performance48.78% (±\pm 0.57%)47.41% (±\pm 0.22%)49.94% (±\pm 0.63%)50.82% (±\pm 0.50%)50.53% (±\pm 0.45%)

Comparison to REBAR and RelCon:

Compared to instance-based base objectives such as REBAR and RelCon, MorphCL adds corpus-level supervision by bringing together distinct windows that exhibit similar motion patterns across recordings and participants. Adding MorphCL to REBAR and RelCon improves WISDM ViT-S linear probing by 8.3 and 15.4 F1 points, respectively, showing that this supervision complements learned pairwise similarity. Standalone MorphCL reaches 49.56% here, and its reliance on the smaller, filtered subset may explain its less consistent performance across settings. Joint training lets the encoder learn these cross-window relationships alongside the broader range of motion available in the full corpus.

Rationale for Context Window Clustering

We chose to cluster context windows as we expect broader context as well as overlap among windows to better help cluster windows in a meaningful manner regarding both their local and global context. Following the reviewer’s suggestion, we compared input- and context-window clustering on WISDM for SimCLR and MTL separately (see Table 2). Experiments showed that performance is higher when using context windows for both SimCLR (49.92% vs. 50.82%) and MTL (48.94% vs. 50.53%). As motifs represent atomic sub-movements whose unique combination characterize repeated motion, we do not compute clustering features directly based on them.

Table 2: Downstream comparison of using Input Window vs. Context Window for morphological clustering

Baseline+ MorphCL (Input Window)+ MorphCL (Context Window)
SimCLR48.78% (±\pm 0.57%)49.92% (±\pm 0.37)50.82% (±\pm 0.50%)
MTL47.41% (±\pm 0.22%)48.94% (±\pm 0.33)50.53% (±\pm 0.45%)

Analysis of Impact of Motif Discovery:

To accomodate character restrictions in our rebuttal, we refer to our initial response to Reviewer BFTa in which we provide an extended analysis of impact of the motif discovery.

Thank you for adding the MTL + SimCLR baseline and responding to my other concerns. Could the authors also report the fine-tuning results? Given the small difference in linear probing, it would be useful to see whether the improvement persists after fine-tuning.

I can see that adding MorphCL brings better performance than adding another SSL method. Adding SimCLR to MTL improves by 2.53%, which is 0.59% less than adding MorphCL to MTL. Also, MTL + SimCLR actually performs slightly better than MTL + MorphCL with direct clustering (49.94 vs. 49.61, from the response to BFTa). This makes me question how much of the improvement comes from the proposed pipeline rather than simply adding another SSL objective.

If we could see the full results for each downstream task, not just the average, on both WISDM and CAPTURE-24, along with a few more combinations of SSL methods, it would be much more convincing. Basically, like the results in the main paper, but rather than just showing that adding MorphCL improves existing SSL methods, that it improves them more than combining other SSL methods. Of course, I understand that running the full set of experiments within this short rebuttal window is unrealistic.

Another point I'm still not sure about is why MorphCL improves REBAR and RelCon so much, given that they already learn motion similarity between different windows, including across participants. I understand that MorphCL defines similarity differently, but I would imagine combining REBAR or RelCon with MTL or SimCLR could also bring similar improvements. However, I've worked with both methods before, and their reported performance can be quite difficult to reproduce, especially RelCon because of its two-stage training. Given how often they underperform simpler baselines here, I'm not sure whether they are performing at their best. This also makes me cautious about comparisons involving these methods, even if additional SSL combinations are tested, unless we can be confident that the baselines are reasonably well-tuned.

The technique proposed in this work seems valuable, but specifically as an add-on rather than a standalone technique. My main concern is that the current experiments and comparisons don't clearly show what makes MorphCL a better add-on than simply combining existing SSL methods, especially considering the additional preprocessing it requires. If this were shown more clearly, I could see this technique being widely used.

Thank you for the follow-up. We chose MTL and SimCLR because they are among our strongest standalone baselines and use different pretext tasks. Tables 4 and 5 report per-dataset results for a WISDM-pretrained ViT-S.

Adding SimCLR to MTL raises mean linear-probing macro-F1 from 47.41% to 49.94%. After fine-tuning, MTL + SimCLR scores below MTL alone on all five datasets, with mean macro-F1 decreasing from 57.51% to 55.71%. MTL + MorphCL reaches 58.23%, exceeding MTL alone by 0.72 points and MTL + SimCLR by 2.52 points on average, with gains over MTL + SimCLR on all five datasets. Adding MorphCL to MTL + SimCLR also improves macro-F1 in linear probing and fine-tuning (Tables 6 and 7). We compare an unweighted sum of the losses with a sum that gives equal weight to the instance- and corpus-level SSL objectives. We hope these results show that the gains stem from MorphCL’s novel approach to corpus-level supervision.

As you point out, MTL + SimCLR (49.94%) slightly outperforms the direct-clustering variant of MTL + MorphCL (49.61%). We included this variant to assess the contribution of motif discovery. The ablation removes the entire motif-based window selection stage while keeping MTL and the MorphCL loss fixed. The comparison shows that motif-based window selection can provide useful supervision for MorphCL.

For REBAR and RelCon, we used the authors’ released implementations and anchor–candidate batch structure. We trained the similarity metrics on the corresponding WISDM or CAPTURE-24 training splits and monitored both stages of training. Under our cross-dataset protocol, both methods score below MTL and SimCLR on average. The original REBAR study uses within-dataset splits, and the original RelCon model was pretrained on a substantially larger corpus. REBAR and RelCon use learned similarities to select or rank candidate windows. MorphCL adds supervision from fixed groups formed across recordings.

We would be happy to prioritize whichever additional SSL combination you find most informative. The CAPTURE-24 runs require more pretraining time than the rebuttal window allows. We will report per-task results for both protocols in the camera-ready version. With the rebuttal deadline approaching, we would appreciate your consideration of a higher overall score if the new fine-tuning results address your remaining concerns.

Table 4. Linear probing results on combining pretraining objectives

DatasetSimCLRMTLMTL + SimCLRSimCLR + MorphCLMTL + MorphCL
WEAR53.08±0.6250.05±0.5953.29±0.5556.30±0.2153.55±0.23
Wetlab25.94±0.4122.39±0.2426.19±0.2525.98±0.5426.53±0.30
PAMAP264.00±1.2563.84±0.4265.27±1.0066.00±0.2067.17±1.20
Hang-Time35.73±0.4433.71±0.9837.49±0.4637.36±1.0835.82±0.51
RWHAR65.14±1.4467.03±0.8067.47±0.9068.46±2.5469.60±1.20
Avg.48.78±0.5747.41±0.2249.94±0.6350.82±0.5050.53±0.45

Table 5. Fine-tuning results on combining pretraining objectives

DatasetSimCLRMTLMTL + SimCLRSimCLR + MorphCLMTL + MorphCL
WEAR63.12±0.4166.02±0.3363.03±0.3864.23±0.0566.62±0.24
Wetlab32.71±0.2433.25±0.3032.69±0.1833.69±0.0334.46±0.24
PAMAP267.35±0.8770.05±0.7068.82±1.5368.62±0.3470.82±0.16
Hang-Time47.96±0.3049.67±0.1548.19±0.1148.78±0.4749.30±0.09
RWHAR65.86±0.4368.58±0.4565.83±0.4365.93±1.1569.93±0.22
Avg.55.72±0.2557.51±0.2755.71±0.5356.25±0.1958.23±0.16

Table 6. Adding MorphCL to MTL + SimCLR (linear probing).

DatasetMTL + SimCLR+ MorphCL (weight 1)+ MorphCL (weight 2)
WEAR53.29±0.5555.32±0.1256.01±0.28
Wetlab26.19±0.2527.39±0.3627.14±0.36
PAMAP265.27±1.0065.19±0.2567.21±0.87
Hang-Time37.49±0.4637.69±1.0737.41±0.36
RWHAR67.47±0.9068.78±0.7769.63±0.58
Avg.49.94±0.6350.87±0.5251.48±0.49

Table 7. Adding MorphCL to MTL + SimCLR (fine-tuning).

DatasetMTL + SimCLR+ MorphCL (weight 1)+ MorphCL (weight 2)
WEAR63.03±0.3864.73±0.3865.32±0.16
Wetlab32.69±0.1833.95±0.2234.52±0.09
PAMAP268.82±1.5369.41±0.8771.26±0.67
Hang-Time48.19±0.1148.31±0.0849.13±0.29
RWHAR65.86±0.4366.20±0.9066.78±0.82
Avg.55.72±0.2556.52±0.4957.40±0.41
Reviewer_BFTaOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces MorphCL, a self-supervised learning framework for inertial-based human activity recognition. It combines motif discovery and clustering of domain-specific features to construct morphological groups, which provide positive pairs for an auxiliary contrastive objective. The framework complements existing SSL methods and is evaluated using two pretraining datasets and five downstream benchmarks. The results show improvements across most evaluated settings, particularly in linear probing. Overall, I find the approach well motivated and the empirical evaluation substantial.

Critical strengths
  • The paper addresses a meaningful issue in wearable data.
  • MorphCL can be combined with several existing SSL objectives without changing the downstream architecture. The improvements across different objectives and backbones support its practical usefulness.
  • The evaluation is thorough. It includes both linear probing and finetuning, leave-one-subject-out evaluation, and three random seeds.
  • The appendix provides useful ablations on data filtering, sampling strategies, loss formulations, and hyperparameters.
Critical weaknesses
  • The subset experiments help distinguish the effects of filtering and the auxiliary loss. However, the specific necessity of motif discovery remains somewhat less clear. A comparison with morphological groups obtained by directly clustering the same handcrafted features, without motif discovery, would strengthen the justification for this stage.
  • The MorphCL ResNet-18 slightly exceeds the large-scale pretrained model in average probing performance, but remains behind after finetuning. Discussing this difference would help clarify the scope of the data-efficiency result.
Questions for authors
  • Have the authors tried constructing morphological groups without the motif-discovery stage?
  • How much does motif discovery contribute beyond feature-based clustering?
3 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer BFTa

We thank the reviewer for their thoughtful and constructive feedback. Below, we address each point in the order raised. We remain happy to provide any additional experiments the reviewer may request.

Analysis of Impact of Motif Discovery:

While lab-recoded datasets such as WISDM are inherently balanced through their data collection protocol, motif discovery and masking of similar motif candidates helped us find repeated patterns in the CAPTURE-24 dataset which is dominated by low-motion activities [1]. This is especially seen in the median acceleration-magnitude amplitude (P95–P5) rising from 0.030 to 1.139, AMI increasing to 0.4033 from 0.2704 and cluster membership accounting for 51.9% of dominant-frequency variation instead of 26.6% when motif discovery is applied.

Following the reviewers suggestion, we report results of MTL combined with an MorphCL objective that uses cluster labels obtained from directly applying HDBSCAN on the unfiltered WISDM dataset. MTL alone achieves 47.41 ±\pm 0.22 macro-F1 (see Table 4 in the main paper). Direct clustering improves this to 49.61 ±\pm 0.85, and adding motif discovery raises it to 50.53 ±\pm 0.45, a further 0.92 points. We hypothesize that this is due to cluster groups being able to distinguish motions of similar intensity by rhythm when defining contrastive pairs, as windows in clusters within the motif filtered variant explain more variation in dominant frequency than direct clustering (46.2% vs. 36.7%).

Obtained morphological clusters in CAPTURE-24 (see Table 2) ranged from low-amplitude windows (e.g. shared across sitting, standing, and walking) to specialized clusters such as ‘sports’ and ‘manual work’. Though only a fraction of the original CAPTURE-24 dataset they still retain windows of all ten named activity categories (as defined by Willetts et al., 2018 [1]) with distribution among these being more balanced. Similarly, for WISDM (see Table 3) the eight morphological groups account for 75.2% of the variation in acceleration-magnitude amplitude, 70.8% in within-window magnitude variance, and 46.2% in dominant frequency. Morphological clusters that span activity labels have interpretable motion characteristics consisting with for example one cluster consisting of typing and writing (low median amplitude), and another cluster consisting of walking, stair use, and kicking (dominant-frequency interquartile range of 1.56–1.95 Hz).

Table 2: Clustering results with and without motif discovery on CAPTURE-24

n_clusters% noiseDBCVAMI (vs. activity)
Without motif discovery44696.620.35980.2704
With motif discovery6797.180.38150.4033

Table 3: Clustering results with and without motif discovery on WISDM

n_clusters% noiseDBCVAMI (vs. activity)
Without motif discovery1079.97%0.21880.7408
With motif discovery878.83%0.24340.7122

Finetuning Gap Biobank Model

Compared to the CAPTURE-24 dataset the much larger UK Biobank cohort provides a wider range of participants and person days. We thus hypothesize that a trained backbone on the UK BioBank dataset can thus learn more useful motion features that become useful when the encoder is adapted to each target dataset. This is also in line with Narayanswamy et al. (Section C.2. in their supplementary material) [2] which also observed that increasing pretraining data can improve finetuning performance even when linear probing performance is slightly lower (see Table 10 in their supplementary material). We will discuss pretraining scale as a plausible contributor to the finetuning gap, alongside differences in the pretraining corpora and recipes in a camera-ready version of our manuscript.

[1] Haresamudram et al. Assessing the State of Self-Supervised Human Activity Recognition Using Wearables. IMWUT. 2022
[2] Narayanswamy et al. Scaling Wearable Foundation Models. ICLR. 2025

Dear Reviewer BFTa,

With the rebuttal deadline approaching, we wanted to follow up on our initial rebuttal and the additional experiments addressing your comments. We would be happy to clarify any remaining points. If the new evidence addresses your main concerns, we would be grateful if you considered raising your overall score.

Thank you for your response. As I am not familiar with this area, I will let the AC and other reviewers decide and will lower the weight of my review accordingly.

Reviewer_XtsHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MorphCL, a self-supervised learning framework for inertial-based human activity recognition (HAR) that aims to capture global motion structure beyond instance-level comparisons. The method combines motif discovery, clustering of handcrafted morphological features, and a group-based contrastive objective to construct morphology-aware pseudo-labels for pretraining. Experiments on two pretraining datasets and five downstream HAR benchmarks show improvements across several SSL methods and encoder architectures. The authors also demonstrate competitive linear probing performance against a foundation model trained on substantially more data.

Critical strengths
  1. The authors identify a reasonable limitation of instance-level contrastive learning for wearable data, where recurring motion patterns may be incorrectly treated as negatives. MorphCL incorporates morphological grouping into an auxiliary contrastive objective that can be integrated with existing SSL frameworks without modifying their backbone architectures.

  2. The experiments cover six SSL methods, two encoder architectures, and five downstream HAR benchmarks, using leave-one-subject-out evaluation. Improvements are observed across multiple configurations, and the authors provide several ablations examining data filtering, sampling strategies, and loss formulations. The competitive linear probing results against a model pretrained on substantially more data are also encouraging.

Critical weaknesses
  1. The proposed method combines several established techniques, including motif discovery, handcrafted feature clustering, and group-based contrastive learning. While PaPaGei (ICLR 2025) is discussed, other relevant recent works such as MoPFormer (NeurIPS 2025) and Bio-PM (ICML 2026), which also exploit motion primitives and domain-informed movement structure for wearable representation learning, are not considered. Although their learning objectives differ, the underlying motivations overlap considerably. In particular, Bio-PM provides publicly available code and pretrained models. This makes the novelty and advantages of MorphCL over existing structure-aware wearable SSL methods difficult to assess.

  2. Although the subset ablations demonstrate improvements beyond data filtering alone, it remains unclear whether the proposed motif discovery and morphological clustering pipeline is necessary. Comparisons against simpler alternatives, such as clustering handcrafted features directly without motif discovery or grouping windows using basic motion statistics, are missing. Such comparisons would help establish whether the improvements arise from the proposed morphological structure or from introducing additional pseudo-label-based contrastive supervision more generally.

  3. Only 47.3 out of 3004.6 hours (~1.6%) of CAPTURE-24 training data are retained for morphology-based supervision. Although the original SSL objectives still use the full dataset, it is unclear whether such a small subset adequately represents the global structure of human motion, particularly rare, non-periodic, or transitional activities. The paper does not sufficiently examine the coverage or representativeness of the discovered clusters.

  4. A central motivation is that randomly sampled batches in highly imbalanced wearable datasets can lead to problematic contrastive comparisons. However, removing balanced sampling produces comparable or better results on CAPTURE-24, with SimCLR + MorphCL improving from 50.97 to 52.08 (Tables 16-17). The authors suggest that this may result from reduced gradient conflicts between MorphCL and SimCLR, but this explanation is not directly validated. These results also raise questions about the necessity of balanced sampling, which is presented as an important component of the proposed framework.

  5. In the main full-corpus pretraining comparison, the improvements over the strongest SSL baselines are relatively small. For CAPTURE-24 with ViT-S, the best average linear-probing F1 increases from 51.36 (SimCLR) to 52.12 (MTL + MorphCL), while adding MorphCL to SimCLR actually reduces its performance to 50.97 (Table 2). For fine-tuning, the best average F1 increases from 60.26 to 60.68 (Table 3). Although MorphCL provides substantial improvements for some SSL objectives, its advantage over already competitive methods is less convincing than the maximum reported gains suggest.

Questions for authors
  1. How does MorphCL differ from recent motion-structure-aware SSL methods, particularly MoPFormer (NeurIPS 2025) [1] and Bio-PM (ICML 2026) [2]? Given that Bio-PM has released its code and pretrained models, could the authors provide a comparison under a consistent evaluation protocol?

  2. How much of the improvement can be attributed specifically to motif discovery? Would directly clustering handcrafted features from uniformly sampled windows without motif discovery achieve comparable results when using the same contrastive objective and training setup?

  3. Given that only ~1.6% of CAPTURE-24 is retained for morphology-based supervision, what kinds of motion patterns are represented in the discovered clusters? How well do these clusters cover rare, non-periodic, or transitional activities, and how does this affect downstream performance?

  4. Can the authors provide further evidence for the explanation that unbalanced sampling reduces conflicting gradients? More generally, what evidence supports the need for balanced sampling in addressing the proposed problem? Can the authors quantify whether morphological grouping reduces semantically incorrect negative pairs compared with standard instance-level contrastive learning?

[1] MoPFormer: Motion-Primitive Transformer for Wearable-Sensor Activity Recognition [2] Bio-Inspired Self-Supervised Learning for Wrist-worn Accelerometer Data

4 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer XtsH

We thank the reviewer for their thoughtful and constructive feedback. Below, we address each point in the order raised. We remain happy to provide any additional experiments the reviewer may request.

Comparison to existing motion structure approaches:

MoPFormer and Bio-PM share our motivation of exploiting motion structure for wearable self-supervised learning. MoPFormer quantizes short IMU segments into discrete codes and predicts masked codes from their temporal context. Bio-PM segments wrist acceleration at zero crossings and pretrains a transformer to reconstruct masked movement segments. Unlike these two instance-based SSL frameworks, MorphCL derives contrastive supervision from relationships across the pretraining corpus and allows to add a corpus-level morphology supervision to existing SSL objectives. We will clarify this distinction in our related work.

Following the reviewer’s suggestion, we evaluated the released Bio-PM checkpoint under the same downstream protocol as Table 5 of the main paper. We use learned acceleration features alone. Under this protocol, our pretrained checkpoints outperform Bio-PM in both downstream linear probing and in fine-tuning, which only reaches 40.41% and 53.50% macro-F1 score respectively.

Rationale for Motif Discovery & Cluster Analysis CAPTURE-24:

To accomodate character restrictions in our rebuttal, we refer to our initial response to Reviewer BFTa in which we provide an extended analysis of impact of the motif discovery.

Negative Pair Conflict between SimCLR and MorphCL

We would first like to clarify that sampling ablation (as seen in Table 16 and 17 of the main paper) varies only how the auxiliary MorphCL batches are drawn from the same morphology-labeled subset, while SimCLR batches do not get balanced. SimCLR is the only contrastive baseline in our study that restricts positives to augmented views of the same window. MorphCL contrarily defines its loss based on distinct window comparisons. Two distinct windows from the same morphology group can thus become negatives if they co-occur in a SimCLR batch and positives if they co-occur in a MorphCL batch. As the two losses use different training corpora (full dataset and morphological subset), this effect is difficult to measure within the full dataset training. Works such that of ClusterCLHAR [1] highlight SimCLR’s false-negative problem in wearable activity recognition and uses clustering to mask potential false negatives.

As balancing MorphCL batches gives smaller morphology groups more exposure, probability increases that MorphCL positive pairs may collide with SimCLR’s negative pairs. We nevertheless deem balancing a viable option as it encourages the shared encoder to bring windows together across a broader range of motion patterns. We furthermore expect this effect to be more present in CAPTURE-24 which, due to its skew towards stationary motion, will cause SimCLR batches to define negatives among these that collide with MorphCL’s positive pairs. Unbalanced sampling of MorphCL reduces this effect as it decreases the probability of MorphCL batches to contain positives that reflect this conflict. We will present this as a hypothesis for the CAPTURE-24 result in Appendix B.5 and clarify that balancing is an optional way to increase the representation of smaller groups, yet is to be seen as an optional hyperparameter rather than a design choice of our loss.

Overall relative improvements

While we agree with the reviewer, that on the unfiltered CAPTURE-24 MorphCL does not improve the base SimCLR objective during linear probing, it does so on four of five datasets after fine-tuning and across all tested combinations for the WISDM dataset for both backbones, where it is the strongest linear-probing baseline.

Specifically, we would like to highlight that we test MorphCL across a substantial empirical evaluation as also noted by other reviewers, raising mean macro-F1 in 15 of 16 linear-probing settings and all 16 fine-tuning settings, with scores averaged over five downstream datasets. Across individual downstream datasets, the mean improves in 144 of 160 comparisons (90%). The best average score also increases in every tested backbone and pretraining corpus combination under both evaluation protocols. Lastly, MorphCL improves a recent wearable SSL method RelCon (ICLR 2025). The gain holds across all five downstream datasets, showing that MorphCL complements RelCon’s learned similarity objective with a much smaller pretraining corpus.

[1] Wang et al. Negative Selection by Clustering for Contrastive Learning in Human Activity Recognition. IEEE IoT Journal. 2023

Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts, and the new results have addressed some of my concerns.

I think the experiments now provide the evidence that MorphCL is an effective and practical domain-informed auxiliary SSL objective. However, the new ablation also shows that direct feature clustering already achieves most of the improvement, while motif discovery provides a small additional gain. Since domain-informed grouping has also been explored in prior SSL work, I remain less convinced by the methodological novelty of the proposed pipeline.

I would suggest framing the contributions more carefully around what the experiments clearly demonstrate. The practical value of MorphCL is well supported, but the broader novelty claims should be more clearly distinguished from existing approaches.

Thank you for your reply. We agree that on WISDM, direct feature clustering recovers most of the improvement from MorphCL, with motif discovery providing a modest additional gain. As reported in our response to Reviewer jn3q (“Effectiveness of Morphological Groups”), randomly shuffling the group assignments removes the gains for both clustering pipelines. This suggests that the relationships captured by the groups contribute to MorphCL’s effectiveness. In a revised manuscript, we will emphasize the evidence for MorphCL as a practical auxiliary SSL objective and clarify that the benefit of motif discovery may depend on the distribution of the pretraining data.

MorphCL’s methodological contribution is to turn shared motion patterns across the pretraining corpus into an auxiliary SSL signal. This is different to prior work such as MoPFormer and Bio-PM which construct their learning signals at the instance level. We will explain this distinction more clearly in the revision. If you have additional methods in mind, particularly ones that use corpus-level grouping, we would be happy to benchmark MorphCL against them and include the results.

Dear Reviewer XtsH,

With the rebuttal deadline approaching, we wanted to follow up on our response addressing your comments. We would be happy to clarify any remaining points. If our rebuttal has addresses your main concerns, we would be grateful if you considered raising your overall score.

Reviewer_jn3qOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

MorphCL is a self-supervised learning framework for accelerometer-based human activity recognition. It discovers recurring motifs, clusters handcrafted features from their surrounding context, and uses the resulting groups to define an auxiliary contrastive objective. The framework complements existing SSL methods and is evaluated using two pretraining datasets and five downstream datasets, with both linear probing and finetuning.

Critical strengths
  1. The motivation is clear, and the method directly addresses the stated problem. Repetitive, imbalanced inertial recordings can produce inappropriate negatives under instance-level contrastive learning. MorphCL uses morphological groups to establish relationships beyond augmented views of individual instances.

  2. MorphCL offers practical utility as a plug-in objective. It complements existing SSL objectives without replacing them and improves most evaluated configurations across five downstream datasets. The subset, sampling, and loss ablations provide useful evidence about its behavior.

Critical weaknesses
  1. The novelty relative to existing clustering-based approaches requires clearer justification. ClusterCLHAR already uses clustering to address false negatives in sensor-based HAR, while PaPaGei uses domain-specific proxies to define contrastive groups. MorphCL’s distinction lies primarily in its fixed morphological grouping and motif discovery, but their additional value is not sufficiently isolated. A comparison using the same feature clustering and auxiliary objective without motif discovery would help establish the necessity of the proposed pipeline.

  2. The properties that make the fixed morphological groups effective remain unclear. Appendix B.3 provides useful clustering-quality and sensitivity analyses, but agreement with activity labels does not directly establish the suitability of the induced positive and negative relationships. In Table 13, AMI drops substantially while downstream performance decreases only modestly. This leaves unclear whether the gains depend on meaningful morphological relationships or can largely be obtained from coarse grouping. The reported noise rate measures excluded samples, rather than incorrect assignments among retained samples.

Reference

Wang et al. Negative Selection by Clustering for Contrastive Learning in Human Activity Recognition. IEEE Internet of Things Journal, 2023. https://doi.org/10.1109/JIOT.2023.3239945

Questions for authors
  1. Why was DeepConvLSTM evaluated only with WISDM pretraining, whereas ViT-S was evaluated with both WISDM and CAPTURE-24?

  2. Could you clarify the scope of “foundation model” in Section 5.4, distinguishing the terminology used by the cited work from the capabilities evaluated here?

4 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer jn3q

We thank the reviewer for their thoughtful and constructive feedback. Below, we address each point in the order raised. We remain happy to provide any additional experiments the reviewer may request.

Missing Comparison to Related Work

The PaPaGei framework is highly related to MorphCL and shares a similar motivation of injecting domain-informed structure during self-supervised pretraining. Nevertheless, PaPaGei was specifically designed around metrics exclusive for PPG data (sVRI, IPA and SQI) which makes its core framework inapplicable to inertial data. Our ablation though includes a comparison of using the loss formulation as used in the PaPaGei paper (see B.8 in the supplementary).

Based on the reviewer’s suggestion we compare MorphCL with a reimplementation of the ClusterCLHAR framework (see Table 1). ClusterCLHAR addresses limitations of SimCLR and proposes improvements using negative pair removal. Both of these frameworks are instance-based losses as opposed to MorphCL which proposes a novel way to compute morphology supervision based on the full pretraining corpora. Results show that, same as SimCLR, MorphCL is complementary to ClusterCLHAR.

Table 1: Average Linear Probing F1-Score of ClusterCLHAR and SimCLR with and without the proposed MorphCL objective

Baseline+ MorphCL
SimCLR48.78% (±\pm 0.57%$)50.82% (\\pm\ 0.50%)
ClusterCLHAR49.48% (±\pm 0.60%)50.66% (±\pm 0.49%)

Impact of Motif Discovery

Following the reviewers suggestion, we report results of MTL combined with an MorphCL objective that uses cluster labels obtained from directly applying HDBSCAN on the unfiltered WISDM dataset. MTL alone achieves 47.41 ±\pm 0.22 macro-F1 (see Table 4 in the main paper). Direct clustering improves this to 49.61 ±\pm 0.85, and adding motif discovery raises it to 50.53 ±\pm 0.45, a further 0.92 points. We hypothesize that this is due to cluster groups being able to distinguish motions of similar intensity by rhythm when defining contrastive pairs, as windows in clusters within the motif filtered variant explain more variation in dominant frequency than direct clustering (46.2% vs. 36.7%).

Effectiveness of Morphological Groups

To address the reviewers' concerns about how much morphological structure is actually present in coarse as opposed to fine-grained groupings, we examined motion intensity and rhythm in the retained windows of the 2 cluster setting when using an increased support of 400. Even though, within this setup more than 95% of retained windows are assigned to one broad group, the two-group configuration still captures a coarse motion distinction. The smaller group consists of 85% catching and dribbling windows and has about three times the median amplitude of the larger group. This offers one explanation for why it still improves over MTL alone (49.28 vs. 47.41). Opposed to this, our default setup (support=100, ρ\rho=100) resolves finer distinctions, including motions with similar amplitude and different dominant frequencies, and reaches 50.53 macro-F1.

To further show that reported gains of our approach can be attributed to meaningful relationships, we shuffled morphological group assignments in all clusters of our default experiment (support=100, ρ\rho=100) and within the smaller cluster of our increased support setup (support=400, ρ\rho=100), maintaining group sizes of all clusters. Results in Table 2 show that in both cases performance drops closer to the MTL baseline suggesting that discovered morphological groups in both clustering setups hold value.

Table 2: Average Linear Probing F1-Score of MTL + MorphCL with morphological group assignments and random group assignments for both fine-grained and coarse grained groups.

Morphological LabelsShuffled Labels
Support=100, ρ\rho=10050.53% (±\pm 0.45)47.62% (±\pm 0.61)
Support=400, ρ\rho=10049.28% (±\pm 0.47)48.69% (±\pm 0.56)

Missing Results DeepConvLSTM:

We initially excluded running experiments using a DeepConvLSTM backbone on the much larger CAPTURE-24 dataset as, due to its size, we did not expect the DeepConvLSTM to be suited to effectively capture “foundational knowledge”. We are currently running missing experiments and will report per-dataset results in the final manuscript.

Foundation Model Wording Clarification:

We used the term “foundation model” following Yuan et al. (2024), who describe their UK Biobank-pretrained encoder as a reusable starting point for accelerometer-based human activity recognition. We will revise Section 5.4 to specifically state that we examine the “foundation model” of Yuan et al. only with regards to cross-dataset transfer within accelerometer-based HAR.

Dear Reviewer jn3q,

With the rebuttal deadline approaching, we wanted to follow up on our initial rebuttal and the additional experiments addressing your comments. We would be happy to clarify any remaining points. If the new evidence addresses your main concerns, we would be grateful if you considered raising your overall score.

Reviewer_jn3qOpenReview ↗

Follow-up assessment

Thank you for the additional experiments. The direct-clustering and shuffled-assignment comparisons help clarify the contribution of motif discovery and the usefulness of the morphological groups. I also acknowledge the loss comparison with SupCon and PaPaGei in Appendix B.8.

My remaining concern is the rationale for the loss design. The formulation differs from prior approaches, but why aggregating positive probabilities before taking the logarithm is particularly appropriate for these morphological groups remains insufficiently explained. Table 20 supports its empirical utility, but the methodological significance of this change is less clear, especially given the modest additional benefit of motif discovery over direct clustering.

I maintain my rating of 2. I recognize the framework’s practical value, but remain unconvinced that its methodological contribution is sufficiently established.

Thank you for clarifying your remaining concern. Aggregating positives before the logarithm is also examined as by Khosla et al. (2020, Sec. 3.2.2, Eq. 3), who introduced SupCon. We select this formulation for morphological groups whose members share motion characteristics while varying in activity and execution. It maximizes the probability assigned to the anchor’s group, with stronger attraction to more similar positives and weaker attraction to less similar members. This accommodates variation within these groups. We will make this loss rationale and its precedent more explicit in the final manuscript.

MorphCL’s methodological contribution lies in how these groups are constructed from recurring motion patterns and domain-specific descriptors across the pretraining corpus, and how their relationships provide auxiliary SSL supervision. Table 20 shows limited sensitivity to the aggregation rule. Averaged over the five downstream tasks, the tested formulations differ by 0.21-0.62 F1 points across settings, with our selected formulation achieving the highest average in each. This supports the framework’s effectiveness across multiple loss formulations.

The direct-clustering ablation retains the central morphological grouping and auxiliary supervision while removing motif-based selection. Its improvement from 47.41% to 49.61% supports the value of that grouping, and motif discovery provides a further 0.92 points, reaching 50.53%. Both variants therefore demonstrate the value of MorphCL. On CAPTURE-24, motif discovery reduces the dominance of low-motion windows while retaining all ten activity categories. Median acceleration-magnitude amplitude rises from 0.030 to 1.139, and cluster membership explains 51.9% of dominant-frequency variation, compared with 26.6% for direct clustering. We will clarify that the additional benefit expected from motif discovery depends on the distribution of the pretraining corpus in the final manuscript.

If this clarification addresses your remaining concern about the loss rationale and methodological contribution, we would appreciate your reconsideration of your overall assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.