acceptodds
Under review as a conference paper at ICLR 2027

Same Gesture, Different Sensors: Cross-Dataset Representation Learning for Heterogeneous sEMG Gesture Recognition

Abstract

Surface electromyography (sEMG) provides a promising sensing modality for wearable hand gesture recognition, yet existing datasets are typically studied in isolation because they differ in electrode configurations, channel counts, subjects, and acquisition conditions. We propose handling heterogeneous sEMG datasets as complementary sensor observations of shared gesture semantics. Our cross-dataset representation learning framework supports variable-channel inputs through channel-token representations and dataset-specific adapters. During heterogeneous source pre-training, masked channel-representation learning reduces dependence on individual sensor observations, while supervised cross-dataset contrastive learning explicitly aligns same-gesture representations across datasets. We then transfer the learned representation to a previously unseen target dataset through masked adaptation and supervised fine-tuning using only the target training split. Across five independently collected sEMG datasets spanning 2-16 channels, our source-only transfer approach improves average balanced accuracy from 77.52% under single-dataset training to 80.47%, with improvements on all five targets. These results show that shared gesture semantics provide a practical bridge for reusing knowledge across heterogeneous sEMG sensor configurations.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
sEMG, representation learning, cross-dataset learning, heterogeneous sensor configurations, transfer learning, gesture recognition

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_wpQTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper is about representation learning in the cross-dataset scenario for sEMG based gesture recognition. There has been a lot of interest in sEMG gesture recognition over the past decade due to its applications in human computer interaction and for helping disabled people. However, while a lot of work has been done, a variety of sEMG sensors have been used. The diversity is across the number of channels, type and placement of electrodes as well as the sampling rate. Moreover, the gestures studied can also vary. Several such diverse datasets are publicly available. What this paper proposes is the utilize these public datasets by focusing on common hand gestures present in all of the datasets. Using these common gestures, a cross-dataset representation learning method has been developed. This involves pre-training using diverse datasets as indicated in Figure 1(a). This common representation then has been used to successfully recognize gestures in a target unseen dataset (after masked target adaptation and supervised fine-tuning as shown in Fig 1(b)). The experimental results show that representation transfer is indeed possible across heterogeneous sEMG datasets, using common hand gestures.

Critical strengths

(1) The idea of using the diverse existing datasets for more robust representation learning for sEMG based gesture recognition is novel.

(2) The extensive experimental results demonstrate the utility of this approach, as shown by the balanced accuracy results of Table 2. The generalization ability to unseen datasets as shown in the results is also promising.

Critical weaknesses

(1) While the idea of using diverse datasets for better representation learning for sEMG based gesture recognition is novel, the techniques employed to do so are not novel. In fact, as seen in Figure 1(a), the techniques of adapters and encoders along with losses described in equation (2) are fairly standard. Moreover, the target adaptation and supervised fine-tuning summarized in Fig 1(b) and described in Section 3.4 are also fairly standard. So there is no non-obvious methodological contribution. The application is good and its practical utility is real.

(2) It is not clear whether the proposed representation learning method helps improve the recognition rate of the non-common gestures. If this has not been stated, it must be stated. If it has been already stated, it should be made clearer.

(3) For a particular setting for a particular user, one uses a specific type of sEMG device. Just optimizing for that device type using only device specific data (with device specific data augmentation) might actually be better than improving the balanced accuracy using heterogeneous datasets. Please comment.

Questions for authors

(1) On line 51, other methods have been described that use diverse datasets. It will be useful to the reader to point out the substantive differences between those approaches and the proposed one.

(2) The reference for sEMGPoseMIM is missing on line 96.

(3) On line 107, the phrase "In contrast" has been used but it is not clear to the reader in contrast to what?

(4) Section 2.3 is not at all clear. While the second paragraph is supposed to indicate the differences, it seems very similar to the techniques described in the first paragraph. Please improve the exposition in this section.

(5) In Table 2, SAFT performs excellently for the GrabMyo 16ch dataset. It stands out oddly - it would be nice to have some explanation for it.

Reviewer_abBSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses cross-dataset sEMG gesture recognition under heterogeneous sensor configurations, including differences in electrode placement and channel count. It treats shared gesture labels as semantic correspondences across datasets.

The proposed framework uses a shared per-channel STFT encoder, dataset-specific adapters, and a Transformer to process variable-channel inputs. Source pre-training combines gesture classification, supervised contrastive learning, cross-dataset same-gesture alignment, and masked channel-representation reconstruction. The learned representation is transferred to an unseen target dataset through masked target adaptation and supervised fine-tuning using only the target training split.

Experiments cover five sEMG datasets with 2–16 channels and five shared gestures under subject-independent evaluation. The proposed method improves average balanced accuracy from 62.57% for single-dataset training to 74.15%, compared with 70.72% for the SAFT-style baseline. Ablations identify channel masking and target adaptation as the largest contributors, while additional experiments analyze source-dataset count, window stride, class-wise predictions, and representation quality.

Critical strengths

Practical formulation for heterogeneous sEMG data. The paper addresses the consequential problem of reusing datasets with different electrode placements and channel counts. Its shared per-channel encoder and variable-length Transformer preserve each dataset’s native sensor configuration without requiring artificial channel correspondence.

Critical weaknesses

Limited evidence for the claimed core contribution. The cross-dataset alignment objective improves average balanced accuracy by only 0.27 percentage points and slightly degrades performance on two datasets. Meanwhile, larger gains come from channel masking and target adaptation. With only one random seed and no significance analysis or direct alignment metrics, the paper does not convincingly establish that shared-gesture cross-dataset alignment is effective or novel beyond combining existing components.

Overly restricted evaluation setting. All non-shared gestures are removed, leaving only five predefined classes in both training and testing. The evaluation therefore excludes unknown gestures, transitions, and daily non-target hand activities, and cannot measure false activations or rejection ability in continuous use. Consequently, the reported closed-set accuracy provides limited evidence for the method’s practical applicability.

Questions for authors

Are the training, validation, and test sets restricted entirely to the five shared gestures? Were all other gestures, transition windows, and non-target daily activities excluded, and if so, how should the reported results be interpreted beyond closed-set classification?

Can the authors report multi-seed means, standard deviations, and statistical tests? In particular, is the 0.27-point average contribution of LCrossL_{\mathrm{Cross}} larger than normal training variability?

Does LCrossL_{\mathrm{Cross}} directly improve cross-dataset alignment, as measured by same-gesture cross-dataset distances or dataset-identity predictability? Why does removing it improve performance on NinaPro and GrabMyo?

Do SAFT-style Supervised and the proposed methods use matched backbones, parameter counts, optimization schedules, and training budgets? Which comparisons isolate the training objective, and which also involve architectural differences?

Reviewer_GNFdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a cross-dataset representation learning framework for sEMG gesture recognition across datasets with heterogeneous sensor configurations that leverages shared gesture labels as semantic correspondence between datasets. Each channel is encoded by a shared STFT-CNN encoder, followed by a dataset-specific residual adapter, channel-index embeddings and a shared Transformer. Source pre-training combines cross-entropy, supervised contrastive learning, a cross-dataset supervised contrastive term and masked channel-representation reconstruction. For an unseen target dataset, a new adapter is initialised, followed by masked target adaptation and supervised fine-tuning on the target training split. Ablations, source-count and stride sensitivity, and representation analyses (silhouette, cosine margin, UMAP) are provided.

Critical strengths
  • The evaluation protocol is careful about leakage. It is subject-independent, the target dataset is fully excluded from pre-training, and participants are removed from datasets which share subjects. The ablations and source-count analysis also show a consistent trend.
  • The paper tackles an under-explored problem: reusing heterogeneous public sEMG datasets with different channel counts and placements, without forcing a common channel layout.
Critical weaknesses
  • The central claim is not supported by the paper's own ablation. The paper's framing (title, abstract, conclusion) credits shared gesture identity, i.e. the cross-dataset contrastive alignment, for the transfer. Yet removing L_Cross reduces average balanced accuracy by only 0.27 pp, and removing it improves results on NinaPro-12ch and GrabMyo-16ch (Table 3).
  • The statistical evidence and baseline fairness are insufficient. All results use a single seed (42), and Tables 2–3 report no fold variance or significance tests, even though many differences are below 1 pp.
Questions for authors
  1. Can you report mean ± std over multiple seeds (and folds) for Tables 2 and 3, with a significance test? In particular, is the L_Cross contribution distinguishable from noise?
  2. Does the Single-Dataset baseline use the same augmentations, channel masking, SupCon and reconstruction objectives as the target-adaptation stage? Please add an "adaptation-only from scratch" control: the full target adaptation and fine-tuning pipeline without source pre-training. This would separate the effect of transfer from that of regularisation.
  3. Were the SAFT-style and Backbone+Supervised baselines given the same target training budget (including the masked adaptation stage)? Please also describe the SAFT-style re-implementation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.