acceptodds
Under review as a conference paper at ICLR 2027

Latent Manifold Fragmentation in Wearable sEMG: An Empirical Audit of Cross-Subject Generalization

Abstract

Surface electromyography (sEMG) gesture recognition literature frequently reports high offline accuracy (), yet these claims often mask methodological pitfalls such as temporal window leakage and rest-class metric inflation. In this paper, we introduce sEMG-Bench, a standardized, leakage-free benchmark framework, and conduct an empirical audit of four representative deep architectures alongside feature-engineered baselines on NinaPro DB5. Our empirical findings reveal three key insights: (1) Under strict Leave-One-Subject-Out cross-validation (LOSOCV), zero-shot active gesture recognition collapses across all deep backbones (20–25% Active F1), showing a pronounced disparity with reported offline claims. (2) Signal rectification () provides a vital physical energy prior, recovering to Active Macro F1 over raw signed inputs. (3) Paired t-tests () demonstrate statistical parity between lightweight 2D CNNs and complex Transformers, indicating that architectural over-engineering yields no significant generalization advantage under inter-subject distribution shifts. Linear probing reveals the underlying mechanism: frozen latent representations encode biometric subject identity ( accuracy) rather than gesture-invariant topologies, proving that cross-subject adaptation is geometrically essential for consumer-grade wearable interfaces.

Primary area:
datasets and benchmarks
Keywords:
Wearable surface electromyography, Cross-subject generalization, Representation probing, Empirical benchmark

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_DbnKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces sEMG-Bench, a standardized, leakage-free benchmark, and audits four representative deep architectures from the literature together with handcrafted feature baselines and a 1D CNN-GRU baseline under shared training and reporting. It states that high offline accuracy in prior work is often misleading because of temporal window leakage from random splits on overlapping frames and because the rest class dominates duration and inflates overall accuracy and macro F1.

Contributions are an open PyTorch pipeline with repetition-grouped two-fold cross-validation and rest-to-repetition indexing; a literature reimplementation study under pooled multi-participant intrapersonal evaluation, per-subject intrapersonal evaluation in the appendix, and interpersonal zero-shot leave-one-subject-out cross-validation; and mechanistic analysis using frozen penultimate linear probes for subject ID and t-SNE of embeddings to support the claim that representations encode biometric subject identity rather than gesture-invariant structure.

Experiments on NinaPro DB5 E1/E2 (10 subjects) use pooled repetition two-fold CV (three seeds) and zero-shot LOSOCV without target calibration, with active macro F1 as the primary metric. LOSOCV yields about 22–25% active macro F1 on E1 versus much higher pooled accuracy; rectification improves pooled end-to-end active macro F1 by roughly six to twenty points. Zero-shot comparisons show no significant difference for the 1D CNN-GRU baseline versus Chen et al. or Cao et al. (p = 0.594, 0.077); subject-ID probes reach about 86% (10% chance). Appendix results on EMG-EPN612 (306 subjects) follow the same evaluation constraints.

Critical strengths

The paper delivers a standardized, leakage-free evaluation framework (repetition-grouped splits, rest indexing, and explicit reporting of active macro F1) and applies it consistently across multiple reimplemented literature models and baselines. This directly addresses two widespread sources of inflated or incomparable sEMG results that the authors document in Table 1 and Section 2. For a benchmark-and-audit submission, this is the main reason the work is useful: it turns fragmented prior numbers into a single, reproducible protocol that future methods can be judged against.

Critical weaknesses

The scientific novelty relative to existing electromyography benchmarking work is limited. The paper itself cites EMGBench and related lines on out-of-distribution and leave-one-subject-out evaluation; the main additions are a NinaPro DB5–focused reimplementation audit, rectification ablations, and rest/leakage messaging that much of the community already treats as evaluation hygiene. The open pipeline is valuable, but the core empirical message ( zero-shot cross-subject active decoding is poor and heavy architectures do not clearly win) is largely confirmatory rather than a new principle or method. That makes the contribution read as an engineering report and literature correction.

The paper’s central claim about latent manifold fragmentation is weaker than the benchmark results. Subject-ID linear probing uses frozen features from gesture models trained on odd repetitions and tested on even repetitions within the same ten subjects, not under LOSOCV where active macro F1 collapses. t-SNE is qualitative and does not include quantitative alignment measures or adaptation experiments that would link the geometry story to zero-shot failure. Because this narrative appears in the title, abstract, and conclusion, it risks overselling what is otherwise a careful audit.

Questions for authors

For LOSOCV results, what input preprocessing was used for every model (signed vs rectified)? If all runs used rectification, please report the zero-shot active macro F1 gap with and without rectification. This affects how we read the architecture comparisons and the abstract’s rectification claim.

For subject-ID linear probing, please state the exact data split: which subjects and repetitions train the gesture model, which train the probe, and which evaluate the probe. In particular, is any held-out LOSO subject included in gesture-model training?

For reimplemented baselines without official code, what validation did you perform that the implementations match the originals?

Paired t-tests are reported across many model pairs and exercises. Did you apply any multiple-comparison correction, and which comparisons are pre-specified versus exploratory?

Reviewer_GYwqOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors provide sEMG-Bench, a benchmark for sEMG gesture recognition. The authors suggest a protocol to compare models under the same splits and evaluation settings. They show that performance drops sharply on unseen subjects, signal rectification improves performance, and complex models do not consistently outperform simpler models.

Critical strengths
  • The paper addresses a well-known but unresolved problem. I agree that cross-subject evaluation should be the default for sensor-based and health-related tasks.
Critical weaknesses
  • The main findings lacks new contribution. Poor subject generalization is already a well-known problem in sEMG. The paper provides useful measurements, but it is not clear what new understanding these results add.
  • Advocating proper data splits is useful, but LOSO splits are already established in the field. These recommendations could be presented as an evaluation protocol, but the contribution of building a new benchmark around them seems limited.
  • Reporting Active Macro F1 is useful, but handling class imbalance is also a standard part of evaluation. It is not clear how this adds a substantial new contribution to the benchmark. A confusion matrix would also help show whether the models mainly confuse active gestures with rest or with other active gestures.
Questions for authors

I included all the comments in the weaknesses section.

Reviewer_uns9OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper presents a benchmark to compare classification models for hand and finger gesture recognition from EMG signals. They build the work on the existing NinaPro DB5 dataset, adding an evaluation protocol and the related codebase as the main contribution. The authors state that such evaluation protocol is needed because i) there is not a unified protocol on whether to include rest periods in the evaluation samples or not, ii) many existing works suffer from data leakage between train and test and iii) "virtually all published SOTA" lacks cross-subject evaluation. Thus, the authors present an evaluation protocol that is said to tackle the three problems, they implement and evaluate multiple baselines, including published SOTA, and, finally, they provide a kind of mechanistic interpretation for explaining the performance of the models on the cross-subject evaluation. Precisely, they use two out of three sub-datasets of NinaPro DB5. They propose two evaluation protocols: a Pooled one for multi-participant intrapersonal validation (all subjects are aggregated together into a single dataset) and a Leave-One-Subject-Out (LOSO) for cross-subject validation. They evaluate four published methods against three baselines implemented by them. They report the performances and the results of paired t-tests to assess statistically meaningful relations. The main findings are (1) under LOSO, models performances collapse by ~20-25% (2) using the rectified |x(t)| instead of raw x(t) as input substantially improves performance (3) paired t-tests show no statistical advantage in favor of complex Transformers against lightweight 2D CNNs. Finally, they present a mechanistic analysis for explaining (1). They analyze the latent representations generated from the models, obtained by training them for gesture recognition and freezing the backbones up to the classifier head excluded. Then, they train a linear classifier to predict the subject ID, whose performance would be as the one of a random classifier for an ideal latent representation. Indeed, an ideal latent representation should be subject-invariant and only gesture dependent. Their conclusion is that the cross-subject performance collapse is due to the fact that the latents are easily separable by subjects.

Critical strengths
  • The paper identifies and addresses real gaps in the EMG gesture recognition domain: i) temporal leakage from overlapping windows ii) non-standardized rest-inclusion protocol and iii) lack of cross-subject evaluations. The paper presents the performance for both rest-included (F1-macro) and rest-excluded (F1-active) cases, quantifying the gap between the two. Additionally, it quantifies the performance collapse happening between intrapersonal and cross-subject evaluations.
  • Open-source code.
Critical weaknesses
  • Data Leakage. They use the test dataset for early stopping and learning rate scheduling of torch models. They do not mention which validation set is used for early stopping and lr scheduling. The released code uses the test_loader containing the test samples as monitored metric set for such purpose, which invalidates their claim for a leakage-free benchmark. In particular, they restore the checkpoint resulting in the highest test performance and use it for evaluation on the same test set.

  • Statistical tests poorly explained and results overclaimed. Missing explanation on how the paired t-tests are performed. For the interpersonal evaluation I assume they are paired on the 10 subject-held-out folds. For the intrapersonal one it is not clear if they paired on the three seeds averaging the subjects or on the subjects and seeds together. They report only selected tests but it is not clear how many tests were run, making it impossible to assess whether a statistical correction is required. They claim in the Abstract evidence for statistical parity between a 2D CNN (Chen et al., 2020) and a Transformer (Cao et al., 2026) but they never report this comparison: they only report tests between 1d CNN-GRU vs 2D CNN and 1d CNN-GRU vs Transformer.

Questions for authors
  1. Validation set. Can you rerun the experiments with a proper validation set?
  2. Statistical Tests. For each protocol, can you describe in detail how you performed the paired t-tests, which and how many and possibly add statistical corrections.
  3. E3 missing. Is there a reason why you did not include the third portion of the NinaPro DB5 dataset?
  4. Missing baselines. You state that "virtually all published SOTA baselines focus exclusively on Intrapersonal (Within-Subject) protocols". Cross-subject EMG gesture classification is an active field, why did you not include any model from this topic?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.