acceptodds
Under review as a conference paper at ICLR 2027

EMG-FoundBench: A Systematic Benchmark of Foundation Model Transfer and Adaptation for Electromyography

12 views

Abstract

Foundation models (FMs) are increasingly being developed for general time series and physiological signals, yet their transferability to downstream physiological tasks remains poorly understood. This question is particularly challenging for electromyography (EMG), where signal distributions vary substantially across users, sensing configurations, acquisition hardware, and downstream tasks. We introduce EMG-FoundBench, a systematic benchmark for studying foundation-model transfer and adaptation on EMG. EMG-FoundBench unifies 20 public datasets with over 1 million EMG segments and evaluates nine pretrained foundation models across four questions: how pretrained models perform when frozen or fully fine-tuned, how much pretraining helps compared with training the same model from scratch, how well models generalize to new users with limited labeled data, and how performance changes across different EMG tasks. Across the benchmark, linear probing provides useful information about pretrained representations, but full fine-tuning can substantially change downstream EMG performance. Comparing each pretrained model with the same model trained from scratch shows that the benefit of pretraining varies substantially across models and is not universal. Performance decreases when models are evaluated on new users, while five-shot adaptation improves macro-F1 in 70.2% of evaluated model-dataset combinations but recovers only part of the lost performance. Model performance is highly consistent between upper- and lower-limb classification and remains strongly correlated with continuous EMG-to-text decoding. Together, these results provide a systematic view of when pretrained time-series models transfer effectively to EMG and how their performance depends on fine-tuning, user variation, and downstream task.

Primary area:
datasets and benchmarks
Keywords:
Electromyography, Foundation Models, Time Series, Benchmark, Transfer Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_gpyEOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

EMG-FoundBench evaluates the transfer and adaptation of nine pretrained model configurations from seven time-series and electrophysiology model families on 20 public EMG datasets comprising approximately 1.03 million segments. It covers 19 classification datasets and one continuous typing dataset. The evaluation separates frozen linear probing, full fine-tuning, and training the same architecture from random initialization, and additionally examines held-out users, LOSO evaluation, and pooled target-cohort five-shot adaptation. Across 171 classification model-dataset pairs, full fine-tuning improves macro-F1 over linear probing by 16.4 percentage points on average, whereas pretrained initialization improves over scratch by only 3.0 points on average. Held-out-user macro-F1 decreases by 14.9 points, and five-shot adaptation recovers 6.3 points on average. Model rankings are correlated across task categories. The contribution is an evaluation resource and empirical characterization of transfer behavior, rather than a new learning algorithm.

Critical strengths

The paper presents controlled initialization comparisons. The architecture-matched pretrained-versus-scratch comparisons help distinguish the contribution of pretrained initialization from that of the model architecture under the specified training protocol. Also, data set coverage is another contribution of the paper. The benchmark consolidates heterogeneous EMG datasets and includes held-out-user and LOSO evaluations, providing a useful reference for comparing model performance across datasets and users.

Critical weaknesses

First, the present manuscript has limited scientific insight beyond an expanded empirical benchmark. The manuscript addresses a relevant evaluation gap, but the scientific insight remains limited relative to the scale of the experiments. EMGBench [6] already investigates EMG generalization and adaptation, while related EEG work [2] examines the practical value of foundation models and the distinction between frozen and fine-tuned performance.

Second, there is insufficient evidence of robustness to optimization choices. The conclusions about pretraining benefits require stronger evidence that they are robust to the downstream training procedure. Figure 2 shows substantial variation across models and datasets, including negative pretraining gains for some configurations. However, models use a shared learning rate and a fixed maximum training budget for pretrained fine-tuning and scratch training, with limited reported sensitivity analysis. Although an exhaustive hyperparameter sweep across all model-dataset combinations would be costly, targeted sensitivity analyses on representative configurations, together with repeated paired runs, would help distinguish robust initialization effects from dependence on a particular training set-up.

Third, I think that RQ4 findings lacks task diversity and interpretation. Including emg2qwerty is a reasonable way to extend evaluation beyond classification, but a single typing dataset provides limited evidence for consistency across different EMG task types.

Questions for authors

I have a few questions for the authors.

  1. Isn't the biggest advantage of pretraining its efficiency? There is limited discussion of this point.
  2. What kind of EMG information does the model use? The model can use frequency, channel, and temporal information. Masking some of this information or shuffling the temporal order of the data could help identify the main information used by the model.
  3. The 171 model-dataset comparisons are not independent. The NinaPro databases are related, and MOMENT and Chronos-2 each include two configurations from the same model family.
  4. Is pretraining useful, or are its benefits lost during fine-tuning? MOMENT-B performs worse after full fine-tuning than with LP and also performs worse than training from scratch. These results alone do not distinguish whether the pretrained knowledge is unsuitable for EMG or whether useful representations are lost during fine-tuning.
  5. How different is performing well on multiple datasets from reusing one representation across different settings? When models are trained separately on each dataset, evaluating them on 20 datasets does not clearly show whether a single representation can be reused across devices or tasks.
2 comments
AuthorsOpenReview ↗

Response to Reviewer gpyE

We thank the reviewer for the detailed comments. The review questioned the insight beyond prior benchmarks, the robustness to optimization choices, and the task diversity behind RQ4. We have revised the paper to address each of them: we added learning-rate, seed, and training-budget analyses (Appendix A.2.1), input-perturbation experiments (Appendix A.5), and a cluster bootstrap for dependent model–dataset comparisons (Appendix A.7), and we narrowed the RQ4 conclusions. All numbers below come from the revised results (General Response G1 and G2; 162 model–dataset pairs).

W1. Insight beyond EMGBench [6] and the EEG study [2]. EMGBench studies how EMG gesture classifiers with ImageNet-pretrained backbones adapt to new users; it does not evaluate frozen representations or matched random initialization. The EEG study [2], now cited in Section 2.2, compares foundation models with decoders of different architectures. We keep each architecture fixed and change only the initialization, which yields three findings: (i) The benefit of pretraining depends on the model: seven of nine configurations benefit on most datasets, whereas MOMENT-B and PatchTST perform better from scratch. The gain is moderately associated with fine-tuned performance (ρ = 0.65; MOMENT-S and TimesFM reach 62.1% and 62.9% with gains of 3.7 and 19.8 points), and in neither MOMENT nor Chronos-2 does the larger checkpoint gain more. (ii) The best linear-probing model is also the best fine-tuned model on 4 of 18 datasets. (iii) The aggregation should preserve which electrode a signal comes from: permuting the electrode order lowers the macro-F1 of fine-tuned MOMENT-S from 80.1 to 14.2 on GRABMyo (Appendix A.5).

W2. Robustness to optimization choices. Appendix A.2.1 analyzes three backbones whose gains differ in sign and size. Selecting the learning rate separately for fine-tuning and scratch training preserves the sign of the gain in 8 of 9 cases (Table 10). With three seeds, the gain of UniTS is positive in 11 of 11 runs and that of PatchTST is negative in 10 of 12 (Table 11). Within 50 epochs, the best checkpoint precedes the final epoch in 176 of 180 runs. These runs used an earlier assignment of subjects to Pool A and Pool B; within them, the sign of the mean gain is largely stable, and it agrees with the main results for 9 of 11 model–dataset pairs.

W3. RQ4. We agree and narrowed the conclusions to the evaluated tasks (Section 4.5): performance is highly consistent between upper- and lower-limb classification and moderately correlated with EMG-to-text decoding (ρ = 0.65 and 0.61 between the CER ranking and the macro-F1 rankings).

Q1. Efficiency. Pretrained fine-tuning reaches its best checkpoint earlier than scratch training in 62 of 90 model–dataset pairs (median epoch 15.5 vs. 23), most clearly for Chronos-2.

Q2. Information used. Appendix A.5 perturbs the test windows of fine-tuned MOMENT-S on six datasets (Table 17). Permuting electrodes reduces macro-F1 on all six. Shuffling 100-ms blocks, which keeps the signal within each block, reduces macro-F1 by 4 points or less on two datasets, by 8 points on MyPredict, and by 13 to 51 points on the other three, and time reversal reduces performance on all six. The model therefore relies on a fixed mapping between electrodes and features and, on several datasets, on temporal structure beyond 100 ms.

Q3. Dependence between pairs. A two-way cluster bootstrap (Appendix A.7) that resamples model families (MOMENT and Chronos-2 variants grouped) and dataset families (NinaPro DB1–DB7 grouped) gives 95% intervals of +11.9 [+6.7, +16.6] accuracy points for fine-tuning over linear probing, −20.1 [−32.6, −12.5] for new users, and +15.4 [+7.4, +24.3] macro-F1 points for five-shot adaptation. The interval for the pretraining gain is wide, +7.4 [+0.3, +15.0] macro-F1 and +6.4 [−0.04, +13.3] accuracy points, because the gain differs between families: it is positive for five of seven families and negative for MOMENT and PatchTST, consistent with our conclusion that the benefit is model-dependent.

Q4. MOMENT-B. Its frozen representation ranks fourth under linear probing and exceeds frozen MOMENT-S by 10.6 accuracy points, so its pretraining captures EMG-relevant information. Full fine-tuning then lowers its accuracy below linear probing on 13 of 18 datasets. This points to the adaptation step rather than to unsuitable pretrained knowledge, although our experiments do not isolate the specific cause (Section 4.3).

Q5. Representation reuse. Linear probing reuses one frozen representation across all datasets with a dataset-specific linear head, and its gap to full fine-tuning (11.9 accuracy points on average) shows how much dataset-specific adaptation adds. Transferring one fine-tuned EMG representation across devices or tasks is a separate question that our protocols do not cover.

I appreciate the authors’ substantial efforts to strengthen the paper through additional experiments and clarifications. However, my core concerns regarding the robustness of the findings under the final evaluation protocol and the depth of scientific insight remain insufficiently resolved. I therefore maintain my original rating.

Reviewer_8KfVOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

EMG-FoundBench examines how pretrained time-series models transfer to EMG. Its RQ1-RQ4 structure covers frozen and fine-tuned representations, comparisons with architecture-matched models trained from scratch, adaptation to new users with limited labels, and performance across EMG tasks. The evaluation spans multiple datasets and adaptation settings, with analyses of five-shot variability, LOSO statistics, and rank correlations. I find the benchmark useful, particularly the pretrained-versus-scratch comparisons. Uneven dataset coverage and unmeasured source-training variability make some of its broader conclusions harder to assess.

Critical strengths

Comparing each pretrained model with the same architecture trained from scratch helps separate the contribution of pretraining from that of the architecture. This is the part of the benchmark I find most useful: a high downstream score alone would not answer whether pretraining helped.

The RQ1-RQ4 structure connects representation quality and fine-tuning with adaptation to new users and variation across tasks. Evaluating these questions within one framework, across several datasets and adaptation settings, makes the results easier to compare. The existing statistical analyses give future EMG benchmarks a useful starting point.

Critical weaknesses
  1. Dataset coverage. The dataset coverage does not fully support the breadth of the cross-task conclusions. Typing uses one dataset, and lower-limb coverage is relatively limited. More participants within a dataset do not provide the variation in acquisition protocols that additional datasets would offer. I would narrow the conclusions about task generalization or explain more clearly why the selected datasets support them.

  2. Training robustness. I remain uncertain about the stability of the pretrained vs scratch findings. A shared 50-epoch limit does not show that the models have converged comparably. The adaptation and LOSO analyses are useful, but they do not measure variability from repeated source training.

  3. Preprocessing and evaluation details. The preprocessing of already rectified NinaPro DB1 signals, checkpoint and adapter choices, and construction of the typing splits need clarification, since these choices may affect rankings and negative-transfer conclusions.

  4. Channel aggregation. A controlled comparison of averaging and concatenation would help establish whether model rankings depend on this choice and how it interacts with pretraining.

  5. Introduction. The opening motivation makes the setup harder to follow. The RQ1-RQ4 structure is clearer; linking each question to its main finding could reduce repetition with Section 2.3.

  6. Figure 2. The figure would be easier to interpret if preprocessing, model adaptation, and evaluation protocols were visually separated.

Questions for authors
  1. Why were 50 epochs selected? What do existing learning curves and repeated source-training runs indicate about convergence, model rankings, and the stability of negative-transfer findings?

  2. Why apply a 20 Hz high-pass filter to the already rectified NinaPro DB1 signals?

  3. How are typing users, sessions, and CTC targets handled across training, validation, and testing? Please also explain the CapgMyo dimensional discrepancy and the reported lower-limb participant/window counts.

  4. Which checkpoints and adapters are used? Please specify which components are pretrained, frozen, or randomly initialized, and how different context lengths are handled.

  5. What dataset-selection criteria support the scope of RQ4? Was the Camargo locomotion dataset considered? I am asking for the selection rationale rather than an additional benchmark during rebuttal.

  6. How was channel aggregation chosen for each model? Do existing analyses indicate whether averaging versus concatenation changes model rankings or the measured benefit of pretraining?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 8KfV

We thank the reviewer for the detailed review and for finding the pretrained-versus-scratch comparison useful. The review raised concerns about training robustness, channel aggregation, NinaPro DB1 preprocessing, evaluation details, and the dataset coverage behind RQ4. We have revised the benchmark to address each of them: we added learning-rate, seed, and training-budget analyses, re-ran all main experiments with one channel aggregation and a rebuilt NinaPro DB1, documented the evaluation details, and narrowed the RQ4 conclusions. Details follow (revised text is marked in blue).

W2/Q1. Training budget and repeated runs. We use a shared 50-epoch budget to keep the benchmark affordable. For five configurations, the median selected epoch is 15.5 for fine-tuning and 23 for scratch training, and the best checkpoint precedes the final epoch in 176 of 180 runs. With three seeds, the gain of UniTS is positive in 11 of 11 runs and that of PatchTST is negative in 10 of 12. Selecting the learning rate separately for fine-tuning and scratch training preserves the sign of the gain in 8 of 9 cases. These runs used an earlier subject assignment; within them, the sign of the mean gain is largely stable across seeds and learning rates, and it agrees with the main results for 9 of 11 model–dataset pairs (Appendix A.2.1).

W4/Q6. Channel aggregation. In the submission, MOMENT concatenated and the other models averaged their channel representations. All main results now use concatenation (General Response G1). In a controlled comparison (Appendix A.5), concatenation gives higher linear-probing macro-F1 in all 54 comparisons (mean +23.8 points), the relative order of the models is largely preserved (median Spearman ρ = 0.94), and the sign of the MOMENT-S pretraining gain is the same under both aggregations on all six datasets.

W3/Q2. NinaPro DB1. We agree that a 20 Hz high-pass filter is not appropriate for DB1, which is a 100 Hz RMS envelope. The submitted experiments did not filter DB1; Appendix B.4 incorrectly suggested otherwise and is corrected. We rebuilt DB1 with all 52 gestures and a trial-level split. In a controlled comparison in which two versions of DB1 differ only in this filter, filtering lowers fine-tuned macro-F1 for MOMENT-S (48.4 to 39.7), PatchTST (19.6 to 10.7), and UniTS (34.7 to 14.5), with the same pattern for scratch training. This comparison used an earlier subject assignment and, for PatchTST and UniTS, averaging, so its absolute values are not comparable with the result tables.

Q3. Typing, CapgMyo, and lower-limb counts. Typing (Appendix B.3): each user's 3-second windows from all sessions are ordered by recording time and split 70/15/15 into training, validation, and test. The CTC target of a window is the sequence of characters whose key-press time falls within it; spaces are kept, other special keys are removed, and windows without characters are discarded. Each 1-second CapgMyo trial is repeated three times along the time axis to obtain 3000 samples; Appendix B.1.4 previously stated resampling. The lower-limb counts in Table 1 sum HD-EMG (10 participants / 1,608 windows), Mendeley Knee (61 / 4,480), SIAT-LLMD (40 / 17,038), and MyPredict (45 / 31,827), which includes the MyPredict1 and MyPredict2 subsets (Appendix B.2.4).

Q4. Checkpoints and adapters. Appendix C.9 lists the checkpoint and input-length handling of every model. No adapter modules are added: the only randomly initialized components are the linear head and, for UniTS, its task-specific prompt and classification tokens, which are random in both settings and frozen in linear probing. Linear probing freezes the backbone, full fine-tuning updates all parameters from the checkpoint, and scratch training starts from random weights.

W1/Q5. Dataset coverage and RQ4. Performance is highly consistent between upper- and lower-limb classification and moderately correlated with EMG-to-text decoding (ρ = 0.65 and 0.61 between the CER ranking and the macro-F1 rankings); Section 4.5 notes that more datasets are needed to extend this to other tasks. Removing any single lower-limb dataset keeps the upper- versus lower-limb ρ between 0.88 and 0.96. We selected public multichannel surface EMG datasets with labels suited to windowed classification or sequence decoding, covering different body regions, hardware, and tasks. We will consider the Camargo et al. (2021) dataset, which adds a different locomotion protocol, in future releases.

W5/W6. Introduction and figure. The contribution list pairs each research question with its main finding, and Section 2.3 positions the benchmark relative to prior work. The overview figure (Figure 1) follows the order of the paper: datasets (Section 3.1), model adaptation (Section 3.3), and evaluation protocols (Section 4.1). We thank the reviewer for both suggestions.

Reviewer_8KfVOpenReview ↗

Dataset coverage

Thank you for the explanations. However, the limited dataset diversity remains, particularly for typing. I therefore consider this concern partially addressed and maintain my rating.

Reviewer_2s87OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

the paper benchmarks nine pretrained model configurations across 20 EMG datasets covering gesture recognition, lower-limb activities, and typing. It compares linear probing, full fine-tuning, training from scratch, and adaptation to unseen users. Fine-tuning generally improves performance, while the benefit of pretraining varies substantially across architectures and datasets.

Critical strengths
  1. broad, useful evaluation: Twenty datasets cover diverse acquisition devices, user populations, and tasks. The unified data resource and evaluation protocols provide a useful foundation for subsequent EMG transfer research.

  2. architecture-matched pretraining comparisons: Comparing pretrained and randomly initialized versions of each architecture separates the contribution of initialization from architectural differences. Evaluating unseen users and limited-label adaptation also increases practical relevance.

Critical weaknesses
  1. channel handling confounds comparisons between models. Appendix C.9 concatenates channel representations for MOMENT but averages independently encoded channels for TimesFM, Lag-Llama, and PatchTST. The latter readouts discard electrode ordering, while MOMENT preserves it and uses a larger classification head. Consequently, performance differences cannot be attributed cleanly to backbone quality or transferability. This concern does not invalidate the matched pretrained-versus-scratch comparisons within each architecture.

  2. preprocessing needs validation for different signal types. The pipeline applies a common bandpass procedure, but NinaPro DB1 contains RMS-rectified signals recorded at 100 Hz. Applying the stated 20 Hz lower cutoff may suppress useful envelope information. Independently normalizing each channel within each window also removes relative amplitude information. Sensitivity checks are needed to establish how these choices affect the conclusions. The original dataset documentation confirms DB1’s acquisition characteristics.

Questions for authors
  1. do the model rankings persist when independently encoded channels use comparable readouts that preserve electrode identity, with classification-head capacity controlled?

  2. what exact filtering was applied to NinaPro DB1? How do its results change with envelope-appropriate preprocessing, and do the main findings persist under normalization that preserves relative channel amplitudes?

1 comment
AuthorsOpenReview ↗

Response to Reviewer 2s87

We thank the reviewer for the careful reading and for recognizing the architecture-matched comparisons and the unified data resource. The review raised two concerns: the inconsistent channel aggregation across models and the preprocessing of NinaPro DB1. We have revised the benchmark to address both. All models now use the same channel aggregation and all main results were re-run, and the relative order of the models is largely preserved. NinaPro DB1 was rebuilt without the 20 Hz filter, and a controlled comparison confirms that this filter degrades performance. Details follow (revised text is marked in blue; see also General Response G1 and G2).

W1 / Q1. Channel aggregation, head capacity, and rankings

  • Same aggregation: all nine configurations now concatenate their channel representations (Sections 3.3 and 4.1, Appendix C.9); LOSO and emg2pose keep the original aggregation, as noted in their captions.
  • Rankings: in a controlled comparison on six datasets (Appendix A.5, Table 16), concatenation improves linear-probing macro-F1 in all 54 cases (mean +23.8 points), while the relative order of the nine models is largely preserved (median Spearman ρ = 0.94). After the full re-run (Table 3), the Chronos-2 models remain the strongest after fine-tuning, and TimesFM is now third.
  • Head capacity: the classifier input is C × d (channels × representation width), so head capacity is identical within each pretrained–scratch pair and not matched across architectures (d from 128 to 2048). A wider head does not translate into higher accuracy: the 768-dimensional MOMENT and Chronos-2 checkpoints perform worse than or similarly to the 512-dimensional ones (48.8% vs. 62.1%; 71.1% vs. 71.0%). Projecting all representations to a common width would provide a stricter control.
  • Within-architecture comparisons: for MOMENT-S, the sign of the pretraining gain is the same under both aggregations on all six datasets.

W2 / Q2. NinaPro DB1 and normalization

  • Exact filtering: DB1 is the 100 Hz RMS-rectified output of Otto Bock electrodes. The submitted experiments applied no band-pass or notch filter to DB1; the signals were resampled to 1 kHz and z-scored per channel and window. Appendix B.4 incorrectly suggested that the 20 Hz filter was applied, and we corrected Appendices B.1.1 and B.4.
  • Envelope-appropriate preprocessing: we rebuilt DB1 with all 52 gestures, a trial-level split, and no additional filtering, and re-ran all models. To isolate the filter, we trained MOMENT-S, PatchTST, and UniTS on two DB1 versions that differ only in the band-pass filter of Appendix B.4 (20–45 Hz at 100 Hz). Filtering lowers fine-tuned macro-F1 from 48.4 to 39.7, 19.6 to 10.7, and 34.7 to 14.5, with the same pattern for scratch training. This comparison used an earlier subject assignment and, for PatchTST and UniTS, averaging, so its absolute values are not comparable with the result tables; the comparison between the two DB1 versions is unaffected.
  • Normalization: per-window, per-channel z-scoring removes relative amplitude across channels. All evaluated foundation models except BRANT also normalize each channel inside their input layer (for example, RevIN in MOMENT and TimesFM and instance normalization in Chronos-2 and UniTS), so relative channel amplitude is removed inside these models regardless of the input normalization. Preserving it would require changing these input layers, a promising direction for EMG-specific models.

We hope these revisions address the reviewer's concerns and would be happy to clarify any remaining questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for their constructive comments. The reviews raised four shared concerns: the inconsistent channel aggregation across models, the preprocessing of NinaPro DB1, the robustness of the pretraining gain to training choices, and the scope of the cross-task conclusions. We have revised the benchmark to address each of them. We re-ran all main experiments with the same channel aggregation for all models and a rebuilt NinaPro DB1, and we added experiments on channel aggregation and input perturbations, on learning rates, random seeds, and the training budget, and on uncertainty under dependent model–dataset comparisons (three new appendix sections and five new tables). The revised results support the main conclusions of the paper. Revised text is marked in blue; affected tables and figures are updated.

Summary by concern:

  • Channel aggregation (2s87, 8KfV): all models now concatenate their channel representations, and all main results were re-run; the relative order of the models is largely preserved (median Spearman ρ = 0.94; G1).
  • NinaPro DB1 preprocessing (2s87, 8KfV): DB1 was rebuilt without the 20 Hz filter and re-run for all models; in a controlled comparison with three models, the filter lowers macro-F1 by 8.7 to 20.2 points (G2).
  • Robustness to training choices (8KfV, gpyE): the sign of the pretraining gain is preserved in 8 of 9 learning-rate settings, and repeated seeds reproduce the positive gain of UniTS and the negative gain of PatchTST (G3).
  • Scope of cross-task conclusions (all reviewers): RQ4 now describes consistency across the evaluated tasks (G4).

G1. One channel readout for all models (2s87, 8KfV)

In the submission, only MOMENT concatenated its per-channel representations; the other models averaged them. All nine configurations now pool each channel over time and concatenate the channels before the linear classifier (Sections 3.3 and 4.1, Appendix C.9). LOSO and emg2pose (Appendix A.4) were not re-run and keep the original readout (see their captions). Appendix A.5 shows that this choice matters:

  • Under linear probing, concatenation gives higher macro-F1 than averaging in all 54 model–dataset comparisons (mean +23.8 points).
  • Permuting the electrode order at test time lowers the macro-F1 of fine-tuned MOMENT-S from 80.1 to 14.2 on GRABMyo, whereas averaging is unaffected by construction.
  • The sign of the MOMENT-S pretraining gain is the same under both readouts on all six datasets.

G2. NinaPro DB1 (2s87, 8KfV)

DB1 is distributed as the 100 Hz RMS-rectified output of Otto Bock electrodes. The submitted experiments resampled it to 1 kHz without band-pass or notch filtering; Appendix B.4 incorrectly suggested that the 20 Hz filter was applied. We rebuilt DB1 with all 52 gestures and a trial-level split, re-ran all models, and corrected Appendices B.1.1 and B.4. In a controlled comparison in which two DB1 versions differ only in this filter, filtering lowers the fine-tuned macro-F1 of MOMENT-S, PatchTST, and UniTS by 8.7, 8.9, and 20.2 points.

G3. Robustness to training choices (8KfV, gpyE)

Appendix A.2.1 tests whether the pretraining gain depends on training choices:

  • Training budget: for five configurations, the median selected epoch is 15.5 for fine-tuning and 23 for scratch training, and the best checkpoint precedes the final epoch in 176 of 180 runs.
  • Learning rate: for MOMENT-S, PatchTST, and UniTS on three datasets, selecting the learning rate separately for fine-tuning and scratch training preserves the sign of the gain in 8 of 9 cases.
  • Seeds: the gain of UniTS is positive in all 11 runs and that of PatchTST is negative in 10 of 12.

These runs used an earlier assignment of subjects to Pool A and Pool B; their gain signs agree with the main results for 9 of 11 model–dataset pairs.

G4. Updated findings and scope (all reviewers)

  • RQ1: full fine-tuning improves accuracy over linear probing in 140 of 162 model–dataset pairs (86.4%).
  • RQ2: pretrained models outperform their scratch counterparts in 121 of 162 pairs (74.7%), with mean gains of 6.4 accuracy and 7.4 macro-F1 points. Seven of the nine configurations benefit on most datasets, with the largest gains for TimesFM and Chronos-2; MOMENT-B and PatchTST perform better from scratch.
  • RQ3: accuracy decreases for new users in 147 of 162 pairs, and five-shot adaptation improves macro-F1 in 131 of 162 pairs (80.9%), recovering more than half of the average loss.
  • RQ4: model performance is highly consistent between upper- and lower-limb classification (Spearman ρ = 0.90 in accuracy and 0.93 in macro-F1) and moderately consistent with continuous EMG-to-text decoding (ρ = 0.65 and 0.61 between the CER ranking and the upper- and lower-limb macro-F1 rankings). Given four lower-limb datasets and one typing dataset, we now describe these as consistency across the evaluated tasks.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.