acceptodds
Under review as a conference paper at ICLR 2027

PI-EMG: Enhancing Surface Electromyographic Decoding using Privileged Information

12 views

Abstract

The _emg2pose_ benchmark facilitates training generic models for surface electromyography (sEMG) hand pose decoding, which can be used for novel intuitive human-computer interaction modalities. This benchmark defines two tasks: tracking, where hand pose is predicted from sEMG starting from a ground-truth initial pose, and regression, where the initial pose is not provided. After stabilizing position training, position decoding reduced tracking errors by 3.5% – 5.1% relative to matched velocity models. On the regression task, we find that initial pose conditioning provides valuable privileged information (PI), with multi-task tracking and regression training reducing regression task error by 5.3% – 8.0%. Other privileged information that is not available at test time, such as metadata, can also be used to enhance training. To take advantage of metadata such as user ID and sample ID, we implemented Metadata-conditioned Layers (McLayers), which stochastically use either metadata-conditioned mappings or generic mappings during training, but default to generic mappings at test-time. Our model, PI-EMG, incorporates architecture and hyperparameter optimization, multi-task training, and train-time metadata-conditioning, to reduce causal pose decoding error by 10.0% – 16.4% for tracking and 10.0% – 15.7% for regression on unseen users relative to the vemg2pose baseline. PI-EMG therefore achieves a new published state-of-the-art for causal sEMG decoding on the _emg2pose_ benchmark. Likewise, for the _emg2qwerty_ benchmark, which evaluates two-handed sEMG typing, adding user-wise McLayers to the prior published state-of-the-art model reduced beam search character error rates by 18% for unseen users.

Primary area:
applications to neuroscience & cognitive science
Keywords:
surface electromyography, electromyography, hand pose estimation, human-computer interfaces, emg2pose

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_cpnAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents a framework for improving surface electromyography decoding through architectural refinements and training-time privileged information. The approach combines delta-position prediction, joint tracking and regression training, causal temporal feature modulation, and Metadata-conditioned Layers. McLayers stochastically alternate between metadata-specific and generic transformations during training, while using only generic transformations at inference. Experiments demonstrate improvements on emg2pose and emg2qwerty, including generalization to unseen users.

Critical strengths
  • Generalization across users is a central challenge in sEMG decoding. Evaluation on both continuous hand-pose estimation and typing provides useful evidence that McLayers can benefit different decoding tasks. The reduction in emg2qwerty character error rate is particularly encouraging.
  • The paper includes multiple training seeds, user-level statistical comparisons, and extensive ablations covering feedback, conditioning granularity, layer placement, routing probabilities, and computational cost. Matching the position and velocity decoders also makes their comparison more informative.
  • The inference design is practical, McLayers exploit training metadata without requiring user identity or sample-specific parameters at test time. Their transformations can potentially be merged into adjacent layers, making the approach attractive for deployment.
Critical weaknesses
  • My major concern is about the insufficient comparison with closely related works. The main methodological contribution is closely connected to existing approaches that separate shared and subject-specific parameters or exploit privileged information during training. Although relevant methods such as DResNet and TRAM are discussed, direct comparisons are missing. Consequently, the advantages of stochastic metadata routing over neighboring approaches remain unclear.
  • Improvements from sample-specific mappings suggest that additional training capacity or stochastic regularization may contribute substantially. Controls using size-matched random pseudo-user groups and metadata-independent stochastic transformations would help determine whether genuine user structure is essential.
  • Streaming and long-horizon behavior remain incompletely validated. The standard five-second evaluation provides limited evidence for sustained tracking, and the longer-horizon result is primarily illustrative. Furthermore, the temporal alignment of linear feature upsampling requires clarification to establish end-to-end causality.
  • Some gains require stronger attribution controls. The overall system combines several effective modifications, while the independent improvement from user McLayers on emg2pose is approximately 3–6% relative to the stronger baseline. For emg2qwerty, it is also important to establish whether the unmodified baseline was retrained with the same additional gradient clipping and evaluation configuration.
Questions for authors
  1. How do McLayers compare with a shared/subject-specific parameter baseline and a representative privileged-information method under matched training conditions?
  2. How much improvement remains when training examples are assigned to fixed random pseudo-user groups with approximately the same group sizes as the real users?
  3. What exactly constitutes a “sample” for sample-conditioned mappings, and at what granularity are routing decisions sampled?
  4. Does multi-task training outperform a regression-only baseline under matched computational budgets? Was the emg2qwerty baseline retrained using identical gradient clipping and evaluation settings?
  5. Can the authors clarify timestamp alignment during upsampling, provide a future-input perturbation test for causality, and report aggregate long-horizon tracking results across users?
5 comments

Thank you for your highly detailed review. We address each point individually, though the reviewer may prefer to read the Global Response:

W1: My major concern is about the insufficient comparison with closely related works. The main methodological contribution is closely connected to existing approaches that separate shared and subject-specific parameters or exploit privileged information during training. Although relevant methods such as DResNet and TRAM are discussed, direct comparisons are missing. Consequently, the advantages of stochastic metadata routing over neighboring approaches remain unclear.

We agree that our paper would greatly benefit from the inclusion of relevant baselines such as DResNet and TRAM. We now include these baselines. We found that TRAM improved decoding, though less than McLayers, while we were unable to find a DResNet configuration that improved decoding (Tables R1 and R2). Please see Global Response section 1 for details on implementation.

W2: Improvements from sample-specific mappings suggest that additional training capacity or stochastic regularization may contribute substantially. Controls using size-matched random pseudo-user groups and metadata-independent stochastic transformations would help determine whether genuine user structure is essential.

We agree that the extent to which user structure helps could be better determined by such controls. We now include size-matched random pseudo-user groups, formed by randomly shuffling the user-side metadata across training windows once at the start of training. We also test metadata-independent stochastic transformations by sampling the user-side metadata (with replacement) for every sample at every training step according to the global training distribution. Both approaches underperform McLayers that use unaltered metadata. Please see Global Response section 2 and Table R3 for more details.

W3: Streaming and long-horizon behavior remain incompletely validated. The standard five-second evaluation provides limited evidence for sustained tracking, and the longer-horizon result is primarily illustrative. Furthermore, the temporal alignment of linear feature upsampling requires clarification to establish end-to-end causality.

We agree that long-horizon behavior remains incompletely validated in emg2pose’s original 5-second evaluation setting, and have now added long horizon evaluations using 40-second windows (around the minimum length for individual recordings, beyond which many recordings become ineligible for this analysis). We find that PI-EMG-U outperforms all baseline models across time. We include Table R6 (new Table A.2) in the global response and add Figure A.4 to the paper itself.

W4: Some gains require stronger attribution controls. The overall system combines several effective modifications, while the independent improvement from user McLayers on emg2pose is approximately 3–6% relative to the stronger baseline. For emg2qwerty, it is also important to establish whether the unmodified baseline was retrained with the same additional gradient clipping and evaluation configuration.

We agree that McLayers are not the sole contributor to our overall performance improvement. To reduce confusion, in the Abstract we now separate the individual contribution of McLayers from the global improvement relative to Salter et al.'s baseline.

We have since retrained Ortone et al.'s S4D-large with gradient clipping on emg2qwerty over 4 training seeds, which achieves CERs of 40.4 and 32.0 for greedy and beam search decoding, respectively, using Sivakumar et al.'s evaluation protocol (Table R4). We now update the relative improvements from adding user McLayers S4D-large to 9.9% (greedy) and 13.1% (beam search). Additional testing on the CNN-Conformer from Wang et al., 2026 yielded improvements of 4.1% (greedy) and 7.5% (beam search) relative reductions in CER.

(1/2)

Q1: How do McLayers compare with a shared/subject-specific parameter baseline and a representative privileged-information method under matched training conditions?

We now compare McLayers to a shared/subject-specific parameter baseline (DResNet) and a privileged-information method (TRAM). While TRAM does improve decoding to a lesser degree than McLayers, DResNet does not improve decoding (Global Response section 1).

Q2: How much improvement remains when training examples are assigned to fixed random pseudo-user groups with approximately the same group sizes as the real users?

Some of the improvement of McLayers remains when training examples are assigned to fixed random pseudo-user groups with the same group sizes as the real users. For example, for User-Stage Tracking, the baseline ΔPos, TFiLM achieves 9.97° AE, while shuffled metadata achieves 9.87° and PI-EMG-U achieves 9.53°. Please see Global Response section 2 for more details.

Q3: What exactly constitutes a “sample” for sample-conditioned mappings, and at what granularity are routing decisions sampled?

For sample-conditioned mappings, a "sample" refers to a window of data associated with a particular anchor start time step and dataset index. For routing, all time steps within a window are routed identically, to either the generic mapping or the mapping associated with that dataset index.

Q4: Does multi-task training outperform a regression-only baseline under matched computational budgets? Was the emg2qwerty baseline retrained using identical gradient clipping and evaluation settings?

For multi-task training, we trained a version of Pos MT -FB whose computational budget matches that of single-task models, by sampling across tasks rather than weighting their respective losses. We found that sampled multi-task training, which is more efficient, matched the performance of weighted multi-task training (Global Response section 5, Table R7). We have now retrained the S4D-large emg2qwerty baseline using identical gradient clipping and evaluation settings, and have updated all of our comparisons (Global Response section 3, Table R5).

Q5: Can the authors clarify timestamp alignment during upsampling, provide a future-input perturbation test for causality, and report aggregate long-horizon tracking results across users?

We clarify timestamp alignment during upsampling and report a causality test using backpropagation as well as aggregate long-horizon tracking results across users in Global Response section 4.

(2/2)

Dear Reviewer cpnA,

We thank you again for your constructive feedback. We wish to politely remind you that the end of the author-reviewer discussion period draws nigh, and want to check if our previous responses and updated manuscript have addressed each of your concerns, as well as whether there are any last-minute questions or concerns.

Kindest regards, The Authors

Reviewer_aMZfOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PI-EMG improves surface EMG decoding through joint tracking/regression training, metadata-conditioned layers used during training, delta-position prediction, and causal temporal feature modulation. Experiments on emg2pose and emg2qwerty demonstrate improved generalization to unseen users without requiring their metadata at inference.

Critical strengths
  1. consistent gains across two distinct tasks: On held-out users, PI-EMG reduces tracking angular error from 7.70° to 6.93° and regression error from 12.24° to 11.01°. Adding user-conditioned layers to S4D-large reduces language-model-assisted typing CER from 34.0% to 27.8%. This supports practical relevance beyond one decoder or output task.

  2. strong component analysis: Multiple training seeds, user-level statistical comparisons, feedback ablations, identity-mapping controls, and computation-matched TFiLM experiments strengthen the evidence. The corrected position-versus-velocity comparison is particularly informative.

Critical weaknesses
  1. the specific value of meaningful metadata remains incompletely isolated. User-ID probes and sample-conditioned variants provide useful evidence, but a capacity-matched random-grouping control is missing. Direct comparison with an established privileged-information approach, such as TRAM, would also clarify the contribution beyond existing methods.

  2. the typing comparison includes another training change. Appendix A.8 introduces gradient clipping alongside McLayers, but does not report an otherwise identical S4D-large baseline trained with that change. This leaves uncertainty about how much of the reported typing improvement comes specifically from McLayers.

Questions for authors
  1. how do McLayers perform when training examples are randomly assigned to groups with the same group sizes and parameter count as the user-conditioned model? This would distinguish meaningful user conditioning from additional training capacity and stochastic regularization.

  2. was the S4D-large baseline retrained with the same gradient clipping and training configuration as the McLayer variants? If so, please report that matched comparison.

2 comments

We thank the reviewer for recognizing our method’s consistent gains across two distinct tasks and the strong analysis of the contributions of individual components.

W1. the specific value of meaningful metadata remains incompletely isolated. User-ID probes and sample-conditioned variants provide useful evidence, but a capacity-matched random-grouping control is missing. Direct comparison with an established privileged-information approach, such as TRAM, would also clarify the contribution beyond existing methods.

We now include both randomized controls and comparisons to existing methods. When user metadata is shuffled across training samples or randomly sampled during training, McLayers retain the same parameter count and stochastic routing, but achieve substantially smaller improvements than when using accurate user metadata. For example, on User-Stage Tracking, shuffled and randomized metadata achieved errors of 9.87° and 9.84°, respectively, compared to 9.53° with accurate metadata and 9.97° for the baseline (Table R3). These results suggest that while stochastic regularization contributes to the improvement, accurate user metadata provides substantial additional benefit. We also implemented three variants of TRAM, the best of which achieved 9.85° on User-Stage Tracking, compared to 9.53° for McLayers (Table R1). These results further support the effectiveness of our proposed approach relative to existing privileged-information methods.

W2. the typing comparison includes another training change. Appendix A.8 introduces gradient clipping alongside McLayers, but does not report an otherwise identical S4D-large baseline trained with that change. This leaves uncertainty about how much of the reported typing improvement comes specifically from McLayers.

We agree that the contribution of gradient clipping should be isolated from that of McLayers. We have now retrained S4D-large using the same gradient clipping and evaluation protocol as S4D-large +U. Gradient clipping alone reduces greedy and language-model-assisted CER from 42.7% and 34.0% to 40.4% and 32.0%, respectively. Adding McLayers further reduces these errors to 36.4% and 27.8%, corresponding to relative improvements of 9.9% and 13.1% over the matched baseline (Table R4). These results demonstrate that the majority of the original improvement comes from McLayers rather than gradient clipping.

Q1. how do McLayers perform when training examples are randomly assigned to groups with the same group sizes and parameter count as the user-conditioned model? This would distinguish meaningful user conditioning from additional training capacity and stochastic regularization.

TRAM and random grouping comparisons

After applying user-conditioned TRAM to the ΔPos, TFiLM model, we found that TRAM does improve decoding errors, but not to the degree that user-conditioned McLayers do (Table R1).

Randomized controls

When shuffling or randomly sampling user metadata, we found that generalization did improve over the baseline, though much less than using unaltered metadata (Table R3). These results suggest that stochastic regularization does contribute to the performance improvement, but the full improvement requires the structure represented in accurate user metadata.

Q2. was the S4D-large baseline retrained with the same gradient clipping and training configuration as the McLayer variants? If so, please report that matched comparison.

After submission, we retrained S4D-large with gradient clipping for 4 seeds, and found that it achieved average CERs of 40.4 and 32.0 for greedy and beam search decoding, respectively. Thus the user-conditioned McLayers reduced errors by 9.9% and 13.1% for S4D-large.

(1/1)

Dear Reviewer aMZf,

We thank you again for your constructive feedback. We wish to politely remind you that the end of the author-reviewer discussion period draws nigh, and want to check if our previous responses and updated manuscript have addressed each of your concerns, as well as whether there are any last-minute questions or concerns.

Kindest regards, The Authors

Reviewer_iy5HOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper revisit the emg2pose benchmark, and defines 2 new tasks: [1] tracking, which uses sEMG to predict hand pose with its initial position provided; and [2] regression, where the initial pose is not provided. With this distinction, the paper frames the initial position as a privilege information problem. Paper also proposed to add metadata using a Metadata-conditioned Layers (McLayers) to help with training. The new baseline model PI-EMG improves the previous emg2pose baseline by a decent margin.

Critical strengths

Paper presents solid experimental effort to update baseline performance on the emg2pose dataset, which is a very meaningful contribution to the community. The proposed method, McLayer, is simple in design, and its effectiveness is demonstrated experimentally.

Critical weaknesses

It would be nice to investigate why McLayer works. Authors could consider benchmarking with alternative way of using user identity.

Framing the work as a privileged information learning problem is interesting, but the authors should benchmark against other PI-related methods to support this framing.

More experimental results on emg2qwerty would be helpful.

Questions for authors

NA

2 comments

We thank the reviewer for recognizing the simplicity and effectiveness of McLayers as well as the importance of our updated baseline experiments.

We respond to your points one-by-one:

W1: It would be nice to investigate why McLayer works. Authors could consider benchmarking with alternative way of using user identity.

We tested McLayers when metadata was shuffled or randomly sampled (Table R3). In both cases, the models performed better than the baseline, but less than McLayers with unaltered metadata. These suggest that some of the improvement stems from the regularization effect from random routing, with the rest of the improvement requiring metadata that accurately represents user identity.

W2: Framing the work as a privileged information learning problem is interesting, but the authors should benchmark against other PI-related methods to support this framing.

We now test the ΔPos, TFiLM model by adding a different PI method, TRAM, and found that its method of using privileged information does not perform as well as McLayers (Table R1). We similarly tested DResNet, but found that it did not improve performance over the baseline (Table R2).

W3: More experimental results on emg2qwerty would be helpful.

We now additionally test McLayers as a drop-in addition to a CNN-Conformer model for emg2qwerty (Wang et al., 2026). We found that it improved character error rates (CER) by 4.1% and 7.5% for greedy and beam search decoding, respectively (Table R4). This result suggests that the improvements we find for the S4D baseline is not unique to that architecture.

(1/1)

Comments on the paper

3 comments
AuthorsOpenReview ↗

Global Response

Global Response

We thank the reviewers for their thoughtful responses. For convenience, we provide this global response to points from all reviewers, as well as respond more concisely to each review individually. We describe each method in detail and include tables only once to reduce redundancy.

1. Comparing to subject-specific weights (DResNet) and existing privileged information (TRAM) baselines

We add these analyses as new appendix section A.9.

TRAM

Following Collier et al., we apply the one-step TRAM method, which uses a shared feature extractor φ. For this feature extractor we choose the TDS network with TFiLM layers, i.e. the same as PI-EMG models. For the PI-path feature extractor ψ, which directly follows φ, we implement three variants: 1) an affine layer per user-side pair (similar to user McLayers), 2) a 512-dimension 1-layer MLP, and 3) an affine layer. For variants 2 and 3, we concatenate a 64-dimension user-side embedding, shared across time, to the shared feature extractor's 64 features. We use separate but architecturally identical decoder modules for the non-PI and PI paths. We report the results of the three variants in the below table. Please note that due to the stop gradient operation, the relative loss weighting between PI and no-PI outputs has little to no effect, thus we do not sweep over this hyperparameter.

Table R1 (new Table A.7). Angular error (°) for ΔPos, TFiLM variants with persistent reference pose inputs. ΔPos, TFiLM and PI-EMG-U have 3 seeds, while other rows have 1 seed.

ModelUser-Stage TrackingUser TrackingStage TrackingUser-Stage RegressionUser RegressionStage Regression
ΔPos, TFiLM9.977.209.9514.2911.3813.69
ΔPos, TFiLM, TRAM, per-user-side affine layers10.067.2310.1114.2511.5314.13
ΔPos, TFiLM, TRAM, concatenated user-side embeddings, MLP9.957.1410.0414.2611.4213.96
ΔPos, TFiLM, TRAM, concatenated user-side embeddings, affine9.857.149.9814.0911.4013.78
PI-EMG-U (i.e. ΔPos, TFiLM +User McLayer)9.536.999.6513.4911.0313.28

We find that variant 3, with the affine PI-path feature extractor ψ, does improve upon its baseline ΔPos, TFiLM, but not as much as user McLayers. This may suggest that jointly learning non-PI and PI paths, as in McLayers, may have advantages over TRAM, which learns the PI path independently of the non-PI path.

DResNet

We tested DResNet parameter splitting for three variants: to all CNN feature extractor layers, to linear layers at the start and end of the CNN, or only at the end of the CNN. The baseline model was ΔPos TFiLM. The former two suffered from unstable training. For the latter, we tested three domain classifier loss weights, [1e-2, 1e-3, 1e-4], with one seed each, with no configuration showing improvement. We report these results below:

Table R2 (new Table A.8). Angular error (°) for DResNet over several domain classification loss weights. Shared and user-specific parameters were only applied to an affine layer after the feature extractor, as other variants had unstable training.

ModelUser-Stage TrackingUser TrackingStage TrackingUser-Stage RegressionUser RegressionStage Regression
ΔPos, TFiLM9.977.209.9514.2911.3813.69
DResNet, λ=1e-211.388.4912.1914.6711.9315.47
DResNet, λ=1e-311.227.8811.8815.1011.9515.81
DResNet, λ=1e-410.917.9611.6115.0012.0015.74

Unfortunately, Ma et al. do not report the actual architecture for their models beyond describing the hand-crafted features and naming a "random search strategy" for their deep models without further explanation. Additionally, we were unable to find a reference repository on the GitHub profiles of any of the listed authors, limiting the reproducibility of DResNet. We therefore cannot determine the layer types, depth, and other hyperparameters of their models. These discrepancies may suggest that improvements from using DResNet do not extrapolate to other applications of neural decoding that do not use hand-crafted features or that have deep architectures.

References

Mark Collier, Rodolphe Jenatton, Effrosyni Kokiopoulou, and Jesse Berent. Transfer and marginalize: Explaining away label noise with privileged information. In International conference on machine learning, pp. 4219–4237. PMLR, 2022.

Bo-Qun Ma, He Li, Wei-Long Zheng, and Bao-Liang Lu. Reducing the subject variability of eeg signals with adversarial domain generalization. In International Conference on Neural Information Processing, pp. 30–42. Springer, 2019.

(1/3)

2. Isolating the role of metadata

To better understand the effect of user ID and hand side metadata, we shuffled user-side pairs across samples, where each sample during training represents a 2.895 second window of data (5790 input time steps, 4000 output time steps) associated with a particular dataset index. Within each McLayer, all time steps within each window are routed identically. Routing for separate McLayers were sampled independently. The metadata shuffling ensures that the counts in each user-side pair remain constant, but the samples corresponding to each pair are reassigned. We also tested performance when the user-side metadata was randomly sampled (with replacement) from its distribution in the training set, such that the groups are not static.

Table R3 (new Table A.9). Angular error (°) for models where user-side metadata were shuffled across training windows.

ModelUser-Stage TrackingUser TrackingStage TrackingUser-Stage RegressionUser RegressionStage Regression
ΔPos, TFiLM9.977.209.9514.2911.3813.69
PI-EMG-U, shuffled user9.877.179.9714.0611.2713.34
PI-EMG-U, random user9.847.139.7613.8411.2813.14
PI-EMG-U9.536.999.6513.4911.0313.28
PI-EMG-S9.687.029.6113.7511.1113.16
PI-EMG-US, shuffled user9.586.979.5813.7311.0913.31
PI-EMG-US9.426.939.5913.5411.0113.51

We find that McLayers with randomized metadata significantly improved performance over the baseline, consistent with a regularization effect from random routing. Beyond this, utilizing accurate user metadata significantly enhances these gains for all tasks and conditions except Stage Regression (seen users, unseen movements). In particular, user conditioning additionally significantly improved performance for unseen users. Please see Table R3b for statistical tests regarding these comparisons.

For sample-conditioned mappings, a "sample" refers to a window of data associated with a particular anchor start time step and dataset index. For routing, all time steps within a window are routed identically, to either the generic mapping or the mapping associated with that dataset index.

3. Like-for-like emg2qwerty comparison

We have now retrained S4D-large with gradient clipping (+clip), and found that it contributes to a minority of the improvement between S4D-large vs. S4D-large +clip +U. We additionally trained three seeds of the best variant of Wang et al.'s CNN-Conformer, which we report in the updated table. We have updated the relative improvement values to reflect these changes. These results demonstrate that drop-in addition of McLayers can consistently improve performance on the emg2qwerty task.

Table R4 (Paper Table 3). Character error rates (CER) on unseen users. Percentages represent the relative reduction in CER relative to the previous row.

ModelGreedy zero-shot CER6-gram LM zero-shot CER
S4D-large, as reported by Ortone42.7 ± 0.634.0 ± 0.7
S4D-large released checkpoint, re-evaluated42.235.0
S4D-large +clip40.4 ± 0.6 (-4.3%)32.0 ± 0.7 (-8.6%)
S4D-large +clip +U36.4 ± 0.6 (-9.9%)27.8 ± 0.3 (-13.1%)
CNN-Conformer (Wang et al., 2026)29.619.9
CNN-Conformer +U28.4 ± 0.8 (-4.1%)18.4 ± 0.9 (-7.5%)

Regarding evaluation of the reported models, Mehlman's evaluation scheme is a clear outlier, as it does not follow the repository and dataset pipeline set forth by Sivakumar et al. For completeness, we report evaluation on 5-second windows (which includes 900ms past and 100ms future context; only 4 seconds are predicted), with a window stride of 4 seconds. This evaluation protocol seems to favor transformer models, which are able to attend to the entire 5-second sEMG window, unlike the S4D-large model, whose constituent layers are causal.

Table R5. Character error rates (CER) when evaluating using 5-second windows rather than full recording sessions.

ModelGreedy zero-shot CER
Distilled transformer tiny (Mehlman et al., 2025)31.9
S4D-large +clip +U39.1
CNN-Conformer +U29.3

References

Yuanxi Wang, Lun Shu, and Yasuharu Koike. sEMG Typing Transcription with Conformer Encoding and Beam-Aware Large Language Model Correction. Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pp. 13763-13780, 2026.

(2/3)

4. Causality and long horizons

Upsampling of features was performed using PyTorch's torch.nn.functional.interpolate(pred, size=n_time, mode="linear") . This is identical to the upsampling used by Salter et al. Causality testing using backpropagation demonstrated that additional latency due to upsampling never exceeds 1 feature frame (40 ms). We add this description to Appendix A.7.

We now aggregate long-horizon decoding errors (up to 40 seconds), and show them in new Figure A.4 and new Table A.2. Qualitatively, we find that PI-EMG-U outperforms Salter et al.'s vemg2pose model across all time steps.

Table R6 (new Table A.2). Angular error (°) comparison for 40-second decoding (3 seeds).

ModelUser-Stage TrackingUser TrackingStage TrackingUser-Stage RegressionUser RegressionStage Regression
vemg2pose (Salter)14.3410.7114.4817.2013.7316.64
Vel ST14.0610.8914.0116.5713.5615.96
Vel +MT13.9610.8413.7014.7212.4513.84
ΔPos MT -FB13.4610.3313.5314.5111.9313.82
ΔPos, TFiLM13.1710.2213.2914.2811.7713.68
PI-EMG-U12.429.7712.6713.5311.4413.33

5: Matched computation

We trained a version of Pos MT -FB where the initial position was randomly sampled to be provided (probability 87.5%) or omitted (probability 12.5%), such that probabilities matched the multi-task loss weights. In this setup, the computation budget matches that of the single-task training runs. This produced comparable error rates to its baseline.

Table R7 (new Table 6). Angular error (°) comparison for sampled multi-task training (3 seeds).

ModelUser-Stage TrackingUser TrackingStage TrackingUser-Stage RegressionUser RegressionStage Regression
Pos MT -FB10.267.3410.2814.4711.5114.02
Pos MT-sampled -FB10.267.3610.2314.4711.4714.04

Additional clarifications

Clarifying relative errors

We calculate the relative reductions in error of B relative to A as 1 - error_B/error_A. Thus, an improvement has a positive error reduction. We apologize for the confusion regarding relative improvements in the abstract vs. results. The abstract refers to the range of error reduction across both angular error and landmark distance metrics, whereas in our initial submission, the reduction values were specifically described as the range for angular errors. For simplicity, we revise our abstract, introduction, and results to summarize both metrics in a single range.

We additionally separate the relative performance improvements of McLayers by comparing PI-EMG variants vs. ΔPos TFiLM. We revise the total improvements compared to Salter's from 10.0-16.4%/10.0-15.7% for tracking/regression, which previously compared the best models to Salter's for only the User-Stage and User generalization conditions, to simply 8.8-16.4% which covers angular and landmark distance errors across all PI-EMG variants and across both tracking and regression tasks. We hope that these changes can reduce confusion for all readers.

Selecting hyperparameter p_g

The generic probability hyperparameter p_g was chosen based on preliminary training runs using only the validation set. We now clarify this in the corresponding caption (old Table A.10, new Table A.15), as well as include the below table as new Table A.16. Validation errors from these preliminary training runs are shown below. These checkpoints were never tested on the test set. Table R8 (new Table A.16). Validation losses for selecting the generic routing probability hyperparameter p_g.

Sample p_g \ User p_g0.1250.250.50.75
0.250.3160.3120.3140.316
0.50.3150.3150.3170.321
0.750.3170.3150.3190.323

(3/3)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.