- Rating
- 1(−1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper aims to decode keystrokes live from surface electromyography. This study shifts the focus towards the decoder part instead of the encoder. Further, the proposed method expands the limited context size for better decoding. The proposed method achieves state-of-the-art performance on the emg2qwerty benchmark. An ablation study verifies the effectiveness of the proposed components.
Critical strengths
-
Performance: the proposed method achieves state-of-the-art performance on the emg2qwerty benchmark in the generic setting and presents a comparison of model sizes.
-
Ablation: the paper presents ablation studies in several settings to verify the effectiveness of the proposed components.
Critical weaknesses
-
Experiment setting: the emg2qwerty benchmark includes not only generic but also personalized settings. This study does not perform experiments in the personalized setting.
-
Incremental engineering effort: it seems the proposed method combines the reported research outcomes for better performance. Yet, the performance gain does not seem significant, given that this model uses extra data (Sec 3.3) and is x8 larger (2.2M vs 16.26M).
-
User study: the user study would verify the paper's claim on real-time decoding.
Questions for authors
-
Does your model run in real-time? What is the computational cost in edge computing settings?
-
Please clarify the CER of your causal encoder, which is the removal of the typing model from your model (causal encoder + typing model) in Table 1.
5 comments
I sincerely appreciate the author taking time for preparing this response.
I think my concerns on the parameter/computational efficiency and the significance of the technical contribution remain unresolved.
- NVIDIA H100 PCIe is not suited for practical deployment.
- The distilled model (2.2M) is also included as a baseline, and the proposed model has 16.24M parameters.
- The improvement from 6.91 to 6.53 does not seem significant taking the extra data into account.
We thank the reviwer for his comments and take this oppertunity to provide furthur evidence below,
1) Parameter size, distillation and extra data
We agree that a 2.2 million parameter model is better suited to edge deployment than a 16 million parameter model, and we acknowledge that our model has seen more data, but note that we already show in the main paper how this extra data benefits previously published baselines (Table 1, “Mehlman et al. Large + our typing model”), with potential benefits for future models. Mehlman et al. also use distillation to obtain their Tiny model. Following this approach, we extend the comparison by distilling both our encoder and typing prior to 1.91 million parameters combined, and evaluating their Tiny model with the same distilled typing prior. When we distill our system to a similar model size, we still outperform their model, both with and without our typing prior. We would also like to point out that this extra typing data is one of the central contributions of this paper.
| Model | Total parameters | Cross-user CER (%) | Personalized CER (%) |
|---|---|---|---|
| Mehlman Tiny + our distilled typing prior | 2.98M | 30.11 | 6.74 |
| Mehlman Tiny alone | 2.20M | 31.90 | 9.70 |
| Our distilled encoder + distilled typing prior | 1.91M | 28.17 | 6.54 |
Our model reduces cross-user CER by 1.94 percentage points compared with Mehlman Tiny using the same prior, and by 3.73 percentage points compared with Mehlman Tiny alone.
2) Edge computing and runtime
The reviewer is right that an H100 is not an edge device. We have therefore added streaming encoder measurements on an RTX 2080 Ti and CPU-only hardware. Encoder computation runs faster than real time in all three configurations, including 5.2× real time using a single CPU core.
Table R5: Streaming encoder on non-H100 hardware (1.9 M parameter model)
| Measurement | RTX 2080 Ti (11 GB) + 1 CPU core | CPU only, 1 core | CPU only, 4 cores |
|---|---|---|---|
| Compute real-time factor ↓ | 0.033 | 0.191 | 0.121 |
| Compute throughput | 30.6× real time | 5.2× real time | 8.3× real time |
| Chunk compute latency, median | 16.7 ms | 97.8 ms | 62.0 ms |
| Chunk compute latency, 95th percentile | 17.1 ms | 98.9 ms | 63.2 ms |
| Peak memory | 2.9 GB | 0.86 GB | 0.86 GB |
The H100 affects the reported latency. The fact that we show peak GPU tensor memory under 1 GB in the earlier table shows there is promise for our work on the edge.
3) Extra data
Creating the extra typing data is one of the central contributions of this paper. We also evaluate its benefit on previous models through our typing prior:
- Main paper, Table 1: Mehlman’s Large encoder with our typing prior achieves 26.77% cross-user CER, compared with its published 30.5%.
- Table R2: Released personalized TDS encoders improve from 6.91% to 6.53% CER, a 5.5% relative reduction, with improvements for all eight users.
- The new distillation comparison above: Mehlman Tiny improves from 31.90% to 30.11% cross-user CER when given our distilled typing prior.
The quoted 6.91% → 6.53% result therefore already measures transfer to an existing encoder. Tables 2–3 further examine the contribution of the typing data and how it is generated.
Response to Reviewer 8Kes
We thank the reviewer for the thoughtful review and for acknowledging our performance and ablation studies. We address the weaknesses and questions below.
W1 — Evaluation on the personalized benchmark
We additionally evaluate our typing prior with released personalized TDS encoders across all eight users. It reduces mean CER from 6.91% with the six-gram prior to 6.53%, improving performance for every user (Table R2).
Table R2: Personalized performance, CER (%).
W2(a) — Novelty and significance beyond combining existing components
Our main contribution researches a simple question: if the task is decoding typing, why is typing behavior not modeled?
We test this by varying the prior’s training data (Table 2), comparing random errors with our fitted typing process (Table 3), and removing individual retokenizer components (Appendix A.8, Table 12). We also introduce a new method for turning ordinary text into typing data, including errors and corrections which directly benefits EMG decoding.
Our system achieves state-of-the-art cross-user CER. Our typing prior also improves the previous SOTA's Transformer [1] (Table 1), showing that the benefit extends beyond our encoder. As our decoder is compatible with previous works, we hope future methods can build upon our work as a promising model for typing behavior.
W2(b) — Model size and additional training data
Our full system uses 16.24M parameters, compared with 109M for [1]’s strongest encoder (Table 1). Their best 2.2M model is distilled from a 109M teacher, so the comparison should also mention the larger model used to train it.
Creating the additional typing data is also one of our contributions. We turn ordinary prose into keystroke sequences using an error-and-correction process fitted to real typing. Tables 2–3 examine the benefit of this training data.
As mentioned previously, our typing prior also improves Mehlman et al.’s Transformer [1] (Table 1), showing that it can benefit other encoder architectures.
W3 — Live user study
Like prior work on emg2qwerty, we evaluate on publicly released recordings. We do not have access to the research wristbands needed for a live user study. We additionally test streaming by replaying recorded signals at their original acquisition rate. A live user study remains future work.
Q1(a) — Real-time decoding
We replayed a 20.1-minute recording at its original acquisition rate on one NVIDIA H100 PCIe, running the encoder and typing prior. Below are our statistics.
Table R3: Real time decoding statistics
Q1(b) — Edge computing and memory
Ideally, we would measure runtime on the intended edge device using live wristband input. We do not have access to that setup. We instead measured greedy encoder inference on an H100 as stream length increased. GPU memory plateaus near 71 MiB as the stream grows to 20 minutes, and the SSM state remains fixed at 0.79 MB (Appendix A.2, Table 6). Table R4 shows memory does not grow with how long the user has been typing, thus our model is lightweight and deployable.
Table R4: Streaming memory statistics
Q2 — Encoder-only performance
Our encoder-only performance is 37.58 ± 0.50% CER with greedy CTC decoding across five final encoder seeds. Please see Figure 1, Tables 2–3, Appendix A.5 (Table 9) and Appendix A.6 (Table 10) for the encoder-only comparisons and ablations. The drop in performance illustrates the importance of typing.
[1] Mehlman, N., Gagnon-Audet, J.-C., Shvartsman, M., Niu, K., Miller, A. H., & Sodhani, S. (2025). Scaling and Distilling Transformer Models for sEMG. Transactions on Machine Learning Research. See Appendix C.4.