acceptodds
Withdrawn from ICLR 2027

Scaling Wrist sEMG for Hand Pose Estimation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Surface electromyography (sEMG) offers a promising alternative to vision for hand pose estimation, providing information that is complementary to visual observations. However, unlike vision, sEMG-based hand pose estimation has received limited attention. In this work, we present CycloFormer, a rotation-invariant Transformer that encodes the cyclic symmetry of the wrist-worn sEMG electrode ring into its architecture. It achieves state-of-the-art landmark-distance performance on emg2pose, and exhibits systematic scaling with model size and training data. Through comparison with the leading vision baseline, we show that an sEMG-based pose-estimation pipeline becomes more accurate when visual evidence is limited. We also derive a scaling law for sEMG hand-pose regression in the data-scarce regime. It identifies the loss-minimising model size at the current data scale and suggests that large-scale data collection is a key direction for advancing sEMG-based hand pose estimation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Surface Electromyograms, Deep Learning, Hand Pose Estimation, Scaling Laws, Vision Hand Pose Estimation, Bio-signals

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_wZTeOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes CycloFormer, a Transformer that estimates hand pose from a 16-electrode wrist sEMG band and is exactly invariant to cyclic shifts of the electrodes, i.e. to rotating the band. It reports the best landmark distance on all three emg2pose splits (Table 1). It then compares sEMG with monocular vision (HaMeR, HandOccNet) at different levels of fingertip occlusion, on rendered emg2pose poses and on real paired EgoEMG data, and finds that sEMG is more accurate when most fingertips are hidden. Finally, it fits a scaling law over a 32-run grid of model width and training fraction and concludes that the task is data-limited. No new data are collected; the data axis subsamples the existing emg2pose training set.

Critical strengths

The rotation-invariant design is simple and, as far as I can see from App. B.1.1 and the released code, correct: the error is flat over all 16 shifts (Fig. 5). It removes the need for band-rotation calibration, which matters for a device that people put on every day, and the LD gains over vemg2pose (1.1/1.9/1.6 mm, Table 1) are clear. Code and checkpoints are released. The cross-modal setup is also built with care (renderer check on FreiHAND, retargeting floor, real paired data).

Critical weaknesses
  1. Misleading title and an unsupported scaling claim.

The motivation and design of the scaling study do not hold up, starting with the title. "Scaling Wrist sEMG" suggests that sEMG data are scaled up, but no new data are collected: the study trains eight model widths (1.49M to 84.77M) on 20–100% subsets of the existing emg2pose training set. In my view, the title is therefore kinda misleading and overclaims what the paper actually does. The message that large-scale data collection is a key direction is therefore an extrapolation beyond the largest dataset tested. For contrast, EgoScale (Zheng et al., 2026), which the paper itself cites, trains on 20,854 hours of egocentric human video and reports a log-linear relation between data scale and validation loss. I think this is the right kind of design for testing a scaling law, because the data themselves are scaled up. I also do not see why larger models should be the goal for wrist sEMG, whose appeal is accurate pose from a cheap signal with lightweight, real-time inference on the device. The paper reports neither on-device cost (only A100 time, Table 18) nor a causal model. The fitted law itself is confounded by the training protocol. All runs use 100 epochs and the same learning-rate schedule while the per-GPU batch grows with width (64 to 768, Table 7), so at a fixed GPU count, large models on little data get far fewer updates. This alone could produce the kind of N/D penalty that Eq. 1 adds. The U-shape rests mostly on one single-seed run (84.77M at 20%), and App. E.1 notes that ε∞ is poorly identified, so the claim that the task is data-limited (β > α) is not established. This is the paper's main framing, and fixing it needs new data or a re-run of the grid, which is beyond a rebuttal.

  1. Confounded comparison with vision.

The occlusion crossover is confounded with hand pose. The paper sorts frames into groups by how many fingertips are occluded (0 to 5). sEMG cannot see occlusion, yet CycloFormer's own error drops in the high-occlusion groups (EgoEMG: 17.30 to 14.56 mm, Table 3; renders: 12.36 to 10.59 mm, Table 25). These groups probably hold different, more closed poses, on which CycloFormer's PA-MPJPE is lower anyway: 11.02 mm on closed hands against 12.71 mm on open hands, and Table 26 reports no open-hand value at 5/5. On real EgoEMG data, vision is more accurate from 0 to 4 occluded fingertips, and sEMG wins only when all five are occluded (14.56 vs. 20.61 mm, Table 3). The paper does not say how many frames, recordings or subjects fall into this 5/5 group. Fully occluded frames are likely rare (the paper notes that most emg2pose frames have few occluded fingertips), so if the group is small or comes from a few subjects, the advantage may reflect chance or a few individuals rather than a general effect. The 95% intervals in Fig. 3b do not settle this, because the paper does not say how they were computed for EgoEMG, unlike the session bootstrap it uses for the renders (p. 8). If frames rather than subjects were resampled, the intervals would look tighter than they should, since frames from the same recording are strongly correlated. The vision side is also not the strongest available: HaMeR (CVPR 2024) is called "the leading vision baseline", but newer models report better accuracy on standard benchmarks, for example WiLoR (CVPR 2025), which the paper itself cites. As it stands, the evidence does not separate "sEMG is robust to occlusion" from "high-occlusion frames are easier poses".

Taken together, the framing in the title and abstract and two of the three contributions are not supported by the current evidence. This is why I recommend rejection despite the solid invariance result.

Questions for authors

Positive answers to Q2 and Q3 would move me toward weak rejection; the title and the lack of new data would remain. Q1 and Q4 affect how far I can trust the reported gains.

  1. Attention implementation (clarification).

App. B.1 states that applying the rotary embedding before W_q and W_k "destroys the symmetry". Yet the released code keeps "legacy" attention modes for "pre-RoPE-bug-fix checkpoints" (cs_tds_ct_arch.py, l. 129–152): one applies RoPE before the projections, and two rotate over the head axis instead of the electrode or time axis, so they carry no positional information. Were all 80× results (e.g. Table 2, Fig. 5, Table 27) produced with rope_mode="fixed"? If any were produced with a legacy mode, those results, in particular the CRoPE ablation in Table 2, would need to be redone.

  1. The EgoEMG 5/5 group (clarification and analysis).

How many frames, recordings and subjects fall into each EgoEMG occlusion group, especially 5/5? Were the 95% intervals in Fig. 3b computed by resampling frames or subjects? Please also check whether the 5/5 advantage holds under two stricter tests. (a) A subject-level bootstrap, which resamples whole subjects rather than single frames. Frames from the same subject and recording are highly correlated, so resampling frames makes the intervals too narrow, and an advantage that comes from only a few subjects would not survive this test. (b) A pose-matched comparison, which compares sEMG and vision on the same or similar hand poses at different occlusion levels, for example by rendering each emg2pose pose from several viewpoints, or by comparing within groups of similar poses on EgoEMG. The high-occlusion groups are likely dominated by closed hands, on which CycloFormer is more accurate anyway (11.02 vs. 12.71 mm, Table 26), so this test separates the effect of occlusion from the effect of hand pose.

  1. The scaling grid (additional analysis).

Does the upturn at (84.77M, 20%) persist when that run gets as many updates as the smaller models (or uses the same batch size) and a tuned learning rate? Please also report the training loss for each cell (at present it is shown only for the 1.49M and 4.18M seed runs, Fig. 6).

  1. Causality and deployment (clarification and analysis).

The temporal attention has no causal mask and the convolutions use symmetric padding (cs_tds_ct_arch.py, e.g. l. 511 and l. 779), so each frame also sees future samples within the 5-s window. vemg2pose, in contrast, uses a left context and a sequential LSTM rollout (App. B.4.4–B.4.5). Are the Table 1 comparisons made under the same causal constraint, and what is the latency on wearable-class hardware?

Reviewer_PFh7OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces CycloFormer, a rotation-invariant Transformer for hand pose estimation from wrist-worn sEMG signals. The proposed architecture incorporates the cyclic structure of the electrode array through channel-shared convolution, spatial-temporal attention, and circular positional encoding. The authors evaluate CycloFormer on emg2pose, compare sEMG-based and vision-based pose estimation under different levels of fingertip occlusion, and investigate the effects of model size and training-data availability through an empirical scaling law.

Critical strengths
  1. The proposed architecture incorporates the cyclic structure of wrist-worn electrodes, ensuring invariance to cyclic channel shifts by design rather than relying solely on data augmentation. This provides a principled approach to handling cyclic variations in electrode-channel alignment.
  2. The comparison between sEMG and vision-based methods using paired RGB-sEMG data provides useful insights into their complementary strengths under different levels of occlusion.
  3. The scaling-law analysis offers valuable insights into the relationship between model capacity and training data, highlighting the importance of data availability for improving sEMG-based hand pose estimation.
Critical weaknesses
  1. The rotation-invariance experiments rely on synthetic cyclic channel shifts rather than actual wristband displacement. Since invariance to these shifts is guaranteed by the architecture, evaluating physical rotation or repositioning would provide stronger evidence of robustness under realistic electrode-placement changes.
  2. The comparison with vision-based methods is limited by differences in temporal input. While CycloFormer uses sEMG sequences, HaMeR primarily processes individual images. Although temporal smoothing is evaluated, comparing against video-based hand pose estimators, such as Deformer (Fu et al., ICCV 2023) and H2ONet (Xu et al., CVPR 2023), would further clarify the relative advantages of sEMG under severe occlusion.
  3. The generalizability of the proposed scaling law remains unclear, as the experiments are limited to a specific architecture family and dataset. Although the proposed formulation outperforms the Chinchilla-based baseline in two of three held-out evaluations, it performs worse in the remaining setting. Additional experiments on different sEMG datasets or architectures would help establish its broader applicability.

Fu, Q., Liu, X., Xu, R., Niebles, J. C., & Kitani, K. M. (2023). Deformer: Dynamic fusion transformer for robust hand pose estimation. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 23543-23554). IEEE. Xu, H., Wang, T., Tang, X., & Fu, C. W. (2023). H2onet: Hand-occlusion-and-orientation-aware network for real-time 3d hand mesh reconstruction. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 17048-17058). IEEE.

Questions for authors
  1. How does CycloFormer perform under actual physical rotation or repositioning of the wristband, rather than synthetic cyclic channel permutations?
  2. Since removing CRoPE preserves cyclic invariance, what additional advantages does it provide over simpler permutation-invariant architectures?
  3. Would the relative performance of sEMG and vision-based approaches change when using a video-based hand pose estimator that explicitly models temporal information, such as Deformer or H2ONet?
  4. How well does the proposed scaling law generalize to different sEMG datasets or model architectures? Are the fitted scaling parameters expected to remain consistent across different electrode configurations or recording conditions?
Reviewer_ugxhOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents CycloFormer, a rotation-invariant Transformer for 3D hand pose estimation from wrist sEMG signals. The method uses shared CNN weights, circular rotary position embeddings, and spatial-temporal attention to handle cyclic shifts of wristband electrodes. Experiments on emg2pose demonstrate competitive performance. The paper further compares CycloFormer with vision-based methods, including HaMeR, showing that sEMG is more robust under severe fingertip occlusions.

Critical strengths

The paper addresses the important problem of rotation invariance in wrist sEMG-based hand pose estimation and provides both quantitative and qualitative results to evaluate the proposed method.

Critical weaknesses
  1. The paper highlights the complementary benefits of sEMG and vision, but only compares the two modalities separately. A simple vision+sEMG fusion experiment would help quantify how much CycloFormer improves vision-based hand pose estimation, particularly under severe occlusions, and better support this claim.

2.The paper provides limited descriptions of the datasets and evaluation protocols. In particular, the User, Stage, and User+Stage splits are not clearly explained.

3.The paper could be better organized. Some important implementation details are only provided in the appendix, making the experimental setup difficult to follow.

4.In Table 1, CycloFormer does not consistently outperform EMGFormer-S.

Questions for authors

see the weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.