acceptodds
Under review as a conference paper at ICLR 2027

OmniIMU-Bench: Benchmarking Wrist IMU Representations Across Kinematic, Semantic and Clinical Tasks

Abstract

Wrist-worn inertial measurement units (IMUs) provide a common sensing modality for tasks ranging from activity and gesture recognition to pose estimation, inertial odometry and clinical motor assessment. Existing wearable benchmarks evaluate performance across multiple tasks, but leave open how supervision for one task af- fects transfer to others and whether these tasks benefit from a shared representation. We present OmniIMU-Bench, a benchmark of ten tasks from eight datasets for studying IMU multi-task performance and cross-task transfer. We compare six self- supervised methods pretrained on unlabelled recorded and simulated wrist motion, evaluating classification, pose and velocity estimation, and IMU–language retrieval. We find that masked autoencoding achieves the best or joint-best frozen-probe performance on nine of ten tasks, and fine-tuning improves on training from scratch on nine of ten tasks, including full-body pose, odometry and retrieval. Across trans- fer and joint-training experiments, supervision that benefits one target can reduce performance on others, with gains in kinematic estimation accompanied by losses in language retrieval and joint-training benefits differing between paired tasks. These findings distinguish broad downstream usefulness from complementary task supervision. Code, splits, and a leaderboard will be released.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Representation Learning, IMU

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 5 reviews, above 27% of the others.
Reviewer_Yhi6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a framework for inertial measurement using 10 tasks from 8 different datasets to examine the learned representation performance in pose, gesture, language retrieval, clinical, and activity tasks. They present a masked auto-encoder network and examine how it, and 5 other self-supervised IMU models, perform in cross-task transferability between frozen transfer, fine-tuned transfer, and training from scratch. IMU benchmarks are fragmented across tasks, so the unification of many common IMU tasks helps to provide a more general guiding framework to improve IMU foundation models. The comparison of performance for downstream tasks to address their complementary features similarly helps to identify what commonalities can be exploited for different training subsets.

Critical strengths

The paper seeks to improve current IMU model benchmarking and uses a broad set of tasks to analyze different task performance across different self supervised models. Comparing the supervision downstream performance is a very useful proxy to determine what tasks may benefit from a shared representation and help to guide designers in the creation of effective foundation IMU models.

Comparing the supervision downstream performance is a very useful proxy to determine what tasks may benefit from a shared representation and help to guide designers in the creation of effective foundation IMU models.

Critical weaknesses

Much of the data provided has little analysis or explanation beyond what the values are. There is very little discussion of results.

The relevant works section is brushed over with minimal description and detail of past works.

Some of the figures are very confusing or difficult to read or add little to the discussion. The radar plot for Figure 2 is very difficult to read. The following table provides much more useful information for comparing the performance of the different models. The space of the image could have been used to analyze the results more in-depth.

Figure 3 doesn't have axis labels for the map. Shading makes it harder to understand the difference compared to a table

Questions for authors

n/a

Reviewer_LpY8OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

OmniIMU-Bench evaluates wrist-IMU representations on ten tasks from eight datasets, including activity recognition, gesture recognition, clinical classification, pose estimation, odometry, and motion–language retrieval. It compares six self-supervised methods and introduces InrtlWrist, a masked autoencoder. The experiments cover frozen probing, fine-tuning, cross-task transfer, and joint training. The results show that a representation can perform well across several tasks even when supervision from those tasks does not consistently help each other.

Critical strengths
  1. The range of tasks is useful. Testing pose, odometry, and language retrieval alongside recognition gives a more complete picture of what wrist-IMU features can support.

  2. The paper includes controls that help interpret the results. In particular, the random-encoder and decoder-capacity experiments show how much the downstream decoder can affect the apparent benefit of pretraining. The OpenWatch duplicate audit is also a useful contribution.

Critical weaknesses
  1. Section 6 does not fully separate task relationships from dataset similarity. The three kinematic tasks all use Nymeria, while the other tasks use different datasets. Their stronger transfer could partly come from shared recording conditions. Appendix E addresses participant overlap, but the tasks still share the same dataset. This limits how broadly the transfer results can be interpreted.

  2. The pose and odometry results are difficult to assess from average errors alone. Table 3 reports improvements, but there are no examples showing predicted motion or trajectories against ground truth. For full-body prediction from one wrist, it would be useful to see which body movements are recovered and which are missed. Without these examples, the practical meaning of the improvements remains unclear.

Questions for authors
  1. What can the existing experiments establish about task relationships after accounting for the fact that all three kinematic tasks use Nymeria?

  2. Can you show successful and failed pose and trajectory predictions from the existing test set? Cases with similar wrist motion but different lower-body movement would help explain the limits of the model.

Reviewer_BmHDOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces OmniMU-Bench, a benchmark comprising a ten wrist-IMU tasks across eight datasets, coveraging eactivity recognition, gesture recognition, clinical classification, pose estimation, inertial odometry, and IMU-language retrieval. The authors also propose InrtWrist, a masked-autoencoder model incorporating an attention architecture inspired by VGGT. The experimental results show that InrtlWrist outperforms other self-supervised methods on most downstream tasks, and fine-tuning generally improves performance compared to training from scratch. Cross-task transfer experiments reveal positive transfer between some kinematic tasks but also negative transfer across tasks, suggesting that task-specific supervision does not consistently benefit other tasks.

Critical strengths
  1. Comprehensive benchmark. The benchmark covers a diverse set of tasks beyond conventional activity recognition, including kinematic estimation and semantic retrieval. This provides a useful setting for evaluating the generality of pretrained IMU representations.
  2. Extensive experimental evaluation. The paper includes comparisons across multiple pretraining methods, frozen probing, fine-tuning, and training from scratch. The additional architecture and decoder-capacity analyses help contextualize the reported performance.
  3. Interesting cross-task transfer analysis. The transfer experiments reveal that supervision beneficial to one task may negatively affect others.
Critical weaknesses
  1. Insufficient motivation and unclear significance of cross-task supervision transfer. The motivation for investigating cross-task supervision transfer is not sufficiently established. While the paper highlights that existing evaluations do not examine how supervision from one task affects others, it remains unclear why understanding these relationships is particularly important for wrist-worn IMU representations.
  2. The evidence for transferable task relationships is limited to the evaluated settings. The cross-task experiments in Figure 3(a) show positive transfer among related kinematic tasks. However, these tasks share the Nymeria dataset. Although the participant-disjoint control in Appendix E, Table 13 is valuable, the transfer gains decrease substantially. Consequently, it remains unclear whether the observed relationships reflect broader task compatibility or are specific to the datasets and supervision settings considered.
Questions for authors

Paper writing and clarity

  1. What are the most important new scientific insights provided by OmniIMU-Bench that could not be obtained from existing wearable representation benchmarks?
  2. Could the authors provide a clearer overview of how the eight datasets and 10 tasks are used across the different experiments? It would help readers better understand the experimental setup.

Experimental design and evidence 3. OmniIMU-Bench covers 10 tasks across 8 datasets, but the cross-task transfer experiments in Figure 3 only consider 7 tasks from 5 datasets. What motivated the selection of these particular tasks and datasets? 4. Figure 3(a) shows positive transfer among related kinematic tasks, but these tasks share the Nymeria dataset. Furthermore, the participant-disjoint evaluation in Appendix E, Table 13 shows substantially reduced transfer gains. Could the authors clarify how these findings should be interpreted? In particular, to what extent do the results support generalizable relationships between tasks, rather than transfer effects specific to the shared dataset and supervision settings? 5. In the joint-training experiments, Appendix F indicates that each target task can select its own best-performing checkpoint. Could the authors additionally report the performance of both tasks evaluated at the same checkpoint? This would help clarify whether the reported benefits can be achieved simultaneously using a single shared representation.

Reviewer_F3nEOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces OmniIMU-Bench, a benchmark of ten single-wrist IMU tasks got from eight datasets: four activity-recognition datasets, microgestures (OpenWatch), Parkinson’s disease classification (PADS), three kinematic estimation tasks on Nymeria (arm pose, full-body pose and wrist-frame odometry), and IMU–text retrieval on simulated HumanML3D. The authors pretrain six self-supervised objectives on a shared rouhgly about 280-hour corpus of recorded and simulated wrist data, all with a 21M-parameter ViT backbone. One of them is their proposed MAE, Inrtlwrist, which uses VGGT-style alternating frame-wise and global attention; the other five use vanilla plain attention. The paper has 3 claims: the benchmark itself, Inrtlwrist as a reusable pretrained wrist-IMU model, and an analysis of cross-task transfer and complementarity. These claims are supported by controlled comparisons of frozen probing, fine-tuning and training from scratch, together with analyses of supervised cross-task transfer, two joint-training pairs, decoder capacity, and pretraining data volume and motion coverage. The authors report that:

  • With a frozen encoder, Inrtlwrist comes out on top (or tied) on 9 out of 10 tasks.
  • Fine-tuning it beats training from scratch on 9 out of 10 tasks.
  • The kinematic tasks help each other out, whereas for retrieval, none of the frozen supervised models even come close to the pretrained one.
  • Training two tasks together boosts the partner task, but hurts full-body pose and retrieval.
Critical strengths
  • I feel like the direction is worthwhile. Most wearable SSL papers only evaluate classification. Putting pose, odometry, clinical and indexing under one 6-axis imu is interesting.

  • The paper setup/ablation is controlled. They kept everything identical for all the encoders, plus, they add a vanilla attention variant, an untrained baseline, and a raw-input control just to be thorough in the paper.

  • The data audit in Appendix C is the part I liked most. I appreciate that the authors say openly. Basically, the author said some datasets have training/testing dataset leakage issue during the split:

    • duplicated and mirrored participants in OpenWatch, and the resulting leak in the original split;
    • cross-split overlap in HumanML3D;
    • a unit problem in EatingSpeed.
  • The authors report results that don't flatter them. These include the decoder-capacity study, the participant-disjoint rerun in App. E (which reverses the comparison with Inrtlwrist) and the flat scaling result in App. H.

Critical weaknesses

1. The objective comparison is mixed up with architecture.

Only Inrtlwrist gets alternating attention structure .The five baselines all use plain attention. The paper treats the plain-attention InrtlWrist as the control for this, so I compared that variant with LIMU-BERT, which is also masked reconstruction with plain attention. Under frozen probing, it doesn't come out ahead: LIMU-BERT wins on arm pose (8.14 vs. 8.36 cm), body pose (98.55 vs. 99.96 mm), odometry (0.208 vs. 0.210 m), HHAR (61.2 vs. 60.2) and PADS (50.5 vs. 48.1), and the variant wins the other five. Even leaving out PADS, which I don't think can rank encoders (point 4), it is 4 to 5, so the numbers are very close. So my understanding is that the 9/10 result comes mostly from the architecture, which the baselines never got, rather than from the pretraining objective. I recognize these are single runs, so I may be wrong, but that’s how it looks to me. I’d like to see one of two things: either retrain LIMU-BERT and SimMTM with alternating attention, or use the plain-attention Inrtlwrist as the entry in the matched comparison.

That said, The architectural benefit itself also looks inconsistent. Comparing the two InrtlWrist versions directly, the frozen differences are mostly within 2 points, except on PADS (+7.1), which again can't really rank encoders (See in point 4). After fine-tuning, they swing both ways by large amounts, the alternating version is ahead by 7.7 on HHAR and 7.5 on TransfHAR, but behind by 16.1 on gesture and 9.1 on PADS.

2. Several key details are missing, which matters for a benchmark paper.

Related to point 1, I couldn't work out what actually differs between InrtlWrist-plain and LIMU-BERT. Sec. 4.2 says InrtlWrist's encoder only sees the visible patches, but the paper doesn't say how the LIMU-BERT reimplementation masks its input, which loss it uses, or whether the random-length cropping/trimming applies to all objectives. And several training details are missing:

  • Pretraining: the mask ratio, masking scheme, learning rate, batch size, number of pretraining steps and per-source sampling are not shown.
  • Downstream: the learning rate, number of steps and selection rule are not given either.

Sec. 4.3 says "under the following optimization settings" but then only gives Eq. 1. The Reproducibility Statement says Appendix A has the optimization settings, but it doesn't. For a benchmark that wants outside submissions, these details are essential. I feel like there's a inconsistency here, and I hope author could add these details.

3. The kinematic transfer result may be about knowing the data, not the task.

The paper’s main transfer claim, that supervision transfers within kinematic estimation, rests on the three Nymeria tasks. These share one dataset and one device, and on the main split they also share participants. All 82 evaluation participants also appear in training. So when an encoder trained on arm pose helps with odometry, I can’t tell whether it’s the arm-pose labels that help, or just that the encoder has already gotten used to this device, this recording setup and these group of people.

To their credit, the authors check part of this in App. E by splitting the participants into disjoint thirds. But the result actually weakens the story. The gains over the never-trained encoder roughly halve, and Inrtlwrist, which a kinematic source matches or beats on the shared split, now leads on body pose and odometry. In other words, about half of the original effect seems to have come from participant overlap, although App. E also cuts each source's training data to a third, so some of the drop may simply be less data. The paper does mention this in the introduction and Sec. 6 , but Fig. 3 and statements such as "each kinematic task's best frozen source is another kinematic task" still present the shared-split result as the main finding. I'd suggest making the participant-disjoint split the primary one for the kinematic block.

The half that survives isn’t clean either. The sources still come from the same dataset and device, and they are compared against an encoder that has never seen any data, so familiarity with Nymeria alone could explain the remaining gain. Two experiments would help separate this out. The first is an MAE pretrained only on Nymeria IMU, without labels: if it gets the same gain, the gain comes from the domain, not the task. The second is kinematic source–target pairs drawn from different datasets. One smaller point: in the transfer study, activity recognition is represented only by PRISM, even though the benchmark has four HAR datasets, and the gesture, clinical and retrieval families each have a single dataset. Conclusions such as the 16–18% PRISM gain from pose and odometry sources therefore rest on one dataset each, and PRISM in particular has only six test subjects and a dominant class (Table 11).

4. Some comparisons lean on weak references, especially on PADS.

On PADS, training from scratch reaches only 29.9 macro-F1, barely above the 25.4 of a head that always predicts the majority class, yet it still counts toward the claim that "fine-tuning improves on training from scratch on nine of ten tasks". For its Parkinson's conclusion the paper instead relies on the comparison with the dataset authors' BOSS pipeline, but that pipeline is itself beaten by a never-trained encoder (48.1 vs. 51.7), so under this protocol PADS doesn't seem able to rank encoders at all, and I’d either mark it as unranked or use the binary tasks from the original PADS paper.

Questions for authors
  • If LIMU-BERT and SimMTM got the same alternating attention, would InrtlWrist still come out on top? Or, if the plain-attention InrtlWrist is taken as the matched entry, how do the rankings change?
  • For the transfer study, could you please add an MAE pretrained only on Nymeria IMU dataset, without labels? That would probably show how much of the kinematic transfer comes from simply having seen Nymeria data, rather than from the task labels.
  • Why is the PADS scratch baseline so low (29.9 F1 / 45.5 BA), close to always predicting the majority class? Just curious, was it tuned separately? Would the binary tasks from the original PADS paper give a task that can rank encoders?
Reviewer_1iKYOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces OmniIMU-Bench, which integrates ten tasks across eight datasets to systematically evaluate the performance of wrist-worn IMU representations in areas such as classification, kinematic estimation, and IMU-language retrieval. This is a valuable benchmarking effort that covers scenarios often overlooked in previous benchmarks. The experiments are comprehensive, the descriptions are detailed, and the writing is clear.

As a benchmark study, this paper still needs to further refine constructive conclusions and systematic summaries of experience, especially clarifying the reasons why different self-supervised pre-training strategies have different effects on different downstream tasks, and the guiding significance of these findings for subsequent method selection and design.

Furthermore, the necessity of introducing the author's own model and its independent contribution have not been fully explained. The design of InrtlWrist does not appear to offer significant novelty, and its performance advantages across various tasks require further explanation. It is also necessary to verify whether the other baseline models were adequately trained and reasonably optimized. The description of the InrtlWrist methodology spans only about 50 lines and lacks a formal model architecture diagram; the explanation of the masked reconstruction training process and key design choices is too brief to clearly convey the model's technical details or the underlying reasons for its performance advantages.

Critical strengths

This benchmark offers broad coverage, expanding the scope for evaluating representations derived from wrist-worn IMUs. By integrating activity and gesture recognition, kinematic estimation, clinical classification, and IMU-to-language retrieval into a unified evaluation framework, it addresses task scenarios often overlooked by previous benchmarks, providing a valuable platform for the systematic study of cross-task representation applicability.

Critical weaknesses
  1. The design and performance advantages of the proposed model lack sufficient justification. InrtlWrist is primarily based on masked reconstruction, offering limited novelty in its design. The paper’s description of the method spans only about 50 lines; it lacks a detailed model architecture diagram, and the explanations regarding the training process and key design elements are overly brief. Nevertheless, the model achieves state-of-the-art results across most tasks, yet existing analysis fails to adequately explain the underlying reasons for this superior performance.

  2. There is a lack of systematic analysis regarding different pre-training strategies. Although the paper compares six self-supervised pre-training methods and examines factors such as decoder capacity and data coverage, it falls short of providing an in-depth investigation into why these strategies yield varying performance across different downstream tasks. The paper does not thoroughly explore the relationship between the information learned by these strategies and the specific requirements of downstream tasks; consequently, it offers limited guidance for the selection and design of future methods.

Questions for authors
  1. Could the authors provide further explanation regarding the performance advantages of InrtlWrist? Could they also elaborate on the key design elements of the model, particularly in comparison to the standard MAE (Masked Autoencoder) architecture?

  2. Could the authors discuss which representational features are most critical for downstream tasks? Based on cross-task experiments involving various self-supervised learning strategies, what essential information do the authors believe a general-purpose IMU representation should retain? How might these findings offer systematic guidance for selecting pre-training objectives and designing representations?

  3. To what extent do the evaluation conclusions depend on the design of the probe? Could the author further verify (i.e., use a different probe) whether the relative ranking of different self-supervised methods remains stable under different reasonable decoder capacities?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.