acceptodds
Under review as a conference paper at ICLR 2027

IMUDINO: Self-distilled Foundation Model for Inertial Sensing

Abstract

We present IMUDINO, the first foundation model (FM) for inertial sensing to achieve competitive performance across diverse sensing tasks including human activity recognition (HAR), health, motion capture, and robot odometry. Prior inertial FMs often focus on one subfield of inertial sensing and sometimes underperform state-of-the-art supervised methods (e.g., in HAR), making their scope considerably narrower than FMs for vision and natural language. Moreover, they often require fixed input length and sampling rate, and generalize poorly to unseen sensor locations. To build a general-purpose foundation model for inertial sensing, we adapt the DINO self-supervised learning (SSL) paradigm to inertial data through a set of semantics-preserving motion transformations that simulate variations in motion pattern and sensor setup. For model architecture, we use a plain transformer encoder with frequency-aware positional encoding that natively supports varying input length and sampling rate. This simple setup proved remarkably effective for learning global and dense features generalizable across users, sensor setups, tasks, and, in the case of robot odometry, embodiments. We hope IMUDINO can serve as a backbone for inertial sensing and kickstart the development of general-purpose inertial FMs. Code and checkpoints will be released.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Inertial Sensing, Foundation Model, Self Distillation, Self-supervised Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_S1EEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a foundation model that uses the DINO self supervised learning model and adapts it to inertial sensor data. The model used is a transformer encoder with frequency aware positional encoding. The training objective of the proposed IMUDINO model combines a global DINO loss, a patch level (iBOT++) loss and a KoLeo regularisation loss. IMUDINO's performance is evaluated on the HAR (Human Activity Recognition) task, the motion capture task and the inertial odometry task.

Critical strengths
  • a step towards general purpose inertial foundation models via an adaptation of dino self distillation to inertial sensing
  • competitive performance on a broad range of tasks (global and dense prediction tasks)
  • able to learn with a simple model and orders of magnitude less training data
Critical weaknesses
  • the adaptation of dino to inertial sensing seems straight forward - e.g. applying ideas of patch level objective and KoLeo regularization loss previously applied in visual tasks

  • In some tasks, e.g. in the Inertial MoCap results, it does not beat the sota, DynaIP (though the authors explain that IMUDINO is not aided by physical priors or does not have access to raw gyro readings).

Questions for authors
  • One of your claims is that IMUDINO could learn with orders of magnitude less data than competing techniques. It would be interesting to show how the SOTA and IMUDINO's ability to learn depends on the size of the data.

  • The impact of some elements of the training, e.g. the KoLeo regularizer is not always positive, eg. in MoCap performance you observed a drop, while in HAR there was a small improvement. Similarly channel masking led to improved HAR performance but reduced MoCap performance. Frequency-aware position encoding was also found to improve HAR and odometry but hinder slightly MoCAP. Could you explain the reasons / provide some intuition of why the MoCap task is not as much improved by these elements of the training recipe while the other tasks are? Does it have to do with the task itself, or other aspects of training or the datasets used for training?

2 comments
AuthorsOpenReview ↗

Response to Reviewer S1EE (1/2)

Thanks for the insightful and constructive review. We hope the results and discussion below address your questions.

Q1 Effect of Data Scaling: To further explore the effect of data scaling on IMUDINO and RelCon, we repeated the experiment in Section 4.2 using subset with different sizes of our training set using the IMUFormer-6M model. The pre-training compute budget was fixed (i.e., smaller subsets got more training epochs to match the 60-epoch full dataset protocol in terms of compute).

The results show that IMUDINO is more data efficient, as it nearly reached full potential at just 800 hours of training data, while RelCon saw more improvement when we scale from 800 hrs to 11600 hrs. However, it has to be noted that IMUDINO-6M nearly saturates some of the downstream benchmarks (e.g. ~98 mF1 on MotionSense, ~95 mF1 on HHAR, >99 mF1 on Shoaib et al.'s dataset), which may dampen the observed overall gain in HAR performance.

Pre-training Data (Hours)MethodHAR Avg. mF1
800RelCon67.13
800IMUDINO88.15
3200RelCon67.19
3200IMUDINO88.30
11600 (Full dataset)RelCon69.64
11600 (Full dataset)IMUDINO88.35

[1] Xu, Maxwell, et al. "Relcon: Relative contrastive learning for a motion foundation model for wearable data." International Conference on Learning Representations. Vol. 2025. 2025.

[2] Yuan, Hang, et al. "Self-supervised learning for human activity recognition using 700,000 person-days of wearable data." NPJ digital medicine 7.1 (2024): 91.

AuthorsOpenReview ↗

Response to Reviewer S1EE (2/2)

Q2 Effect of Certain IMUDINO Elements on MoCap: Here are some intuition regarding the observed ineffectiveness of certain IMUDINO elements on the MoCap task.

  • KoLeo Loss: the KoLeo regularizer is applied to the CLS token only and its intended role is to encourage the global representations (CLS tokens) to spread out. As a result, it is not intended to improve the dense features used by the MoCap task. We also hypothesize that, as a regularizer, it may mildly interfere with the optimization of the patch-level objective.
  • Channel Masking: Channel masking creates training samples where some channels are zeroed out, and may induce to model to adapt to such samples. Such adaptations, while useful for mitigating overreliance on certain channels for global tasks like HAR, do have the potential to harm MoCap performance. Our hypothesis is that MoCap is a very fine-grained task requiring information from all 6 channels, and the learned adaptation to missing channels may take model capacity away from learning fine-grained 6-channel features.
  • Frequency-aware PE: We hypothesize that the ineffectiveness of frequency-aware positional encoding on MoCap could be attributed to MoCap datasets having a sampling rate (60 Hz) closest to the native sampling rate of the pretraining dataset (100 Hz). In contrast, most HAR datasets are recorded at 25 to 50 Hz, and the TartanIMU Challenge use 200 Hz inputs. While resampling augmentation may mitigate the difference, it is only applied at a probability of p=0.5p=0.5, and the underlying real data is still 100 Hz. As a result, adaptation to varying sampling rate could be less useful in MoCap than in HAR or odometry, since MoCap input is closest to the training corpus in terms of native sampling rate.

We leave more concrete exploration of how elements of inertial SSL recipes interact to future work, which could lead to more optimal combinations of training objectives.

Reviewer_u6eAOpenReview ↗
Rating
1(−1)
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This work adapts DINO-based self-distillation to inertial sensing, incorporating global semantics-preserving view transformations. The authors evaluate the proposed method using 11,600 hours of wearable inertial data spanning diverse scenarios and sensor configurations. Its effectiveness is further demonstrated across multiple downstream applications, including human activity recognition (HAR), health monitoring, motion capture, and robotic odometry.

Critical strengths
  1. This work explores the application of DINO to inertial sensing, which is interesting.
  2. This work demonstrates the applicability of the proposed method across different inertial sensing setups.
Critical weaknesses

First, the novelty of the proposed approach is insufficiently established, particularly given existing studies on foundation models for inertial sensing. Second, the methodological design lacks a clear connection to the fundamental challenges of inertial data, such as sensor noise, distribution shifts, and unseen activities. Third, the considerable computational complexity of the proposed Transformer-based architecture is not adequately justified by its performance gains over lightweight alternatives. More comprehensive experimental evaluations are needed to demonstrate the model's generalization, robustness, computational efficiency, and practical advantages.

Questions for authors

I have some concerns on the designs and case studies:

  1. In the proposed training objective, global and local views are treated differently in the teacher and student models. What is the motivation behind this design choice? The current framework does not appear to fully explore the interactions between global and local views, which may limit its ability to learn comprehensive motion representations. Furthermore, the authors mention using a batch size of 1,536 samples, resulting in 3,072 global views and 9,216 local views per batch, while limiting global views to 1,024 frames and local views to 512 frames for training efficiency. However, the rationale behind these design choices and their impact on representation learning remain unclear. Beyond computational efficiency, the model design should more carefully consider the fundamental characteristics of inertial sensing, particularly the heterogeneity of sensor modalities and sensor placement locations. These factors are critical for learning robust and generalizable representations across different sensing configurations.

  2. The datasets used for HAR evaluation, such as MotionSense, HAPT, and HHAR, are relatively small and may not be sufficient to comprehensively demonstrate the generalization capabilities of the proposed foundation model. Since the authors appear to train and evaluate the model separately on these datasets, it would be valuable to investigate joint training across multiple datasets to assess cross-dataset generalization and transferability. Furthermore, the experimental comparisons do not adequately reflect the SOTA methods and their reported performance on these benchmark datasets. More comprehensive comparisons with established SOTA approaches are necessary to convincingly demonstrate the effectiveness and competitiveness of the proposed method.

  3. For the HPE task, several important performance metrics are missing, making it difficult to comprehensively assess the effectiveness of the proposed approach. The authors should report standard evaluation metrics commonly used in HPE, such as MPJPE, PA-MPJPE, and relevant orientation or temporal consistency errors.

7 comments
AuthorsOpenReview ↗

Response to Reviewer u6eA (1/4)

We would like to thank the reviewer for the detailed and constructive feedback. We hope the response below addresses your concerns.

Weakness 1 Novelty: The main novel contribution of our work is demonstrating for the first time that a shared pretrained backbone supports both global motion semantics and dense prediction. Our design is grounded in the core tension in learning general-purpose representation for inertial data, the tension between the need for global semantic invariance and local sensitivity to changes in motion patterns. DINO aligns CLS tokens across differently transformed views, whereas iBOT++ matches corresponding patch tokens within the same transformed global view without requiring patch representations to be invariant across different rotations. This is supported by an analysis of the properties of global and local representations when faced with perturbations (Section 4.6). In the context of HAR-focused inertial FM literature, IMUDINO broadens the scope of inertial FMs considerably and opens up the field for general-purpose inertial FMs.

Q1-1 Representation Learning Framework Design: Our setup, where the EMA teacher receives global crops, while the student receives the same global crops and additional local crops, is inherited from the DINO models [1, 2, 3]. We kept the global-local interactions because it encourages the model to learn local-to-global correspondences where a local crop of a longer motion sequence likely still contains the same semantic concept. This is grounded in the temporal continuity of inertial data, and pushes the student to learn to extract high quality global features from short local crops.

Q1-2 Batch Size and Crop Length: Results from DINO [1] suggest that increasing the number of additional local crops beyond 6 brings relatively small returns while the benefit of larger batch size saturates at around 1024. DINOv2 found that, a batch larger than 1024 brought further (small) benefits when the model and data were scaled up and when a patch-level objective was included. These findings from previous work informed the selection of these hyperparameters in addition to computational efficiency. Finally, truncating global views to 1024 frames and local ones to 512 frames affected only ~5% of local views and ~10% of global views. The length to avoid any truncation is 1032 for local views and 2048 for global views, which would roughly double memory usage and result in around half of the tokens being padding.

Q1-3 Further Adaptation to the Characteristics of Inertial Sensing: In addition to balancing the need for global semantic invariance and local sensitivity to changes in motion patterns, the characteristics of inertial sensing are also baked into the SSL recipe through (1) our choice of training data and (2) the proposed semantics-preserving transformations. Specifically, our training data covers diverse sensor positions including hand, wrist, hip, thigh, ankle, torso, etc., and comes from different sensor hardware. Some semantic-preserving transformations (e.g. resampling), further improves robustness to heterogenous hardware. As for model architecture, we kept it simple with the purpose of isolating the effectiveness of our proposed self-supervised learning recipe, and made no IMU-specific adaptations beyond 1D patching and frequency-aware PE.

References

[1] Caron, Mathilde, et al. "Emerging properties in self-supervised vision transformers." 2021 IEEE/CVF international conference on computer vision (ICCV). IEEE, 2021.

[2] Oquab, Maxime, et al. "Dinov2: Learning robust visual features without supervision." arXiv preprint arXiv:2304.07193 (2023).

[3] Siméoni, Oriane, et al. "Dinov3." arXiv preprint arXiv:2508.10104 (2025).

AuthorsOpenReview ↗

Response to Reviewer u6eA (2/4)

Q2-1 Joint Training: We would first like to clarify that IMUDINO is pretrained on a fixed 11600 sensor-hour dataset (no overlap with downstream datasets), and frozen during downstream evaluation. We trained MLP probes on top of the frozen IMUDINO features for each downstream dataset separately, following standard evaluation procedures in both DINO [1, 2, 3] and inertial FM [4, 5, 6] literature.

We further compare two setups using MotionSense, HAPT, and HHAR: training one MLP probe for each individual datasets and joint training on all 3 datasets. We consider the 5 common activities across the 3 datasets in the experiment, namely walking, ascending stairs, descending stairs, sitting, and standing. Both setups use 5-fold subject-disjoint cross validation, and retain the sliding window settings used in the main paper. Joint training results are similar to training one probe per dataset.

MotionSense mF1HAPT mF1HHAR mF1
Dataset-specific Probe98.12$\pm$1.3399.00$\pm$0.5493.70$\pm$4.68
Joint Training98.15$\pm$1.3398.99$\pm$0.4993.54$\pm$4.28

Q2-2 Comprehensive Comparisons with Established SOTA: We ran a frozen-backbone linear probe HAR experiment using Haresamudram et al.'s exact evaluation protocol [6], which was also used in RelCon [4], a SOTA inertial FM. Since these results are obtained using acceleration-only input, we masked the gyroscope channels for IMUDINO in this experiment. This allows for a fair and comprehensive comparison. The RelCon numbers are directly cited from [4], and the remaining baselines use Haresamudram et al.'s results [6]. All IMUDINO models far outperform RelCon under the same downstream evaluation protocol, including on the HHAR-Wrist dataset, which is within the training domain of both models.

HHAR (Wrist)MotionSense (Waist)PAMAP2 (Leg)
Supervised MethodsDeepConvLSTM54.39 ± 2.2884.56 ± 0.8551.22 ± 1.91
ConvNet55.43 ± 1.2189.25 ± 0.5059.76 ± 1.53
LSTM37.42 ± 5.0486.74 ± 0.2948.61 ± 1.82
Self-supervised MethodsAug Pred50.95 ± 2.7074.96 ± 1.3746.90 ± 1.14
SimCLR55.93 ± 1.7583.93 ± 1.7850.75 ± 2.97
SimSiam45.36 ± 4.9871.91 ± 12.347.85 ± 2.48
BYOL40.66 ± 4.0866.44 ± 2.7643.89 ± 3.35
MAE43.48 ± 2.8461.14 ± 3.4542.32 ± 1.63
CPC56.24 ± 0.9872.89 ± 2.0645.84 ± 1.39
Autoencoder53.57 ± 1.1455.12 ± 3.4650.79 ± 1.09
RelCon57.63 ± 3.2480.35 ± 0.7153.98 ± 0.76
IMUDINOIMUDINO-2.7M75.82± 8.9695.08 ± 1.8075.38 ± 4.62
IMUDINO-6M76.47 ± 9.0296.05 ± 1.3075.81 ± 4.20
IMUDINO-85M78.96 ± 7.8296.66 ± 2.1778.24 ± 4.27

Weakness 3 Efficiency: In the comparison above, IMUDINO-2.7M, which is more lightweight than RelCon (3.96M), still retains most of IMUDINO-85M's performance gains. We also tallied the MFLOPs used for the inference of a single, typical 2.56-s, 50 Hz window: 131.6 for TSF, 70.3 for IMUDINO-2.7M, and 217.7 for RelCon. This addresses the third weakness, showing that IMUDINO performs well at smaller scales.

References

[1] Caron, Mathilde, et al. "Emerging properties in self-supervised vision transformers." 2021 IEEE/CVF international conference on computer vision (ICCV). IEEE, 2021.

[2] Oquab, Maxime, et al. "Dinov2: Learning robust visual features without supervision." arXiv preprint arXiv:2304.07193 (2023).

[3] Siméoni, Oriane, et al. "Dinov3." arXiv preprint arXiv:2508.10104 (2025).

[4] Xu, Maxwell, et al. "Relcon: Relative contrastive learning for a motion foundation model for wearable data." International Conference on Learning Representations. Vol. 2025. 2025.

[5] Yuan, Hang, et al. "Self-supervised learning for human activity recognition using 700,000 person-days of wearable data." NPJ digital medicine 7.1 (2024): 91.

[6] Haresamudram, Harish, Irfan Essa, and Thomas Plötz. "Assessing the state of self-supervised human activity recognition using wearables." Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6.3 (2022): 1-47.

AuthorsOpenReview ↗

Response to Reviewer u6eA (3/4)

Q3 HPE Metrics: The "Pos" metric reported in our tables is root-relative MPJPE in centimeters, computed using DynaIP’s official evaluator [1] by averaging Euclidean joint-position errors across joints and frames, while orientation errors are already reported via SIP error and global angular error (Ang). In addition to these metrics, we further evaluated PA-MPJPE and jitter error across all four test sets and report the results in the following tables. The jitter error follows the definition in [2].

AnDy PA-MPJPE (cm)UNIPD PA-MPJPE (cm)CIP PA-MPJPE (cm)Natural Motion PA-MPJPE (cm)Average (cm)
IMUDINO-85M+MLP2.792.563.934.373.41
IMUDINO-85M+Transformer2.602.333.404.423.19
IMUDINO-85M+MLP (Backbone Finetuned)2.431.952.974.072.86
AnDy Jitter (km/s$^3$)UNIPD Jitter (km/s$^3$)CIP Jitter (km/s$^3$)Natural Motion Jitter (km/s$^3$)Average (km/s$^3$)
IMUDINO-85M+MLP0.3040.3230.4170.3200.341
IMUDINO-85M+Transformer0.2600.2820.3690.2470.290
IMUDINO-85M+MLP (Backbone Finetuned)0.2470.2670.3450.2160.269

[1] Zhang, Yu, et al. "Dynamic inertial poser (dynaip): Part-based motion dynamics learning for enhanced human pose estimation with sparse inertial sensors." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.

[2] Yi, Xinyu, et al. "Physical inertial poser (pip): Physics-aware real-time human motion tracking from sparse inertial sensors." 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022.

Reviewer_aFGbOpenReview ↗
Rating
2(−1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents IMUDINO, a self-supervised foundation model for inertial sensing that adapts the DINO/iBOT++ self-distillation framework to IMU signals. It combines motion-oriented augmentations, a patch-level iBOT++ objective, and KoLeo regularization, with a transformer backbone that uses frequency-aware positional encoding to handle varying sequence lengths and sampling rates. Pretrained on about 11,600 hours of wearable inertial data, IMUDINO produces transferable global and dense motion representations and performs well across human activity recognition, health and gait analysis, inertial motion capture, and robot odometry. Notably, it supports cross-domain transfer from wearable sensing to robot odometry with a frozen backbone.

Critical strengths
  1. Unlike many existing inertial representation-learning methods that focus mainly on HAR, this work evaluates learned representations on recognition, health, motion capture, and robot odometry. The results provide meaningful evidence of transferable representations, especially the frozen-backbone transfer to robot odometry.

  2. The paper successfully combines global self-distillation, dense patch-level learning, and motion-oriented augmentations using a relatively simple transformer backbone. Its frequency-aware positional encoding is a useful adaptation to the heterogeneous sampling rates encountered in inertial sensing.

  3. The authors also investigate model scaling, loss components, data augmentation, and representation sensitivity to rotation and resampling.

Critical weaknesses
  1. The robot odometry results are promising, but they are obtained on the official validation split with supervised task-specific probes trained on labeled robot data. The transformer probe contains a nontrivial number of trainable parameters, and the training protocol employs densely overlapping windows. Thus, the experiment demonstrates transferability of frozen pretrained features rather than zero-shot robot odometry. A comparison in which TartanIMU and IMUDINO use matched downstream decoders, window sampling, optimization procedures, and training budgets would better isolate the contribution of the pretrained representation.

  2. Magnitude and temporal scaling can preserve coarse activity identity, but they may alter quantities directly relevant to energy expenditure, gait characteristics, and motion estimation. The paper does not sufficiently investigate how the learned invariances affect downstream tasks whose targets depend on these quantities. A concrete example is freezing-of-gait detection: compared with the official baseline, IMUDINO improves specificity but substantially reduces sensitivity. This trade-off is especially important for a health-related detection task and deserves more careful discussion.

Questions for authors
  1. Can the authors provide a controlled robot-odometry comparison in which TartanIMU and IMUDINO use equivalent downstream prediction heads, training-window strategies, and optimization settings? Can variability across random seeds also be reported?

  2. How do the authors reconcile magnitude and temporal scaling invariance with downstream targets that depend on physical motion intensity or speed? Can task-specific augmentation ablations clarify this issue?

  3. Can the authors explain the substantial reduction in sensitivity on Daphnet freezing-of-gait detection? Would alternative classification thresholds or class-balancing strategies produce a more favorable operating point?

3 comments
AuthorsOpenReview ↗

Response to Reviewer aFGb (1/3)

Thanks for the constructive review! We hope the response below addresses your concerns and questions.

Q1 Controlled Robot Odometry Experiment: We ran an additional experiment using TartanIMU's official public code, where a decoder for frozen IMUDINO features and the full TartanIMU model were trained using the same data preprocessing/windowing settings, identical standard LSTM+linear layer decoders, the standard MSE loss, the same number of epochs, and the same LR schedule. Since TartanIMU is designed as a multi-head model with specialist heads for each platform, we retain that design for the IMUDINO-based models here. We further report IMUDINO+linear layer results obtained under the same protocols. We report the results (mean±std from 3 random seeds) from the corresponding expert for each model. Note that no standard deviation is reported for the official checkpoint, since there was no randomness in the evaluation protocol of a fixed checkpoint.

Under matched protocols, IMUDINO outperforms TartanIMU on all three metrics, demonstrating the transferability of the learned dense representation.

ModelAvg. AVE$\downarrow$Avg. ATE20$\downarrow$Score$\downarrow$
TartanIMU (Official Checkpoint)0.3501.1160.429
TartanIMU (Reproduced)0.318 ± 0.0101.181 ± 0.0110.411 ± 0.009
IMUDINO + TartanIMU-style Decoder0.254 ± 0.0021.050 ± 0.0120.342 ± 0.002
IMUDINO + Linear Head0.275 ± 0.0021.047 ± 0.0080.359 ± 0.003
AuthorsOpenReview ↗

Response to Reviewer aFGb (2/3)

Q2 Scaling vs. Speed/Intensity-sensitive Tasks: We ran the GOTOV activity intensity estimation experiment (Appendix F) using the "w/o scaling" ablation checkpoint. The experiments are done without data augmentation on the downstream dataset to isolate the impact of the learned representations. The results are nearly identical with the main IMUDINO checkpoint that was pretrained with magnitude and temporal scaling augmentations. One possible explanation could be that, in DINO training, the DINO projection head output, not the CLS tokens themselves, are aligned. As a result, while the DINO head output is encouraged to be speed/intensity invariant, the underlying CLS token could still retain speed/intensity features.

CheckpointLOSO Median Fold RMSE$\downarrow$LOSO Median Fold R$^2 \uparrow$
IMUDINO 85M0.960.61
IMUDINO 85M w/o scaling0.960.58
Official Baseline [1]1.350.45

Reference

[1] Paraschiakos, Stylianos, et al. “A Recurrent Neural Network Architecture to Model Physical Activity Energy Expenditure in Older People.” Data Mining and Knowledge Discovery, vol. 36, no. 1, 2022, pp. 477–512

AuthorsOpenReview ↗

Response to Reviewer aFGb (3/3)

Q3 DaphNet FoG: We explored alternative classification thresholds and class-balancing strategies, which did lead to a higher-sensitivity operating point. We also explored using the "w/o scaling" checkpoint, which again yielded almost identical results. Specifically, we considered using focal loss to mitigate class imbalance, and tuning the threshold on the validation set by optimizing Youden's J. The results below, except for the results we reported in the original Appendix, are done without data augmentation to isolate the impact of the learned representations.

CheckpointClassification ThresholdClass-balancing StrategySpecificitySensitivity
IMUDINO 85M w/o scaling0.5None0.9880.181
IMUDINO 85M0.5None0.9870.197
IMUDINO 85M0.5Balanced Oversampling0.8970.544
IMUDINO 85M0.5Focal Loss0.9400.348
IMUDINO 85MTuned on Validation SetBalanced Oversampling0.8110.731
Official BaselineN/AN/A0.8690.781
Results in Paper (with data augmentation)0.5Balanced Oversampling0.9360.551

These results did not beat the official baseline, although the gap is not large. A possible reason could be that our pre-training data is limited to healthy individuals. We will add a discussion on this result and the relevant limitation to the paper.

Comments on the paper

2 comments

Dear Reviewers,

Thanks for your constructive reviews! We have posted the response to your questions, and would gladly engage in further discussion. We will update the PDF later, so that any point raised during the discussion period can be included in the revised version.

Best Regards,

Authors

AuthorsOpenReview ↗

Clarification on contribution, SOTA, and efficiency

We would like to briefly explain again what our main contributions are.

Inertial Representation Learning: We are the first to demonstrate that a single self-supervised inertial FM can simultaneously provide global features for tasks like HAR and dense features for tasks like HPE. Prior inertial FM work mostly focus on HAR alone.

SOTA HAR and Odometry Performance: IMUDINO, as a frozen backbone, beats the supervised SOTA for HAR (TSF) and a SOTA robot inertial odometry FM (TartanIMU), under matched comparisons.

HPE Performance: The HPE results are used to demonstrate that IMUDINO's dense features contain fine-grained patch-level information about motion. We did not claim SOTA HPE, but IMUDINO is the first step towards doing HPE with inertial FMs, and it already performs on par with models from 2022 / 2023. It does not need to beat SOTA to show that the direction is promising and that its dense representation works.

Model Efficiency: IMUDINO-2.7M, which is more lightweight than RelCon (3.96M), still retains most of IMUDINO-85M's performance gains. MFLOPs of a single, typical 2.56-s, 50 Hz window inference: 131.6 for TSF, 70.3 for IMUDINO-2.7M, and 217.7 for RelCon. Please refer to our initial response (2/4) for details.

As a comparison, DeepSense [1] use 38.7 MFLOPs for the same inference, and has already been deployed with low latency and acceptable battery drain on Nexus 5 phones from 2013 [1]. IMUDINO 2.7M is certainly lightweight enough for deployment on a 2026 smartphone, and provide SOTA HAR capabilities.

[1] Yao, Shuochao, et al. "Deepsense: A unified deep learning framework for time-series mobile sensing data processing." Proceedings of the 26th international conference on world wide web. 2017.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.