acceptodds
Under review as a conference paper at ICLR 2027

Variable Input Masked Autoencoders for Wrist-Worn Accelerometry

Abstract

Large-scale repositories of wearable sensor data have enabled the development of foundation models for human activities and behavior. However, existing models are pretrained using fixed input durations and sampling rates, despite substantial heterogeneity in recording conditions, downstream tasks, and deployment constraints. We introduce VarMAE, a variable input masked autoencoder that learns a single representation across sampling rates and temporal contexts. During pretraining we construct two views of the same signal: a long duration high frequency view, and a short duration, lower frequency view, obtained via random cropping and downsampling. We perform masked reconstruction within each view and introduce a cross-resolution masked reconstruction objective that uses visible tokens from the long view to predict masked tokens from the short view. Cropping and downsampling impose complementary information bottlenecks, encouraging the model to learn representations that remain effective across input durations and sampling rate changes. We pretrain VarMAE on the large-scale NHANES dataset, producing a single pretrained encoder matching or outperforming controlled foundation model baselines on eight benchmark datasets. With only 0.7M parameters, VarMAE is parameter efficient and shows that a single foundation model can achieve broad transfer capabilities across sensing configurations.

Primary area:
learning on time series and dynamical systems
Keywords:
foundation models, wearables, accelerometer, activity recognition

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jEeWOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VarMAE, a variable-input masked autoencoder for learning transferable representations from wrist-worn accelerometer signals across different sampling rates and temporal window lengths. The method addresses the limitation of existing accelerometer foundation models that are typically pretrained for fixed input configurations.

VarMAE constructs two views of each input signal: a long-duration, high-sampling-rate view and a shorter, downsampled view. It introduces variable-rate tokenization and reconstruction heads, allowing a shared Transformer encoder to process inputs with different sampling rates. The pretraining objective combines masked reconstruction within each view with cross-resolution masked reconstruction, in which visible information from the long view is used to reconstruct masked tokens in the short view.

The model is pretrained on large-scale NHANES accelerometry data and evaluated on eight human activity recognition benchmarks with different input durations and sensing conditions. The experiments include comparisons with existing pretrained foundation models, controlled specialist models, low-label evaluations, and ablations of the reconstruction objectives and masking ratios. The results demonstrate competitive transfer performance using a single pretrained encoder, with particularly encouraging robustness at reduced sampling rates.

Critical strengths

S1. The paper addresses a genuine limitation of accelerometer foundation models: the dependence on fixed sampling rates and temporal window lengths during pretraining. The proposed variable-rate tokenization and multi-resolution reconstruction framework enables a single encoder to support heterogeneous downstream sensing configurations. This has practical relevance for wearable activity recognition, where data acquisition settings vary substantially across datasets and devices.

S2. The authors evaluate eight downstream datasets spanning different window lengths and compare against publicly available pretrained models and controlled specialist baselines. The controlled Vanilla MAE comparison is particularly useful. Furthermore, the ablations across 50, 25, 10, and 5 Hz provide meaningful evidence that the short-view and cross-resolution reconstruction objectives improve robustness under reduced temporal resolution.

Critical weaknesses

W1. The authors provide useful loss ablations showing that cross-resolution reconstruction improves performance, particularly at lower sampling rates. However, Table 5 also indicates that removing the short-view reconstruction loss often causes substantially larger performance drops, suggesting that exposure to lower-rate inputs is itself an important factor. Furthermore, the closest Vanilla MAE baseline uses fixed 50 Hz inputs and linear patch embedding and reconstruction, whereas VarMAE introduces variable-rate tokenization and reconstruction heads. Although the models share the same basic encoder-decoder design, these differences make it difficult to attribute the overall improvement to the cross-resolution objective. A stronger comparison with alternative multi-rate MAE pretraining strategies under matched architectures and computational budgets would help clarify the specific advantage of the proposed mechanism.

W2. The evaluation demonstrates useful robustness to changes in sampling rate and window duration, but the low-rate experiments primarily rely on resampling existing recordings rather than evaluating independently acquired native-rate datasets under matched conditions. In addition, VarMAE-Small does not consistently outperform Vanilla MAE across all eight downstream datasets, with slightly lower results on HARTH and Capture-24. Table 1 reports five-fold mean macro-F1 scores without paired uncertainty estimates, making small differences difficult to interpret.

Questions for authors

Q1. Section 5.3 provides useful loss ablations, but the closest Vanilla MAE baseline also differs in its tokenizer, reconstruction head, and input-rate exposure. Would it be possible to provide a more tightly controlled comparison using the same architecture, pretraining data, and computational budget, while varying multi-rate exposure and the cross-resolution objective separately? This would help determine how much of the improvement comes specifically from the proposed cross-resolution mechanism.

Q2. Table 1 reports mean macro-F1 across five subject-disjoint folds, but some differences between VarMAE and Vanilla MAE are relatively small. Paired fold-level results or uncertainty estimates would help assess their reliability. If available, results from independently repeated pretraining runs would also clarify the sensitivity of the conclusions to initialization and pretraining randomness.

Reviewer_HejFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VarMAE, a masked autoencoder for accelerometer signals with different window durations and sampling rates. Patches cover a fixed duration in seconds, while a cross-attention tokenizer and a rate-conditioned reconstruction head accommodate different numbers of samples per patch. Pretraining combines reconstruction of a long 30-second view, a shorter downsampled view, and cross-resolution reconstruction of the short view using visible long-view context. The model is pretrained on approximately 16,000 NHANES participants and evaluated with frozen-encoder linear probes on eight activity-recognition datasets, including few-shot and sampling-rate ablations.

Critical strengths
  1. The paper addresses a practical limitation of accelerometer models: downstream tasks use different temporal contexts and sampling rates. Patches and positions defined in physical time allow a compact shared encoder to handle these configurations. Evaluation on eight datasets, with subject-disjoint folds and specialist baselines pretrained on the same source data, demonstrates useful transfer across window lengths.
  2. The loss ablations provide direct evidence for cross-resolution training at low sampling rates. Table 5 shows that removing this loss reduces 5 Hz performance on all four evaluated datasets, including this rate that was not used during pretraining. The few-shot experiments additionally test whether the frozen representations remain useful when labeled data are limited.
Critical weaknesses
  1. The practical advantage over simpler alternatives is not sufficiently established. The controlled specialists are pretrained at 50 Hz, while the low-rate experiments compare only VarMAE variants. A fixed-rate MAE receiving the same low-rate observations after upsampling is missing, so the benefit over a resampling pipeline remains unclear. At 50 Hz, several gains over the controlled MAE are small or negative, and Table 1 supplies no uncertainty for its fold means. The stronger medium model also has 3.82M parameters versus 0.53M for the MAE. These comparisons leave the advantage of the proposed method over a simpler model uncertain, despite the positive loss-ablation results.
  2. Generalization to unseen input configurations is only partially evaluated. The experiments include two sampling rates not used in pretraining, 25 Hz and 5 Hz, but all downstream window durations (1, 2, 5, and 10 s) coincide with the short-view durations used during pretraining. The results therefore support transfer across datasets and some unseen sampling rates, while leaving generalization to unseen window durations untested. The performance decrease at 5 Hz is difficult to interpret without a baseline evaluated on the same low-rate observations.
Questions for authors
  1. At 5, 10, and 25 Hz, how does VarMAE compare with a fixed-rate MAE receiving the same observations upsampled to 50 Hz, using the same folds and linear-probe procedure? Please report paired fold-level differences and model size so that the practical benefit can be assessed.
  2. Does the frozen encoder retain its advantage over a fixed-rate MAE at window durations not used during pretraining, such as 3 s? Please clarify whether the variable-input claim concerns the configurations used during pretraining or also generalization to unseen durations and rates.
Reviewer_MTP8OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents VarMAE, a new Masked Autoencoder setup that allows variable input length and sampling rate. This is done by using patches with a fixed time interval rather than a fixed sample count, and using a learned query with cross-attention. It also introduces cross-resolution masked reconstruction, where the masked tokens of a cropped, potentially lower sampling rate segment are reconstructed conditioned on the visible tokens of the full view. Across 8 different downstream datasets of 4 different window lengths, VarMAE performs strongly, often outperforming public checkpoints, fully supervised models, as well as reimplementations of existing SSL approaches.

Critical strengths

The paper shows potential for building a single encoder that can understand inputs of varying length and sampling rate. The reported results show promise. A single VarMAE trained on {1, 2, 5, 10}s windows collectively performs as well as, if not better than, existing SSL approaches trained on each window length explicitly.

Critical weaknesses
  • The evaluation setup doesn’t clearly establish the improvement of VarMAE over Vanilla MAE. Vanilla MAE with 0.53M params performs only slightly worse, or sometimes even better than VarMAE (small) with 0.70M params. Why not include a larger Vanilla MAE that matches AugPred and SimCLR in size, and is closer to VarMAE (medium)?
  • Whether the proposed approach performs competitively against existing approaches at varying sampling rates isn’t clearly shown. Table 5 in the appendix seems to show similar performance at 25Hz and 50Hz, even though 25Hz is unseen during pretraining, but there is no additional text discussing the results. Also, there is no comparison against Vanilla MAE at these lower sampling rates. For example, on a 20Hz downstream dataset, does VarMAE perform better than a Vanilla MAE that was pretrained on 20Hz? It would also be good to show if VarMAE, trained on only 50Hz and lower sampling rate variants, can perform well even with a higher sampling rate (e.g., 80Hz) compared to a Vanilla MAE trained only on 80Hz. I do not necessarily expect VarMAE to outperform Vanilla MAE in all these cases, but showing competitive performance across different sampling rates would strongly support the claim that a single model can handle varying sampling rates.
Questions for authors
  • This is related to my second weakness, but why not do a Table 1-like experiment with varying sampling rates, comparing a single VarMAE against Controlled Specialist FMs trained at each target sampling rate? This would better support the claim that a single VarMAE can handle different window lengths and sampling rates.
  • Is the 0.17M encoder parameter size difference between “Vanilla MAE” and “Ours (small)” from the difference in variable-rate tokenization?
  • For the Variable-Rate Reconstruction, how are the sampling rate and temporal offset incorporated into the MLP? Section 3.1 is a little difficult to follow because some details are explained very briefly. I think it could benefit from more detailed explanations and figures illustrating how they work.
  • Is it right to understand LcrossL_{\text{cross}} as an easier version of LshortL_{\text{short}}, since it reconstructs the masked patches using additional context from the longer, higher sampling rate view? If so, what is the advantage of adding this objective rather than just using LshortL_{\text{short}}?
  • Is LlongL_{\text{long}} needed at all? Looking at Figure 3 and Table 5, it seems like removing it barely has any negative impact on downstream performance.
  • Why are REBAR and RelCon not included in the comparisons? I see them mentioned in Table 4.

Comments on the paper

2 comments

Thank you for the detailed responses and additional experiments. I have two follow-up questions.

While looking at the additional Vanilla MAE comparisons provided in response to my first weakness, I noticed another potential difference in the pretraining setup. VarMAE uses a 30s long view in addition to the shorter views, while the Vanilla MAE specialists are pretrained using only 1, 2, 5, or 10s windows. This means VarMAE is exposed to considerably more temporal context during pretraining, even with the same batch size and training steps.

I noticed that the new ablation presented in your response to reviewer jEeW could address this concern. Does the Vanilla MAE in row 5 also use the same 30s long view and randomly cropped short views as VarMAE? If not, could some of VarMAE's improvement over Vanilla MAE come from the additional temporal context rather than the proposed method?

I also had a clarification question regarding the new higher sampling rate experiments in response to my second weakness. You mention that the patch size is fixed to 10 samples for both Vanilla MAE and VarMAE. However, VarMAE uses fixed-duration patches of 0.2s, which would require 16, 20, and 40 samples per patch at 80, 100, and 200Hz, respectively. Does VarMAE also use 10 samples per patch in these experiments? If so, why was the patch size fixed rather than preserving the 0.2s duration?

Thank you very much for the detailed response. Please find my reply below.

Regarding W1 and Q1, if I understand the authors’ reasoning correctly, the larger the data transformation, the more the performance of the upsampled Vanilla MAE should drop, and VarMAE should drop less than that. However, in the upsampling comparison table in your response to W1 (Fig. 5 in the revision), the two appear to drop together. At 5 Hz, where the transformation is largest, VarMAE actually drops more. This suggests to me that the advantage of VarMAE may come from a factor other than processing the low-rate input natively. To present these numbers as evidence, I believe a statistical test on the paired fold-level differences requested in Q1 would be necessary. I would appreciate it if you could show this, even briefly, with the fold-level results you already have. In addition, could you kindly report the Linear row of Table 2 at 5, 10, and 25 Hz, with the inputs upsampled to 50 Hz?

The remaining concerns have been well addressed, and I appreciate the effort. However, since the main concern above remains unresolved, I will keep my current score for now.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.