- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes VarMAE, a variable-input masked autoencoder for learning transferable representations from wrist-worn accelerometer signals across different sampling rates and temporal window lengths. The method addresses the limitation of existing accelerometer foundation models that are typically pretrained for fixed input configurations.
VarMAE constructs two views of each input signal: a long-duration, high-sampling-rate view and a shorter, downsampled view. It introduces variable-rate tokenization and reconstruction heads, allowing a shared Transformer encoder to process inputs with different sampling rates. The pretraining objective combines masked reconstruction within each view with cross-resolution masked reconstruction, in which visible information from the long view is used to reconstruct masked tokens in the short view.
The model is pretrained on large-scale NHANES accelerometry data and evaluated on eight human activity recognition benchmarks with different input durations and sensing conditions. The experiments include comparisons with existing pretrained foundation models, controlled specialist models, low-label evaluations, and ablations of the reconstruction objectives and masking ratios. The results demonstrate competitive transfer performance using a single pretrained encoder, with particularly encouraging robustness at reduced sampling rates.
Critical strengths
S1. The paper addresses a genuine limitation of accelerometer foundation models: the dependence on fixed sampling rates and temporal window lengths during pretraining. The proposed variable-rate tokenization and multi-resolution reconstruction framework enables a single encoder to support heterogeneous downstream sensing configurations. This has practical relevance for wearable activity recognition, where data acquisition settings vary substantially across datasets and devices.
S2. The authors evaluate eight downstream datasets spanning different window lengths and compare against publicly available pretrained models and controlled specialist baselines. The controlled Vanilla MAE comparison is particularly useful. Furthermore, the ablations across 50, 25, 10, and 5 Hz provide meaningful evidence that the short-view and cross-resolution reconstruction objectives improve robustness under reduced temporal resolution.
Critical weaknesses
W1. The authors provide useful loss ablations showing that cross-resolution reconstruction improves performance, particularly at lower sampling rates. However, Table 5 also indicates that removing the short-view reconstruction loss often causes substantially larger performance drops, suggesting that exposure to lower-rate inputs is itself an important factor. Furthermore, the closest Vanilla MAE baseline uses fixed 50 Hz inputs and linear patch embedding and reconstruction, whereas VarMAE introduces variable-rate tokenization and reconstruction heads. Although the models share the same basic encoder-decoder design, these differences make it difficult to attribute the overall improvement to the cross-resolution objective. A stronger comparison with alternative multi-rate MAE pretraining strategies under matched architectures and computational budgets would help clarify the specific advantage of the proposed mechanism.
W2. The evaluation demonstrates useful robustness to changes in sampling rate and window duration, but the low-rate experiments primarily rely on resampling existing recordings rather than evaluating independently acquired native-rate datasets under matched conditions. In addition, VarMAE-Small does not consistently outperform Vanilla MAE across all eight downstream datasets, with slightly lower results on HARTH and Capture-24. Table 1 reports five-fold mean macro-F1 scores without paired uncertainty estimates, making small differences difficult to interpret.
Questions for authors
Q1. Section 5.3 provides useful loss ablations, but the closest Vanilla MAE baseline also differs in its tokenizer, reconstruction head, and input-rate exposure. Would it be possible to provide a more tightly controlled comparison using the same architecture, pretraining data, and computational budget, while varying multi-rate exposure and the cross-resolution objective separately? This would help determine how much of the improvement comes specifically from the proposed cross-resolution mechanism.
Q2. Table 1 reports mean macro-F1 across five subject-disjoint folds, but some differences between VarMAE and Vanilla MAE are relatively small. Paired fold-level results or uncertainty estimates would help assess their reliability. If available, results from independently repeated pretraining runs would also clarify the sensitivity of the conclusions to initialization and pretraining randomness.
Thank you for the detailed responses and additional experiments. I have two follow-up questions.
While looking at the additional Vanilla MAE comparisons provided in response to my first weakness, I noticed another potential difference in the pretraining setup. VarMAE uses a 30s long view in addition to the shorter views, while the Vanilla MAE specialists are pretrained using only 1, 2, 5, or 10s windows. This means VarMAE is exposed to considerably more temporal context during pretraining, even with the same batch size and training steps.
I noticed that the new ablation presented in your response to reviewer jEeW could address this concern. Does the Vanilla MAE in row 5 also use the same 30s long view and randomly cropped short views as VarMAE? If not, could some of VarMAE's improvement over Vanilla MAE come from the additional temporal context rather than the proposed method?
I also had a clarification question regarding the new higher sampling rate experiments in response to my second weakness. You mention that the patch size is fixed to 10 samples for both Vanilla MAE and VarMAE. However, VarMAE uses fixed-duration patches of 0.2s, which would require 16, 20, and 40 samples per patch at 80, 100, and 200Hz, respectively. Does VarMAE also use 10 samples per patch in these experiments? If so, why was the patch size fixed rather than preserving the 0.2s duration?