- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents Tile, a self-supervised learning approach for pretraining a network to extract temporal and frequency features of accelerometer signals for multiple downstream tasks. Tiles decomposes raw accelerometer signals into three frequency bands and processes them using separate encoders. The extracted features are then aggregated into multiple time-frequency latents through iterative competitive cross-attention with learnable queries. Band-specific reconstruction losses, including MSE, SSIM, and MS-STFT, are used in the reconstruction-based pretraining phase. Task-specific heads are used for different downstream tasks. Evaluation is conducted on 11 classification and regression tasks. Results show that Tiles achieves strong performance, while ablation studies investigate the effectiveness of its key design choices.
Critical strengths
[+] The paper studies a meaningful problem in accelerometer self-supervised learning. Different downstream tasks may rely on different temporal and spectral characteristics of accelerometer signals. Learning transferable representations that preserve signal information in different frequency bands is a practical research problem. The motivation is clearly presented and aligned with the proposed Tile method.
[+] The paper provides a diverse downstream task study beyond conventional human activity recognition benchmarks. The study on diverse task types helps assess the transferability of learned representations across tasks with different signal requirements.
Critical weaknesses
[-] The methodological novelty is relatively limited. The proposed Tile framework mainly integrates existing techniques, including frequency decomposition, Slot Attention-based aggregation, and frequency-specific losses. While their combination for accelerometer representation learning is potentially useful, the paper does not introduce a fundamentally new learning mechanism.
[-] The performance gap between Tile and the supervised baseline is insufficiently discussed. Although Tile achieves competitive performance against existing SSL methods, it underperforms the supervised baseline on 7 out of 11 downstream tasks, including all four regression tasks. Some performance gaps are substantial, particularly on JumpPower and Writing. However, the paper mainly highlights the tasks where Tiles outperforms the supervised baseline, without adequately discussing the reasons for its performance on the remaining tasks. A more thorough analysis would help clarify the limitations of the proposed approach and their transferability across different tasks.
Questions for authors
-
How much does frequency decomposition contribute compared with the band-specific reconstruction losses? Table 3 shows that removing frequency decomposition leads to only a 0.5% average performance decrease on regression tasks, whereas removing the auxiliary reconstruction losses causes a much larger decrease (33.7%). Would applying the same frequency-aware reconstruction losses to a model that directly encodes raw signals achieve comparable performance?
-
Can the authors further explain why Tiles underperforms the supervised baseline on 7 of the 11 downstream tasks, including all four regression tasks? In particular, is this performance gap mainly caused by the limitations of the learned representations or by the frozen-encoder adaptation setting? Have the authors evaluated full fine-tuning of the pretrained encoder to distinguish these possibilities?
-
How does Tiles compare with related multi-view and time-frequency representation learning methods mentioned in the paper, such as localized time-frequency contrastive learning (Taghanaki et al., 2023) and TSPulse (Ekambaram et al., 2026), under comparable pre-training and downstream evaluation settings?
5 comments
Q1: Frequency decomposition and reconstruction losses.
The “No frequency decomposition” ablation in Table 3 already evaluates a raw-input model with MSE, SSIM and MS-STFT. It uses a single 12-block ResNet encoder and applies these losses to the full-band reconstruction. This model achieves comparable average regression performance to Tiles, while classification performance drops by 5.9% on average.
Q3: Comparisons with multi-view and time-frequency methods.
We evaluated TSPulse (Ekambaram et al., 2026), which learns temporal, spectral and semantic representations, on all 11 downstream tasks. We also included HiMAE (Lee et al., 2026) as a hierarchical representation-learning comparison.
-
HiMAE: The pretrained weights used in the HiMAE paper are not publicly available. We therefore adapted HiMAE-Base to three-axis accelerometer inputs and pretrained it using the same CAPTURE-24 split and pretraining budget as Tiles. We retained its hierarchical encoder–decoder, five-sample patches, 80% masking and per-window normalization to [-1,1]. The autoencoder has approximately 1.25M parameters, and its encoder provides a 256-dimensional representation.
-
TSPulse: We used IBM’s
tspulse-block-dualhead-512-p16-r1checkpoint, recommended for classification. It was pretrained on approximately one billion time points from Monash and LibCity, covering energy, weather, traffic and other time series. We linearly interpolated our 250-sample inputs to 512, disabled masking and retained the full temporal, FFT and register-token embeddings across all three axes. Its frozen backbone and decoder contain approximately 0.37M parameters, compared with Tiles’ 4.8M. However, its full representation contains 9,936 features, compared with Tiles’ 256, meaning that the same MLP configuration gives TSPulse a much larger task-specific MLP head.
Both models were evaluated with frozen representations and using the same folds. We have not evaluated the localized time-frequency contrastive method of Taghanaki et al. (2023).
| Task | HiMAE | TSPulse | Tiles |
|---|---|---|---|
| Basketball ↑ | 60.2 | 60.5 | 67.1 |
| Cooking ↑ | 53.5 | 44.3 | 55.5 |
| Locomotion ↑ | 69.7 | 63.1 | 73.6 |
| TennisShot ↑ | 78.0 | 83.1 | 82.3 |
| Transportation ↑ | 52.6 | 52.1 | 62.8 |
| Workouts ↑ | 67.8 | 48.9 | 71.0 |
| Writing ↑ | 26.4 | 54.4 | 49.0 |
| JumpPower ↓ | 7.81 | 4.86 | 6.46 |
| NumSteps ↓ | 0.825 | 0.644 | 0.69 |
| ThrowSpeed ↓ | 4.46 | 2.86 | 3.03 |
| WalkDistance ↓ | 0.409 | 0.347 | 0.29 |
Tiles outperforms HiMAE on all eleven tasks and TSPulse on six, including five of the seven classification tasks. TSPulse leads on TennisShot and Writing, but trails Tiles by 22.1 points on Workouts, 11.2 on Cooking and 10.7 on Transportation. It achieves lower RMSE than Tiles on three of the four regression tasks.
These differences may reflect pretraining domain and representation choices. TSPulse is pretrained on diverse general time series, normalizes each window and axis using RevIN, and exposes 9,936 temporal, spectral and register-token features. Its detailed features may help estimate quantities related to timing, repetition and waveform shape. Tiles instead uses a compact 256-dimensional representation learned from wearable accelerometer data, which may better capture the movement patterns needed for activity classification.
References
Lee, Simon, et al. “HiMAE: Hierarchical Masked Autoencoders Discover Resolution-Specific Structure in Wearable Time Series.” International Conference on Learning Representations, 2026.
Ekambaram, Vijay, et al. “TSPulse: Tiny Pre-Trained Models with Disentangled Representations for Rapid Time-Series Analysis.” International Conference on Learning Representations, 2026.
Thank you for your reply, which has partly addressed my concerns. I still have some questions and would appreciate a brief clarification.
Regarding Figure 4, while the low-frequency trends seem to be reasonably reconstructed, some periodic patterns, particularly the ECG-like pattern in the second subfigure, appear less well preserved. Could the authors briefly explain why this might be the case? Does this suggest that the model reconstructs different frequency bands with varying accuracy? Also, could the authors provide some quantitative frequency-domain reconstruction metrics for the different frequency bands to better assess how well these patterns are preserved? A brief comparison with the baselines would also be helpful.
Thanks again for your efforts.
We thank you for your thoughtful review and valuable questions that have helped us improve our paper. We address your concerns and questions point by point below.
Tiles’ contribution is a representation that separates accelerometer signals into multiple time-frequency encodings, allowing downstream tasks to use different combinations of them. Frequency decomposition, competitive aggregation and band-specific reconstruction work together to learn this structure.
It is common for frozen SSL encoders to underperform end-to-end supervised models on some downstream tasks (Xu et al., 2025; Haresamudram et al., 2022). This is because supervised training adapts the entire encoder to each task, while frozen evaluation trains only the prediction head. To test how much freezing contributes to our performance gaps, we ran full fine-tuning of Tiles on all 11 downstream tasks.
Fine-tuning improves Tiles’ mean performance on every task and achieves a better mean than the supervised baseline on 8/11 tasks, including Writing. Cooking, JumpPower and ThrowSpeed still favor supervised training. These results support the frozen-encoder setting as a contributor to the original gaps, while showing room for improvement on the remaining tasks.
We will include the full fine-tuning results in the appendix.
References
Xu, Maxwell A., et al. “RelCon: Relative Contrastive Learning for a Motion Foundation Model for Wearable Data.” International Conference on Learning Representations, 2025.
Haresamudram, Harish, Irfan Essa, and Thomas Plötz. “Assessing the State of Self-Supervised Human Activity Recognition Using Wearables.” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 6, no. 3, 2022, article 116, pp. 1–47.