- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents a pretraining framework for cross-dataset multimodal irregular health sensing, which supports both supervised finetuning and zero-shot transfer for downstream tasks. The pretraining is done through masked reconstruction, where time series signals from different sensor modalities are encoded through a shared encoder. The authors use a uniform binning to account for irregularities in the different sensor modalities, and augment the embeddings with physical-duration patch embedding and temporal and variable identity embedding. After pretraining, zero-shot transfer is achieved through a sensor–language bridge, which aligns embeddings with template captions. The proposed framework performs better than the baseline methods in 5 out of 6 downstream datasets (two of which are used for self-supervised pretraining).
However, as the paper stands, there are critical flaws in the experiment design that casts doubt on the validity of the results and the conclusions that rest upon them. I will expand on this in the weakness section. That being said, I do credit the authors for their significant effort in providing the current results and I see promise in this work should the flaws be addressed.
Critical strengths
- The paper provides reasonably well controlled evidence that heterogeneity-aware design offers real gain. Comparing with MIRA in Table 1, which is the only baseline pretrained on the same datasets as HARP, we can clearly see the advantage offered by the authors' heterogeneity-aware design. The fact on regularly sampled, complete benchmarks (Table 7) the gains mostly vanish also helps sharpen this point.
- The authors provide direct evidence that their proposed pretraining recipe outperform other strategies. Table 10 shows that under identical data access, HARP beats the two baseline methods on all datasets and on both metrics.
Critical weaknesses
- The baseline choices in Table 1 are questionable.
- The supervised reference (ATENet) is weak and excluded without proper justification. MTM (2509.17809) reports 88.0/58.6 on P12 and 90.3/58.3 on P19 (AUROC/AUPRC). From Table 3 in MTM, other methods such as ViTST also exceed ATENet on P19. The authors should include stronger baselines or provide reasonable justifications of why ATENet was the only supervised reference.With ATENet trained on the same full set of target labels used in linear probing, the author's excluding it from the ranking seems not sufficiently justified. One potential argument is that ATENet was trained end-to-end while HARP's main weights were frozen and only a linear projection head was allowed to move. To sustain this argument, the authors should provide experiments comparing supervised baselines with full-weight finetuned HARP.
- Pretraining data is confounded with method. According to Appendix A.3, MIRA was the only baseline in Table 1 that was pretrained on the same datasets as HARP. Even so, since MIRA was significantly larger than HARP (113.4M vs 2.0M in params), it could be argued that MIRA was undertrained on such small datasets. Table 11 suggests that pretraining data plays a significant role in downstream performance, further necessitating removing this confound in fair comparisons with other methods.
- Several closely related pretraining methods are missing from the baselines. STraTS (2107.14293) and DuETT (2304.13017) are close relatives to HARP's design and both are absent from comparisons. Moirai (2402.02592) is cited but not evaluated.
- Some basic controls are absent. 2602.19531 shows that summary statistics + XGBoost achieves 85.7 AUROC on P12 and 90.0 on P19. Less critical omissions include HARP trained from scratch, HARP full-weight (end-to-end) finetuned, and a random-init frozen encoder linear probing.
- The paper does not clarify how irregular sensor signals are fed into regular-grid models. In particular, NormWear's 4.45 AUPRC on P19 suggests a potential pipeline failure instead of model limitation.
- The zero-shot transfer performance is weak and baselines are near-chance. Table 2 shows that most LLM and TSLM baselines have near-chance (in some cases, worse than chance) performance. Beating these baselines shows little.
Questions for authors
- Add the SOTA supervised models (e.g. MTM) under the same protocol, and include them in the ranking or provide more justifications for exclusion from ranking.
- Add dataset-matched and capacity matched baselines and the missing baselines from the earlier weakness section.
- Please consider narrowing the scope of your claims about zero-shot transfer given the absolute performance and poor baseline performances.
7 comments
Response to pretraining data confound (W1.2 & Q2)
W1.2 and Q2. Pretraining data confound
We thank the reviewer for this point. Table 1 supports two distinct claims, and only the second requires controlling for pretraining data:
- Existing pretrained models transfer poorly to unseen heterogeneous datasets. Table 1 evaluates each model’s released checkpoint with linear probing, testing whether existing models can be reused as they are. Here, each model’s own pretraining data is part of what is being evaluated, so matching the pretraining data is not required.
- HARP’s pretraining mechanism is more effective than that of alternative irregular time-series foundation models. We agree that this claim is confounded by pretraining data and model capacity.
To address the concern about the second claim, we additionally pretrained the baseline MOMENT from random initialization on the same data as HARP (MIMIC-IV + P12), using its official implementation, and evaluated it under the same linear-probing protocol. Results are shown below (AUROC / AUPRC).
With the same pretraining data, HARP achieves the highest mean AUROC across all six datasets in this comparison. These results indicate that the performance gain of HARP comes from its heterogeneity-aware pretraining design, which embeds temporal and channel identity, rather than from pretraining data or model size. We have revised Section 4.1 to distinguish this controlled comparison from the comparison using released checkpoints.
| Model | MIMIC-III AUROC | AUPRC | MIMIC-IV AUROC | AUPRC | P12 AUROC | AUPRC | P19 AUROC | AUPRC | PMData AUROC | MF1 | PAM AUROC | MF1 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Pretrained on MIMIC-IV + P12, linear probing | ||||||||||||
| MOMENT | 76.86 ± 0.95 | 22.84 ± 1.60 | 78.88 ± 0.80 | 32.27 ± 1.22 | 71.86 ± 2.44 | 29.12 ± 2.80 | 81.23 ± 2.07 | 41.14 ± 5.17 | 53.81 ± 6.24 | 20.10 ± 1.42 | 96.80 ± 0.70 | 82.73 ± 2.90 |
| HARP | 87.27 ± 0.66 | 42.97 ± 1.87 | 88.26 ± 0.24 | 51.94 ± 0.90 | 85.23 ± 1.81 | 51.64 ± 2.96 | 87.80 ± 2.60 | 50.55 ± 3.53 | 60.04 ± 4.38 | 21.06 ± 1.75 | 99.66 ± 0.09 | 96.03 ± 0.74 |
Regarding the MIRA pretraining concern, we observe that MIRA's pretraining had converged: on every retained checkpoint, its forecasting loss is constant on training windows (0.00053) and on held-out patients (0.00049 on MIMIC-III, 0.00056 on PhysioNet-2019), with no train–validation gap, while the learning rate decays sevenfold.
Response to related pretraining method (W1.3) and baseline data processing (W1.5)
W1.3 other related pretraining methods
DuETT: Since DuETT is designed for self-supervised pretraining on a single dataset, we compare in a matched in-domain setting: DuETT and HARP are each pretrained and evaluated on the same target dataset, using DuETT’s official code and the same variable set as all other baselines (e.g., 12 variables for MIMIC-III and MIMIC-IV, following [1]). We did not add STraTS separately, as DuETT reports outperforming it in its original paper [2]. Results are shown below.
In the matched in-domain setting, HARP outperforms DuETT on four of the six datasets, and the differences on MIMIC-III and P12 are within one standard deviation. More importantly, the single HARP encoder from Table 1, pretrained only on MIMIC-IV and P12, outperforms DuETT pretrained on each target dataset on all six datasets, including four never seen during pretraining. DuETT, by design, requires a fixed input variable set and must be pretrained separately for each dataset, whereas HARP transfers one encoder across datasets with different variables and sampling patterns.
| Dataset | DuETT, in-domain | HARP, in-domain | HARP (Table 1) |
|---|---|---|---|
| MIMIC-III | 85.92 ± 0.65 | 85.46 ± 1.26 | 87.27 ± 0.66 |
| MIMIC-IV | 87.31 ± 0.60 | 88.06 ± 0.26 | 88.26 ± 0.24 |
| P12 | 85.01 ± 1.49 | 84.61 ± 1.25 | 85.23 ± 1.81 |
| P19 | 86.31 ± 2.51 | 86.86 ± 2.31 | 87.80 ± 2.60 |
| PMData | 56.89 ± 2.80 | 60.55 ± 3.66 | 60.04 ± 4.38 |
| PAM (macro-F1) | 95.09 ± 1.39 | 97.74 ± 0.42 | 96.03 ± 0.74 |
Bold: best; italic: second-best.
Bold: best; italic: second-best.
Moirai: we added Moirai as another time-series foundation model since it pretrained on different datasets and not fixed to specific data inputs.The performance is as shown. HARP achieves higher mean AUROC than Moirai on all six datasets, with larger gains on the clinical benchmarks. Same as other time-series foundation models trained on regular datasets, it cannot transfer to heterogeneous irregular datasets, further supporting the effectiveness of HARP’s heterogeneity-aware representations. We will include Moirai in the revised Table 1.
| Model | MIMIC-III AUROC | MIMIC-III AUPRC | MIMIC-IV AUROC | MIMIC-IV AUPRC | P12 AUROC | P12 AUPRC | P19 AUROC | P19 AUPRC | PMData AUROC | PMData MF1 | PAM AUROC | PAM MF1 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Chronos-2 | 78.27 ± 0.80 | 26.91 ± 1.15 | 80.49 ± 0.36 | 35.46 ± 1.75 | 72.83 ± 1.83 | 37.33 ± 1.59 | 81.31 ± 2.96 | 40.67 ± 4.07 | 56.87 ± 3.58 | 19.61 ± 0.71 | 90.67 ± 0.64 | 89.79 ± 0.52 |
| (new)Moirai | 78.64 ± 0.71 | 26.34 ± 1.86 | 79.68 ± 0.56 | 33.90 ± 1.27 | 68.78 ± 1.02 | 26.66 ± 0.91 | 79.69 ± 1.75 | 37.59 ± 4.99 | 54.88 ± 2.68 | 23.31 ± 0.54 | 98.91 ± 0.17 | 91.98 ± 0.71 |
| HARP | 87.27 ± 0.66 | 42.97 ± 1.87 | 88.26 ± 0.24 | 51.94 ± 0.90 | 85.23 ± 1.81 | 51.64 ± 2.96 | 87.80 ± 2.60 | 50.55 ± 3.53 | 60.04 ± 4.38 | 21.06 ± 1.75 | 99.66 ± 0.09 | 96.03 ± 0.74 |
[1.] Shukla and Marlin, 2021. “Multi-Time Attention Networks for Irregularly Sampled Time Series”, ICLR.
[2.] Labach et al., 2023. “DuETT: Dual Event Time Transformer for Electronic Health Records”, MLHC.
W1.5 Clarification of regular-grid processing.
-
We have added details to the Appendix A.3 on how irregular records are presented to regular-grid baselines. These baselines take fixed-length inputs split into fixed-step patches. Following mTAN, each variable is represented as a sequence of length 512: its observed values are placed in temporal order, and the remaining positions are zero-padded. For example, a MIMIC-III patient with 216 heart-rate observations has these values in the first 216 positions and zeros in the remaining 296. We then apply each baseline’s own preprocessing and patching: MOMENT uses patches of 8 steps, and SLIP applies mean-absolute normalization, uses patches of 8 or 16 steps, and excludes empty patches when pooling.
-
We thank the reviewer for flagging the NormWear result on P19. We found that its evaluation used only the first 12 hours of each record instead of the full 60-hour window used by all other methods. With the full window, NormWear achieves 79.38 ± 1.88 AUROC and 32.34 ± 4.31 AUPRC, in line with other TSFMs, and HARP remains higher. We will replace the erroneous result and update the affected aggregate gains. | Model and dataset | AUROC | AUPRC | |---|---|---| | NormWear, P19 (60 h) | 79.38 ± 1.88 | 32.34 ± 4.31 | | HARP, P19 (60 h) | 87.80 ± 2.60 | 50.55 ± 3.53 |
Response to supervised baseline and basic controls (W1.1, W1.4, Q1)
General response:
We thank the reviewer for providing insightful comments about well-controlled evidence and baseline selection. Our primary question is how to learn transferable representations across heterogeneous sensing configurations and adapt them to health classification when target outcome labels are limited or unavailable. In response to the reviewer’s concerns about
The details are as follows:
W1.1 & Q1. Baseline selections
Thank you for these suggestions. We have added MTM (KDD ‘25) as a stronger supervised baseline, following its official implementation under our splits and metrics. Our reproduced results (close to those reported) are in the following table. From the newly added results, we observe that MTM is a stronger supervised reference than ATENet. HARP with linear probing still achieves competitive performance.
We exclude supervised models from the Table 1 ranking and use them as references mainly because of a difference in protocol. Supervised models are trained from scratch and update all model parameters using labels. In contrast, Table 1 evaluates frozen pretrained encoders, training only a linear classifier. Since our main question is whether a pretrained encoder can be used on unseen heterogeneous datasets, the fair comparison is among pretrained foundation models under the same frozen-encoder protocol. Under this protocol, HARP outperforms existing foundation models by 5.4 points on average. We will add this clarification underlying Table 1.
W1.4 Basic controls and statistical baselines
Thank you for pointing the baseline and control setting. We have added the requested controls, all evaluated with the same splits, label access, and metrics:
Summary statistics + XGBoost, following its official implementation and using our validation splits and metrics.
HARP from scratch vs. full fine-tuning. With the same architecture, fine-tuning from the pretrained encoder outperforms training from scratch on all six datasets. This isolates the benefit of pretraining. With full fine-tuning, HARP achieves the best AUROC on five of the six datasets, including against MTM.
Bold: best; italic: second-best.