- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies test-time adaptation for medical time series from new subjects. The authors argue that standard entropy minimization updates all LayerNorm parameters along directions of model uncertainty and, when only a few windows are available, can turn correct predictions into incorrect ones. They propose LSTA: a rank-8 LayerNorm update subspace is meta-learned on training subjects. Support windows are used for one entropy-minimization step within this subspace, and the classification loss on query windows is used to learn . At test time, is fixed, and each new subject receives one projected-gradient update using at most eight unlabeled windows. With TeCh as the backbone, LSTA achieves higher average macro-F1 than seven TTA methods across five datasets with subject-level labels and is more stable under settings such as contaminated support windows.
Critical strengths
The problem is clearly defined and practically relevant: personalizing a model for a new subject using a few unlabeled windows. The method is simple, requiring only one additional projected-gradient step at inference time, with overhead comparable to Tent.
Critical weaknesses
- The gains are small and not statistically significant: LSTA scores 79.23 versus 78.19 for LN-TTA (), with improvements concentrated mainly on ADFTD. The simple Frozen-PA baseline outperforms LSTA on PTB but is omitted from Table 1. On Sleep-EDF, the “never below frozen” claim rests on a margin of just 0.06.
- The novelty is limited: is essentially a meta-learned low-rank preconditioner, similar in spirit to Meta-Curvature, CAVIA [1], and LEO [2]. The central premise that subject differences are low-rank is not directly validated.
[1]: Luisa Zintgraf, Kyriacos Shiarlis, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson. Fast Context Adaptation via Meta-Learning. In Proceedings of the 36th International Conference on Machine Learning (ICML), 2019.
[2]: Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Hadsell. Meta-Learning with Latent Embedding Optimization. In International Conference on Learning Representations (ICLR), 2019.
Questions for authors
-
Could the authors include Frozen-PA in Table 1? What happens when LSTA is combined with prediction aggregation?
-
Could the authors test the low-rank assumption directly? For example, they could fine-tune LayerNorm separately for each test subject using labels to obtain oracle updates, then examine the effective rank of those updates and how much of their variance the learned explains.
-
To separate the effect of the learned directions from that of the low-rank constraint, could the authors add a learnable full-rank preconditioner, as in Meta-Curvature, and a CAVIA-style comparison [1] that adapts context parameters?
-
Are the two pools split randomly or into contiguous time segments? Could the authors add a chronological setting in which the first windows are used for adaptation and later windows are predicted? This would more closely reflect deployment.
-
The experiments use only TeCh as the backbone. Do the conclusions hold with Medformer or a pretrained model such as LaBraM?
-
Could the authors provide paired tests at the subject level rather than relying only on sign tests over dataset–seed pairs?
[1]: Luisa Zintgraf, Kyriacos Shiarlis, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson. Fast Context Adaptation via Meta-Learning. In Proceedings of the 36th International Conference on Machine Learning (ICML), 2019.
6 comments
2/3 Response to Reviewer ADC6
Q2: Low-rank assumption.
As you proposed, for each subject we computed the labeled LayerNorm update at the frozen model (class mean removed), and fitted an oracle update with that subject's labels.
| APAVA | TDBrain | ADFTD | |
|---|---|---|---|
| Effective rank of subject updates | 1.6 | 1.2 | 1.8 |
| Unseen subjects' variance inside rank-8 PCA of training subjects | 92.7% | 97.2% | 97.8% |
| … inside a random rank-8 subspace | 0.06% | 0.23% | 0.13% |
| … inside B | 16.2% | 9.1% | 2.3% |
| Oracle gain over frozen kept inside B / random rank-8 | 100% / 58% | 100% / 92% | 99.6% / 84% |
- The low rank and shared assumption holds. Subject updates have effective rank below 2, and test subjects' updates lie almost entirely in a subspace learned from training subjects.
- What B explains. Although B holds only 2–16% of variance, that restriction keeps 99.6–100% of the gain, while a random rank-8 subspace keeps 58–92%. B captures the part of the subject difference that changes predictions, not its largest-variance part, because it is trained for safe unlabeled steps rather than reconstruction.
- A variance basis is less safe. Replacing B with the top-8 PCA basis of the oracle updates is on par on clean data, but significantly less robust under 50% contamination: B is better by +10.75 (APAVA), +2.10 (TDBrain) and +2.07 (PTB-XL).
Q3: Full-rank preconditioner and CAVIA-style comparison
We kept each update operator and trained it under our unlabeled protocol on the same frozen backbone. Mean over 6 datasets:
| Clean | 50% contaminated | |
|---|---|---|
| LN-TTA | 69.91 | 59.54 |
| Learned diagonal preconditioner | 70.36 | 59.86 |
| Random rank-8 subspace | 69.66 | 59.91 |
| LSTA | 70.82 | 64.28 |
We then ran the two under our protocol (unlabeled step, frozen backbone, step size and learning rate chosen on validation): Meta-Curvature with a full-rank matrix starting at identity, and CAVIA with an unconstrained linear map from 8 context parameters into LayerNorm. 50% contaminated support, 3 seeds:
| LN-TTA | LSTA | Meta-Curvature | CAVIA-style | |
|---|---|---|---|---|
| APAVA | 65.18 | 78.08 | 63.30 | 61.69 |
| TDBrain | 88.26 | 89.43 | 61.88 | 78.51 |
| ADFTD | 40.91 | 41.53 | 35.91 | 39.26 |
On clean data all three are within one point except on ADFTD (55.94 vs. 52.92 and 54.03, not significant). This separates the effects you ask about:
- Low-rank restriction. LSTA is significantly better than Meta-Curvature on all three datasets (+5.7 to +27.2) and than the CAVIA-style rule on two (+10.9 to +15.6). Due to restricting unlabeled step to a learned low-rank subspace.
- Learned directions. A random subspace is 4.4 points lower under contamination, and lower on clean APAVA (91.07 vs. 93.18), PTB (81.47 vs. 84.09) and ADFTD (51.08 vs. 55.32).
- CAVIA adapts its context parameters on a labeled support set and meta-trains the whole network. When we replace that with an unlabeled step, as our setting requires, it loses most of its robustness. What LSTA adds is making this kind of low-rank adaptation safe without labels, on a frozen model.
3/3 Response to Reviewer ADC6
Q4: Pools and chronological setting.
The pools are the first and second half of each recording in time, so they are contiguous; support comes from one half, the other half is predicted, and the roles are then swapped. We evaluated on the requested setting (LSTA minus frozen; bold: 95% subject-level interval excludes 0 for rows 1, 3 and 4; PTB-XL: 300-subject subset):
| Setting | APAVA | TDBrain | PTB-XL |
|---|---|---|---|
| Both halves, both directions (paper protocol) | +7.53 | +6.53 | +2.20 |
| First half → predict second half | +6.81 | +4.52 | +1.93 |
| Earliest 8 windows → predict all later | +6.96 | +6.37 | +3.65 |
| 8 windows from first 10% → predict remaining 90% | +12.54 | +5.37 | +2.78 |
- The gain holds when only early windows are used: LSTA is above the frozen model in every chronological setting, by up to +12.54.
- Later visits. On PTB-XL, adapting on a patient's earliest ECG and predicting that patient's later ECGs (315 patients, up to 4.7 years later) gives +6.86 [+4.82, +9.07] over the frozen model, and +8.20 [+1.22, +13.75] more than a year later. On two external cohorts, with no fitting on either cohort: +3.84 [+2.50, +5.33] (MIMIC-IV-ECG, 1,662 patients) and +3.37 [+2.97, +3.77] (CODE-15%, 9,729 patients).
Q5: Medformer and Transformer
We tested our method on backbones: Medformer and Transformer. (LSTA - LN-TTA) on 50% contaminated support:
| Backbone | APAVA | TDBrain | PTB-XL |
|---|---|---|---|
| Transformer | +11.64 | +3.90 | +5.93 |
| Medformer | +22.40 | +5.11 | −0.26 |
- Robustness carries over to both backbones: LSTA is significantly more robust than LN-TTA on 5 of 6 backbone–dataset pairs, by up to 22.40 points, and on none of them significantly below the frozen model.
- Clean gains: LSTA is above the frozen model on all 6 pairs (Medformer: APAVA +7.54, TDBrain +5.59, PTB-XL +1.91; Transformer: APAVA +6.65, PTB-XL +1.53, TDBrain +2.14).
Q6: Subject-level tests
We now use paired bootstrap tests over test subjects (10,000 resamples) comparing LSTA with the frozen model, Frozen-PA, LN-TTA and LN-TTA-Learnable on all seven datasets. (LSTA - Frozen Model) on clean data:
| APAVA (4 subjects) | TDBrain (8) | ADFTD (19) | PTB-XL (3,521) |
|---|---|---|---|
| +7.53 [+1.61, +16.74] | +6.53 [+1.99, +14.68] | +6.95 [+1.16, +12.95] | +2.27 [+1.85, +2.68] |
- We confirmed the gain over the frozen model at the subject level, and pooled over seven datasets the clean gain over LN-TTA is significant (p = 0.0017), with significant per-dataset gains on ADFTD (+3.89), MIT-BIH (+1.18) and Sleep-EDF (+0.14).
- Because some test sets are small, we also ran 5-fold subject-level cross-validation with retrained backbones (3 seeds), so that every subject is tested once. LSTA minus frozen is +7.66 [+4.19, +11.16] on ADFTD (88 subjects) and +2.85 [+1.01, +4.70] on TDBrain (50 subjects), where LSTA is also above Frozen-PA (+1.21 [+0.17, +2.28]).
We thank the reviewer; we will add analysis in the revised paper for each suggestion. We hope these answers address the reviewer's concerns, and we would be happy to answer any further questions.
1/3 Response to Reviewer ADC6
We thank the reviewer for the concrete, testable suggestions and for finding the paper "clear and easy to follow." We ran every experiment in the way you suggested. We will upload the revised paper with changes marked in blue. The method is unchanged in every experiment. Scores are macro-F1; intervals are 95% paired bootstrap over test subjects.
W1 / Q1: Significance, Frozen-PA and Sleep-EDF.
On clean data, LSTA's gain over LN-TTA is significant at the subject level (pooled over seven datasets, p = 0.0017). LSTA is the best method overall, and its advantage is largest where adaptation is unreliable:
Best overall on clean data. LSTA has the highest average macro-F1 (79.23 vs. 76.93 for the best baseline) and the gain over LN-TTA holds on the largest cohort, CODE-15% (9,729 patients): +1.17 [+0.83, +1.52].
Significant gains under unreliable support. With 50% contaminated support LSTA beats LN-TTA by up to 11.8 points (APAVA 77.41 vs. 65.62), cuts confident errors from 17.6% to 0.1% on PTB-XL, and is significantly more robust on 5 of 6 backbone–dataset pairs.
At no extra cost. One gradient step on 8 unlabeled windows, the same FLOPs as Tent (8–21 ms on a GPU).
Frozen-PA. Our main results table compared published adaptation methods, so Frozen-PA which only averages the frozen model's predictions over a subject's windows, was reported in the support-size analysis. We have added Frozen-PA and LSTA + PA to the main results table:
W2: Novelty and the low-rank premise.
We agree with the description: LSTA is a meta-learned low-rank preconditioner, and we stated in the revised related work that we do not claim a new adaptation primitive. The difference to Meta-Curvature, CAVIA and LEO is the setting in which the step is taken: they adapt on a labeled support set and mostly meta-train the network itself. LSTA adapts on a few unlabeled windows of a new patient, on an already-trained frozen model, which is the setting of clinical deployment. Our contribution is to show that in this setting the learned low-rank restriction is what makes the unlabeled step safe: learned full-rank (Meta-Curvature) and CAVIA-style rules trained under the same protocol are significantly less robust under contaminated support: by 5.7 to 27.2 points for Meta-Curvature, and by 10.9 to 15.6 points for the CAVIA-style rule on APAVA and TDBrain (Q3).
The premise is now tested directly, as you proposed (Q2): subject-specific updates have an effective rank below 2, and a rank-8 PCA subspace estimated from training subjects contains 93–98% of unseen subjects' updates (random rank-8: below 0.3%).