- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
PHASE: A Physiology-Guided Hierarchical Foundation Model for Intracranial EEG proposes a two stage approach for iEEG analysis. The first stage is dedicated to multi-scale temporal analysis. The second stage focuses on spatial features and is channel-agnostic. Both backbones are JEPA-based, but this decision is not ablated. Overall, the evaluation is thorough and the results are strong.
Critical strengths
- The paper is well written and clearly structured
- The model is well described in all its components, which are meaningfully justified as well
- The model is lightweight and reaches SoTA performance already
Critical weaknesses
- The JEPA objective is one of the more catchy contributions, but it’s not ablated; indeed, Table 2 seems to suggest that it’s not particularly effective
- The training split of the datasets used for downstream tasks comprise the vast majority of the pretraining data, so this could partly explain the improved performance on them. For example, NeuroCLUS ablates this choice, though in their case one dataset, SWEC, made up for 95% of their training data, so this was even much less important than for this work. Nonetheless, they found a statistically significant improvement at TreeBank Speech.
Questions for authors
- An ablation of the training dataset over the results of the downstream tasks is necessary to distinguish the effects of the architecture choices, following or at least borrowing from NeuroCLUS would be a good idea
- The model scaling is not explored; while this is not quite necessary due to the already SoTA results, the model lends itself well to being scaled upwards both in parameters and in training dataset. A positive scaling result would strengthen the contribution significantly
5 comments
Response to Reviewer r2L9 2/2
[Q2] The model scaling is not explored; while this is not quite necessary due to the already SoTA results, the model lends itself well to being scaled upwards both in parameters and in training dataset. A positive scaling result would strengthen the contribution significantly
Thank you for the encouraging suggestion. We added a model-size scaling analysis. We pretrained PHASE-T encoders at two additional sizes, 18.7M and 40.3M parameters (coarse/fine widths 384/192 and 512/256, versus 768/384 in the 106.7M model of the paper), scaling width and depth together. The pretraining corpus, 60K updates, and optimization settings are the same as for the 106.7M model, with no per-size retuning, and all three encoders are evaluated with the same frozen probes.
| Encoder | HFO F1 | Ictal F1 | Sleep F1 | Lobe-5 F1 | Region-12 F1 | Channel AUC | Outcome AUC |
|---|---|---|---|---|---|---|---|
| 18.7M | 0.7957 | 0.8490 | 0.7213 | 0.4457 | 0.2975 | 0.7875 | 0.6473 |
| 40.3M | 0.8048 | 0.9162 | 0.7311 | 0.4700 | 0.3201 | 0.8102 | 0.6364 |
| 106.7M (Table 1) | 0.8138 | 0.8958 | 0.7413 | 0.4822 | 0.3398 | 0.8394 | 0.6369 |
Performance increases with model size from 18.7M to 106.7M on most scores. Since the data and number of updates are fixed, these gains reflect model capacity. Even the 18.7M encoder is comparable to or better than the released foundation models in Table 1 on most scores. We scaled model size only; scaling the pretraining corpus together with the update budget is left to future work.
The Authors have addressed my concerns regarding the training dataset, but the results are being presented more positively than what is actually shown. Specifically, removing the in-domain data narrows the gap with the previous models substantially, and on one task (sleep) the results are actually below the baselines. This points to substantial contributions from the data mixture, with some improvements coming from the architecture itself (NeuroCLUS, in contrast, saw minor improvements). These should be the new headline results for a fair comparison, but since the manuscript has not been updated I cannot check.
Regarding the JEPA objective, once again the new ablation weakly corroborates the contribution, but the results mostly show that the physiological targets are sufficient, with JEPA only significantly improving on sleep. Moreover, the proper ablation would also replace JEPA with a generic self-supervised term to keep EMA, predictor, and loss weights.
Interestingly, sleep is the only task on which both ablations are significant, suggesting data-dependent behavior that is not explored by the Authors (perhaps a training run of physio-only with BrainTreeBank and no sleep?).
Response to Reviewer r2L9 1/2
We thank Reviewer r2L9 for the constructive review and for finding the paper clearly written, the model components well justified, the evaluation thorough, and the results strong for a lightweight model. We added three new analyses: an ablation that removes the JEPA objective (Physio-only), a PHASE-T model pretrained on BrainTreebank alone (no Omni-iEEG training participants), and a model-size scaling study. The detailed responses follow.
[W1] The JEPA objective is one of the more catchy contributions, but it's not ablated; indeed, Table 2 seems to suggest that it's not particularly effective
Thank you for pointing this out. We added an ablation that removes the JEPA objective. Physio-only drops the latent-prediction term from Equation 2 and keeps the physiological loss; the corpus, 60K updates, optimization settings, and frozen-probe protocol are the same as in Table 2.
Removing the JEPA objective lowers all seven scores. Since Full PHASE-T also exceeds Latent-only on every score, the JEPA objective and the physiological targets both contribute to the effectiveness of the PHASE.
[Q1, W2] The training splits of the downstream datasets make up most of the pretraining data, which could partly explain the downstream gains; NeuroCLUS, whose training data were 95% SWEC, still found a significant improvement on TreeBank Speech. An ablation of the pretraining data, following or borrowing from NeuroCLUS, is needed to separate data effects from architecture choices.
Thank you for this helpful suggestion. We retrained PHASE-T for 60K updates on the BrainTreebank pretraining split alone: 10 participants from a single site, recorded during naturalistic listening. This removes the other eight sites from pretraining, including Detroit and UCLA, the two largest sites by recording hours, and all clinical-monitoring recordings. The resulting model remains comparable to or better than the released foundation models in Table 1 on most scores, so PHASE-T does not rely on the largest sites or on clinical-monitoring data; adding them improves it further, which is also consistent with the in-domain gain NeuroCLUS reports.