- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes EF-GO, a framework for adapting pretrained EEG foundation models to multimodal EEG-fNIRS representation learning. It combines self-supervised fNIRS pretraining, region-prior-guided graph aggregation, and time-aware Neural ODE alignment. Experiments cover six downstream tasks with two EEG backbones. The problem is well motivated, and the results suggest benefits from multimodal adaptation. However, the current experiments do not sufficiently establish the specific contribution of continuous-time, delay-aware alignment or clarify the conditions under which adaptation is beneficial.
Critical strengths
- Adapting pretrained EEG models to paired EEG-fNIRS data addresses a meaningful representation-learning problem involving spatial and temporal heterogeneity.
- The evaluation includes multiple task categories and two pretrained backbones, with improvements over the corresponding EEG-only models on several tasks.
- The ODE module is used during cross-modal post-training and excluded from downstream inference, avoiding ODE integration in the inference path.
Critical weaknesses
- The specific benefit of continuous-time modeling is not isolated. Table 3 shows that adding ODE-based alignment can improve performance, but removing the entire alignment module does not establish whether continuous-time dynamics are responsible for the improvement. The gains could also reflect the additional cross-modal alignment objective or transformation capacity. The task-specific multimodal baselines do not provide a controlled comparison within the same pretrained-backbone adaptation setup. A non-ODE alignment model using the same paired data, alignment targets, and comparable capacity and training budget would help establish the contribution of the proposed formulation. This is a central concern because continuous-time neurovascular modeling is a principal methodological claim.
- The temporal-weighting ablation does not clearly establish the contribution of learned delay preferences. Table 3 does not specify the replacement weights used for “w/o time-aware,” and Figure 4(b) lacks a numerical vertical scale. Appendix F does report that the effective sigmoid weights change approximately from 0.7311 to 0.7291, which provides useful context. However, this does not clarify the ablation configuration or quantify the individual delay-dependent profiles. Since Equation (13) does not normalize by the sum of the learned weights, replacing them with a different constant may also change the overall alignment-loss scale. A fixed-weight comparison with a matched overall scale is needed to distinguish the contribution of relative delay preferences. The reported peak-shaped curves alone do not establish that these preferences account for the downstream gains.
- The backbone-dependent degradation is not adequately investigated. On hBCI-MI, Table 2 shows that EF-GO(C) reduces CBraMod’s balanced accuracy from 0.6377 to 0.5922 and AUC-ROC from 0.7124 to 0.6236. Section 4.1 acknowledges that the backbones have different strengths, but does not explain this degradation. Table 3 only provides EF-GO(L) ablations on Cognitive-WG and hBCI-MA, leaving the affected backbone and task unexamined. This does not negate the improvements elsewhere, but it limits the evidence for the framework’s applicability across backbones. A targeted analysis would help identify whether the degradation arises from fusion, alignment, or downstream adaptation.
- Robustness to the selection of held-out subjects remains unclear. Section 4 uses three validation subjects and three test subjects per dataset. Appendix D reports five random seeds, but does not clarify whether the subject partitions change across these runs. It is therefore unclear whether the reported variability captures sensitivity to held-out-subject selection or only training randomness. Clarifying the split procedure and reporting results across multiple subject partitions would strengthen the subject-independent generalization claims.
Questions for authors
- How does ODE alignment compare with a non-ODE alignment model under matched training conditions?
- What exact weights and normalization are used in “w/o time-aware,” and does the improvement persist when the overall loss scale is matched?
- Can targeted EF-GO(C) ablations explain the hBCI-MI degradation?
- Do the five random seeds change the held-out subjects, or only the training randomness?
2 comments
Q3. Performance degradation with CBraMod on hBCI-MI
The effect of multimodal adaptation varies across both tasks and backbones. There are inherent differences in the contributions of EEG and fNIRS signals across various tasks, such as cognitive, emotional, and motor imagery tasks. At the same time, due to differences in pre-training datasets and pre-training methods, different backbone models inherently exhibit performance variations. For example, EF-GO(C) achieves higher balanced accuracy than EF-GO(L) on both REFED tasks, where the fNIRS-only branch also performs well in the LaBraM ablations.
To investigate the hBCI-MI results, we measured the effective rank of model weights and representations before and after downstream fine-tuning. The mean effective rank of EEG backbone weights increased slightly, while the effective rank of fNIRS branch weights and representations were essentially unchanged. This suggests that the model's performance did not degrade as a result of fine-tuning. However, the effective rank of the EEG representations showed a significant decline, which may indicate that the EEG signals in this dataset are more sensitive to this backbone, and that the EEG representations may have experienced a collapse in representation. This phenomenon may be the cause of the performance decline.
| Quantity | Before fine-tuning | After fine-tuning | Relative change |
|---|---|---|---|
| EEG backbone weights, mean effective rank | 91.83097 | 92.00325 | +0.19% |
| fNIRS branch weights, mean effective rank | 70.42998 | 70.42979 | −0.0003% |
| EEG representations, effective rank | 89.81200 | 69.78312 | −22.30% |
| fNIRS representations, effective rank | 3.21444 | 3.21482 | +0.0119% |
Q4. Cross-subject cross-validation
In the experiments, the subject partitions were fixed across all five random seeds. The seeds varied the training randomness, so the reported variability reflects training variability within a fixed split.
To assess performance across different held-out subject groups, we added five-fold subject-level cross-validation on Cognitive-WG dataset. Each fold held out a distinct group of subjects for testing. Validation subjects were selected from the remaining subjects, and the rest were used for training. Both EF-GO variants achieved higher mean scores than all three task-specific EEG–fNIRS models on all reported metrics. EF-GO(L) achieved the highest mean balanced accuracy, PR-AUC, and ROC-AUC among the evaluated methods. It is worth noting that task-specific EEG-fNIRS models all exhibited a decline in performance under this experimental setup, whereas the Foundation Models remained highly stable under these conditions, which to some extent demonstrates the Foundation Model's robustness to subject-to-subject variability.
| Method | Balanced accuracy | PR-AUC | ROC-AUC |
|---|---|---|---|
| FGA-Net | 0.6340±0.0498 | 0.6521±0.0269 | 0.6715±0.0408 |
| M2NN | 0.6522±0.0219 | 0.7010±0.0261 | 0.7296±0.0295 |
| STA-Net | 0.6114±0.0506 | 0.6517±0.0648 | 0.6654±0.0459 |
| LaBraM | 0.7022±0.0219 | 0.7372±0.0520 | 0.7664±0.0423 |
| CBraMod | 0.6807±0.0484 | 0.7233±0.0506 | 0.7480±0.0498 |
| EF-GO(L) | 0.7386±0.0256 | 0.7900±0.0338 | 0.8139±0.0291 |
| EF-GO(C) | 0.6949±0.0416 | 0.7155±0.0496 | 0.7532±0.0395 |
Thank you for identifying the controls needed to evaluate our alignment method and for raising questions about temporal weighting, the performance drop with CBraMod, and held-out subject selection.
Q1. Comparison with non-ODE alignment
We added three non-ODE variants that use MSE, a cosine-based loss, or a contrastive loss to directly align paired EEG and fNIRS embeddings after graph aggregation. Graph–ODE achieved the highest mean score on all three metrics on hBCI-MA and on balanced accuracy and ROC-AUC on Cognitive-WG. Cosine-based alignment achieved the highest mean PR-AUC on Cognitive-WG. These comparisons quantify the downstream performance of Graph–ODE relative to direct alignment. Isolating the contribution of continuous-time modeling would additionally require a discrete alignment model with matched temporal targets and comparable transformation capacity.
Q2. Temporal weight replacements and alignment-loss scale
In the original
w/o time-awareablation, we set every time-step parameter to 1 before applying the sigmoid in Equation (13), yielding a uniform effective weight of approximately 0.7311. We expanded the ablation to include two uniform settings with pre-sigmoid values of 1 and 10, a fixed profile derived from a hemodynamic response function (HRF), and a random profile. The learned weights achieved the highest mean scores on all three metrics on both tasks.As for Figure 4(b) we will add a numerical vertical axis to show the magnitude of the learned weight differences.