- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces EpiTAR-Bench, a benchmark for evaluating the transferability, adaptation, and reliability of epilepsy models across heterogeneous EEG and intracranial EEG (iEEG) settings. The benchmark covers five datasets, seven transfer directions involving cross-dataset, cross-modality, and cross-time shifts, 16 models from different families, two input window lengths, and five levels of target-domain supervision. The evaluation includes both seizure detection and seizure prediction.
The authors additionally propose RIVER, a model designed to handle heterogeneous electrode configurations through Observability-Aware Partial Transport (OAPT), which uses state-conditioned unbalanced optimal transport to construct shared latent representations.
Experimental results examine the generalization of EEG foundation models, the performance recovery achieved through target-domain adaptation, the influence of input window length, and the relationships among discrimination, calibration, and SOZ evidence. The findings highlight the dependence of transfer performance on specific tasks, transfer settings, and supervision levels.
Critical strengths
S1. An important and clinically relevant research problem.
EEG is a major modality for clinical epilepsy monitoring, while differences in recording conditions, patient populations, and electrode configurations present substantial challenges for model generalization. The paper discusses these practical issues through a unified benchmark spanning scalp EEG and iEEG, including both seizure detection and prediction. The study is relevant to understanding how existing models perform under heterogeneous clinical recording conditions.
S2. Comprehensive experimental settings and systematic evaluation.
The benchmark evaluates a relatively broad collection of models across multiple transfer settings, window lengths, and target supervision levels. The authors also provide extensive experimental results and employ paired statistical tests with multiple-comparison corrections. These efforts contribute to a systematic characterization of model performance and adaptation behavior under different evaluation conditions.
S3. Meaningful evaluation of window length sensitivity and clinically relevant evidence.
Beyond conventional classification metrics, the paper examines the influence of longer input windows, probability calibration, and spatial evidence associated with the seizure onset zone. In particular, the SOZ analysis provides an additional perspective on the relationship between classification performance and clinically relevant spatial information. These evaluation dimensions are valuable for understanding the behavior and limitations of epilepsy monitoring models.
S4. A reasonable and potentially useful design for heterogeneous electrode modeling.
RIVER introduces a state-conditioned unbalanced optimal transport mechanism to accommodate variable electrode configurations and partially observed signals. The use of transported mass to regulate the contribution of observed representations is a reasonable design choice. The authors also provide parameter-matched ablation studies and electrode scarcity experiments, offering useful empirical support for the proposed architecture.
Critical weaknesses
W1. Limited generalizable insights and mechanistic explanations
Many conclusions remain specific to particular dataset–task combinations, with limited investigation into the underlying mechanisms. Although the benchmark provides extensive empirical results, it offers relatively few new generalizable insights into why models exhibit different transfer and adaptation behaviors.
W2. Insufficient discussion of related benchmarks and heterogeneous electrode modeling
Earlier epilepsy benchmarks, including SzCORE and Quantifying the Generalization Gap in Seizure Detection, receive insufficient attention. Moreover, the methodological connections and differences between RIVER and existing heterogeneous electrode modeling approaches, such as LUNA, MVPFormer, and NeuroCLUS, are not adequately discussed. The relevance of cross-physiological-signal transfer methods such as Brant-X also deserves clarification.
W3. Missing important baselines and limited dataset coverage
The evaluation omits representative models such as REVE, as well as iEEG/ECoG-specific foundation models such as NeuroCLUS, which are particularly relevant to the paper's focus. The limited number of datasets and independent transfer configurations, especially for cross-modality and cross-time transfer, also constrains the generalizability of the findings.
W4. Inconsistent performance advantages of RIVER
RIVER does not consistently outperform competing models across tasks and metrics. Some comparisons in Figure 3 emphasize model-family averages rather than the strongest individual baselines, while Figure 4 shows inferior performance in certain cross-time settings. Table 1 also suggests that RIVER does not exhibit a consistent advantage in adaptation gains under limited target supervision.
W5. Insufficient evidence for task-dependent cross-modal transfer conclusions
The scalp EEG-to-iEEG experiments involve few transfer configurations. Detection and prediction also use different source and target datasets, simultaneously changing task type, data distribution, electrode configurations, and patient populations. These confounding factors make it difficult to attribute the observed differences to task characteristics and limit the generalizability of the conclusions.
W6. Potential pretraining–evaluation data overlap
Some evaluated foundation models were pretrained on datasets such as TUEG and CHB-MIT, which may overlap with downstream evaluation data, particularly since TUSZ is derived from TUEG. The authors should verify the pretrained checkpoints, dataset versions, and patient- or recording-level splits to assess potential data contamination and its impact on zero-shot transfer evaluation.
W7. Dataset imbalance and inconsistent label construction
Certain tasks exhibit substantial class imbalance, such as the low positive-window proportion in CHB-MIT Detection. Moreover, seizure prediction datasets use inconsistent temporal exclusion intervals for interictal negative samples. Although partly attributable to dataset constraints, these differences may affect F1, AUPRC, calibration, and cross-dataset comparisons, warranting further examination.
Questions for authors
Q1. The benchmark reveals different adaptation patterns across transfer settings, particularly the diminishing gains beyond 25% target supervision. What factors do the authors believe primarily explain these differences? Can the existing results provide further evidence for these explanations?
Q2. How does the transported mass in RIVER relate to the actual availability of electrophysiological information? Do the authors have additional evidence or analyses showing how this quantity responds to changes in electrode coverage, beyond the reported classification performance?
2 comments
Further Questions Welcome
Thank you again for your thoughtful questions. We realize the discussion period is nearly over, but if time permits, we’d appreciate hearing whether our response addresses your concerns. If any points remain unclear, we’d be happy to clarify.
Response to Reviewer pREW
We sincerely thank the reviewer for the careful assessment! We respond in the order of all weaknesses and questions.
Weaknesses
W1. Generalizable insights and mechanisms
We connect representation transfer and reliability in Section 5, Table 1 and Appendix E.7. We distinguish contact ordering from concentration in Section 4.4 and window effects from rank stability in Section 4.6. We provide 12 active controls and one alignment reference in Appendix D.4. Selected AUROC means from Tables D.5/D.6 are:
These link correspondence and fusion to transfer setting. In cross-modality transfer, attention with fusion leads AUPRC at 0.882 versus Full RIVER's 0.831. Static transport leads balanced accuracy at 0.746 versus 0.734.
W2. Related benchmarks and electrode modeling
We expand Section 2.3, Adaptation and epilepsy benchmarks, with SzCORE and Quantifying the Generalization Gap (Dan et al., 2025, 2026). In Section 2.2, Transfer across heterogeneous EEG settings, we discuss LUNA, MVPFormer, NeuroCLUS, and Brant-X (Doner et al., 2025; Carzaniga et al., 2025; Zheng et al., 2026; Zhang et al., 2024). We distinguish their queries, parallel attention, clustering, and cross-signal transfer from RIVER's assignment/support separation in Appendices C.2/C.5.1.
W3. Missing models and cohort coverage
We add six models in Section 3.3, with details in Appendix B:
We use pre-trained REVE, LUNA, and MVPFormer (El Ouahidi et al., 2025; Doner et al., 2025; Carzaniga et al., 2025). We add extra iEEG dataset HUP in Section 3.2 and Table A.1, SOZ in Section 4.4, Figures 6 and H.4, and four further EEG tasks in Appendix D.5, Tables D.10/D.11. We also discuss NeuroCLUS in Section 2.2.
W4. Variable RIVER advantages
We separate initial scores, gains, and endpoints in Sections 4.1 to 4.6 and Section 5, Discussion. Figure 3 compares group means: RIVER's AUROC is 0.690 versus 0.653 for EEG foundation models. We provide individual results in Appendix F and controls favoring simpler alternatives in Tables D.5 to D.9. We read Table 1 with absolute trajectories, since a stronger start can leave less room for gain.
W5. Task-dependent modality conclusions
We added TUSZ to HUP alongside TUSZ to Epilepsy-iEEG (Figure F.4). Section 4.4 tests spatial correspondence. Simultaneous scalp/SEEG evidence supplies physiological context (Casale et al., 2022). Sections 3.1, 4.3, and 5 interpret these measured cohort/task combinations alongside population, sensing, and label differences.
W6. Pretraining overlap
We provide dataset-level overlap analysis and weight sources in Table B.3. We list benchmark datasets named in the reported corpora and mark parent-archive relationships separately. Zero-shot denotes no benchmark target adaptation.
W7. Imbalance and label construction
We specify 1:2 detection and 1:1 prediction sampling in Appendix A.2, retaining test prevalence in Tables A.6/A.7. We define positive-class Clip-F1 with balanced accuracy/AUPRC in Appendix E.1. We place the common 30min preictal interval and 5min horizon alongside dataset-specific negatives and prior protocols (Xiang et al., 2025).
Questions for Authors
Q1. Why do later gains diminish?
We discuss early target adjustment and later patient heterogeneity in Sections 4.3/4.5, Section 5, and Appendix E.7. Both detection contrasts have Holm p<0.00001. Target exposure adjusts representations and selection together, while calibration and discrimination recover differently. We provide complementary patient-fold results in Appendix H.5, Figure H.6.
Q2. Transported mass and information availability
A meaningful question! We now provide a matched contact-removal analysis in Appendix C.5.2, Figure C.1, using five-fold 12s TUSZ-to-HUP detection. We remove equal numbers of SOZ or non-SOZ contacts with checkpoint weights fixed. Repetitions and ictal clips are averaged within patients. The patient-paired summaries are:
We find greater mass changes after SOZ removal at matched contact counts. This supports selective responses to clinically relevant contact information, beyond classification scores. Log loss increases more after SOZ removal. We define this support in Appendix C.5.1.