- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes SCOPE: prototype-guided adaptation for EEG foundation models (EFMs) under limited labeled data. This work highlights issues in adapting EFMs: overconfident miscalibration, training collapse, and strong representation drift. SCOPE conjectures that these issues stem from limited labeled data and the full fine-tuning approach, which motivates them to 1) build a pipeline to create pseudo-labels for unlabeled data via limited labeled data, and 2) use these combined cohorts to selectively fine-tune the EEG foundation model. SCOPE shows superior performance to other adaptation approaches across several EFMs and datasets.
Critical strengths
-
Well-motivated idea of generating “pseudo” label data to adapt EEG foundation models, despite quality concerns
-
Interesting idea of limited adaptation with confidence metrics
Critical weaknesses
Concerns about the quality of pseudo-labels: SCOPE does in-loop label creation: trains a TPN classifier with a labeled dataset; uses TPN to create a prototype set of unlabeled data; updates the prototype set using the labeled dataset; then uses the prototype most similar to the TPN embedding. This creates a loop and over-relies on TFN prediction. If EFM is not performant for cross-subject prediction, what makes TFN predictions more reliable? In the paper, TFN predictions are not shown to be reliable (Ablation on the training strategy shows that training should anchor to true labeled data; Tab. 27: performance drops as we use more pseudo data). The authors need to work further on this, as it is the main contribution of the work.
Minor issue: The main paper lacks explanations of the scale-shift modulation design. The authors explain why limited adaptation is necessary but do not provide substantial reasons for their modulation design choice.
Questions for authors
Weaknesses above
- Accuracy of predicting pseudo-labels
- How does the number of prototypes affect performance? In Fig. 10, performance varies with the number of prototypes without a clear pattern; could the authors further investigate this? Does the current design need additional propulsive force to push in-class prototypes apart, which is missing from Eq. 5 and Eq. 6?
- I do not see a clear functional separation in the complex in-loop label creation. Further explanations could be useful to make it clear
6 comments
Response to Reviewer 3Krq [2/n]
1) Q1 Pseudo-label accuracy
In the original manuscript, we reported pseudo-label accuracy in Table 39 of Appendix I, where ρ (confidence) = 0.5 corresponds to the accuracy of the filtered pseudo-labels under our main experimental setting (30% labeled training data).
We additionally report pseudo-label accuracy on ISRUC with 20%, 30%, and 50% labeled training data to further evaluate its reliability across different label ratios.
| Labeled Training Data Ratio | Kappa | Weighted-F1 |
|---|---|---|
| 20% | 66.00 | 74.02 |
| 30% | 72.15 | 79.25 |
| 50% | 87.51 | 90.77 |
These results show the quality of our pseudo-labels improves as more labeled training data becomes available.
2) Q3 Functional separation in pseudo-label generation and selection
We provide a more detailed functional separation about the pseudo-label generation pipeline.
- Progressive construction of cohort-level structured prototypes and two complementary predictions.
- Task-prior learning: The TPN, regularized by ETF geometry, provides a global class prior and cluster-friendly embeddings, producing task-prior pseudo-labels.
- Prototype construction: Building upon the TPN embeddings, we further construct prototypes through Sinkhorn-constrained clustering and refine them with labeled supervision to capture local intra-class structure, producing complementary prototype-based pseudo-labels.
- Confidence-aware fusion and selection.
- Rather than directly using either prediction, we fuse their class probabilities to emphasize agreement. Pseudo-labels are retained only when the two predictions agree and the fused confidence exceeds a predefined threshold.
We would like to note that your description in the Weakness section accurately captures the first stage of our pipeline. Importantly, the second stage, confidence-aware fusion and selection, is also important for improving pseudo-label reliability.
Following your suggestion, we have added the following functional separation at the beginning of the External Structured Supervision Construction subsection in the Method section (lines 146-158):
To provide EFMs with reliable supervision, we adopt a two-stage pipeline consisting of three steps:
Stage 1: Cohort-level structured prototypes construction.
- Task-prior learning. Train a task-prior network (TPN) with inter-class geometric regularization to obtain cluster-friendly embeddings and task-prior pseudo-labels.
- Prototype construction. Cluster these embeddings and refine the resulting prototypes to obtain complementary prototype-based pseudo-labels.
Stage 2: Confidence-aware pseudo-label selection.
- Fusion and selection. Combine the two class-probability distributions, and retain pseudo-labels only when both predictions agree and the fused entropy-based confidence exceeds a threshold.
Stage 1 is detailed in Section 3.2.1, and Stage 2 in Section 3.2.2.
Response to Reviewer 3Krq [3/n]
3) W1.1 Reliability of external supervision
SCOPE does not directly rely on TPN predictions as reliable pseudo-labels. TPN provides an initial task-specific prior, which is further complemented by two levels of geometric regularization (ETF and Sinkhorn) and confidence-aware selection to construct reliable pseudo-labels. We provide quantitative analyses below to demonstrate the effectiveness of these strategies.
We analyze pseudo-label quality on ISRUC with 30% labeled training subjects to demonstrate how our three strategies improve supervision reliability beyond the initial TPN predictions.
| Strategy | Primary Function |
|---|---|
| 1. ETF regularization | Improves inter-class separability |
| 2. Sinkhorn-balanced clustering | Improves pseudo-label quality for minority classes |
| 3. Confidence-aware fusion and selection | Selects more reliable pseudo-labels |
-
ETF regularization improves inter-class separability. This improves task prior pseudo-label quality from 46.36 to 51.60 in Kappa and from 55.74 to 60.89 in weighted-F1.
-
Sinkhorn-balanced clustering preserves minority-class representation. Appendix I (Tables 39–40) validates this effect on the imbalanced ISRUC dataset. Without Sinkhorn, the minority N1 class nearly collapses (F1 = 2.09 at ρ = 0), whereas Sinkhorn substantially improves its representation (F1 = 41.57 at ρ = 0).
-
Confidence-aware fusion and selection improve pseudo-label reliability. We retain pseudo-labels only when TPN and prototype predictions agree, achieving 62.49 Kappa and 70.67 weighted-F1 before confidence thresholding (ρ = 0). Further filtering at ρ = 0.5 improves these metrics to 72.15 Kappa and 79.25 weighted-F1 (Appendix I, Table 39).
To further evaluate the effectiveness of our fusion strategy, we compare our multiplicative fusion with two simpler alternatives: average fusion and element-wise maximum fusion. Our multiplicative fusion emphasizes joint support from both prediction sources and penalizes conflicting evidence. We conduct this comparison on ISRUC with 30% labeled training subjects, using the same confidence threshold (ρ = 0.5), and evaluate downstream performance across five EFM backbones:
Average Max Ours Backbone Kappa Weighted-F1 Kappa Weighted-F1 Kappa Weighted-F1 pseudo-label accuracy 68.66 74.23 69.89 75.91 72.15 79.25 LaBraM 53.83±0.85 63.71±1.23 53.51±1.85 63.73±1.33 56.19±0.67 65.15±0.47 CBraMod 62.33±1.78 68.52±1.34 61.86±1.14 68.12±1.22 63.29±1.12 70.74±0.64 CSBrain 60.81±1.09 67.93±0.93 59.90±1.56 68.78±1.85 61.51±0.55 69.86±0.69 EEGMamba 61.17±1.65 70.21±1.70 61.92±1.79 69.80±1.52 63.07±0.89 71.41±0.79 CodeBrain 62.30±1.24 69.34±0.82 61.29±1.55 69.81±1.47 63.46±0.98 71.11±0.74
External supervision addresses unreliable EFM confidence, rather than inaccurate EFM predictions. The primary motivation for introducing the external supervision construction module is that EFM confidence may not reliably reflect prediction correctness, making direct self-training prone to error amplification.
- Evidence of unreliable EFM self-training:
- Figure 2(A) reveals overconfidence in EFM predictions
- Appendix C.2 shows direct EFM self-training yields inconsistent improvements over full fine-tuning
- Table 1 shows unstable performance of FixMatch and FineSSL, two general-purpose self-training baselines
- Reliability of our constructed supervision: As shown in Table 39 (Appendix I), increasing the confidence threshold (ρ) improves the quality of retained pseudo-labels. For example:
- ρ = 0: Kappa = 62.49, weighted-F1 = 70.67
- ρ = 0.5: Kappa = 72.15, weighted-F1 = 79.25
- ρ = 0.8: Kappa = 82.93, weighted-F1 = 88.51
Response to Reviewer 3Krq [1/n]
We sincerely thank you for recognizing our study’s well-motivated approach and interesting ideas.
TL;DR
①
Pseudo-labels(Weakness1 & Question1 & Question3; see our Responses 1-4)We would like to distinguish two challenges in pseudo-labeling:
With this distinction, we would like to emphasize two important clarifications:
To provide a more structured explanation, we organize our responses as follows:
②
Intra-class prototype design(Question2 ; see our Response 5)We provide further analyses of prototype numbers and additional ablations, with the following key findings:
③
Scale-and-shift modulation(Weakness2; see our Response 6)We added the rationale for using feature-wise modulation to incorporate prototype-derived structural supervision.
We provide further details and address each of your concerns point by point below.