- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SemLoop, a closed-loop semantic adaptation framework for EEG foundation models that aims to bridge the gap between task-relevant semantic knowledge and its actual utility to downstream models. SemLoop consists of three components: (1) EEG-Grounded Semantic Alignment (EGSA), which aligns language-derived neurophysiological descriptions with EEG representations through contrastive learning and semantic interaction; (2) Model-Validated Semantic Refinement (MVSR), which employs LLM-based evaluator and planner agents to iteratively revise semantic descriptors based on descriptor-level utility and validation performance; and (3) Experience-Internalized Semantic Planning (EISP), which fine-tunes the planner using successful refinement experiences. The framework is evaluated on nine EEG benchmarks spanning six task categories, with additional experiments involving three EEG foundation model backbones. The authors report improvements over full fine-tuning and predefined semantic-guided approaches, together with ablation studies, refinement analyses, and cross-task experience-transfer experiments.
Critical strengths
- An interesting formulation of model-validated semantic adaptation for EEG foundation models. The paper addresses a meaningful limitation of existing semantic-guided EEG adaptation methods: task-relevant semantic descriptions do not necessarily provide useful information to a particular target model. Using descriptor-level utility and validation feedback to refine semantic guidance offers a practical approach to adapting task knowledge to model-specific requirements. The integration of EEG–text alignment with iterative semantic refinement provides a coherent framework for investigating this problem.
- Broad experimental coverage and reasonably comprehensive component analyses. The evaluation covers nine EEG datasets across six task categories and examines the method with multiple foundation model backbones. The authors also compare several refinement strategies, including unguided refinement, descriptor-utility feedback, performance feedback, and their combination. Additional experiments on experience internalization and cross-task transfer offer useful evidence regarding the behavior of the proposed framework. The generally positive results across diverse tasks suggest that the approach has practical potential.
Critical weaknesses
1.Limited methodological novelty and insufficient evidence for the contribution of EEG-specific semantic reasoning.
The methodological novelty of SemLoop appears relatively limited, as its three main components primarily integrate established techniques rather than introduce fundamentally new learning mechanisms. EGSA combines conventional EEG–text contrastive alignment with attention-based semantic fusion, MVSR employs LLM-based evaluator and planner agents for iterative optimization using model feedback, and EISP applies gain-weighted supervised fine-tuning to successful refinement trajectories. Although integrating these components into a closed-loop EEG adaptation framework is potentially useful, its distinction from existing feedback-driven LLM optimization paradigms remains insufficiently established.
More importantly, the experiments do not adequately isolate the benefits of EEG-specific semantic reasoning from those of generic validation-guided optimization. Although the authors include unguided refinement, utility filtering, and utility ranking baselines, stronger comparisons with budget-matched non-LLM search methods and semantically unrelated or randomly constructed descriptor banks are still missing. Consequently, it remains unclear to what extent the reported gains arise from meaningful neurophysiological knowledge rather than additional optimization over candidate semantic configurations.
Furthermore, the semantic grounding procedure is insufficiently specified. Section 3.2 states that EEG samples are associated with neurophysiological descriptions based on their characteristics. Although Appendix H provides examples of initial and refined descriptors, the paper does not clearly explain how sample-level positive EEG–text pairs are constructed or whether class labels, explicitly computed EEG features, or other supervision are involved. Since these associations directly determine the contrastive alignment objective, this omission affects the validity and reproducibility of the proposed semantic grounding mechanism.
2.Insufficient statistical evidence and robustness analysis for validation-driven semantic refinement.
Section 4.2 reports a two-sided Wilcoxon signed-rank test with p < 0.001 on paired results across nine datasets. However, if each dataset contributes one independent paired observation, the minimum attainable exact two-sided p-value is 0.00390625, even when all nine differences have the same sign. The reported significance level therefore requires clarification regarding the actual statistical units, aggregation procedure, and treatment of different metrics and random seeds.
In addition, SemLoop repeatedly evaluates candidate semantic configurations using validation feedback, selects the best-performing candidates across refinement rounds, and constructs successful experiences according to validation improvements. Although validation-based optimization is legitimate and the paper reports separate test sets, repeated adaptive selection may lead to overfitting to a particular validation partition, especially for datasets with relatively few validation subjects. The robustness of the resulting semantic configurations across alternative subject-level validation partitions is not adequately established.
Finally, the incremental improvements attributable to EISP are modest on several benchmarks. For example, Table 3 shows small gains over Full MVSR on BCIC-2A and FACED, while Table 5 reports only minor improvements over the Base Planner on some cross-task comparisons. Although these results are encouraging, the paper does not provide sufficient paired statistical evidence to establish the reliability of these incremental gains. This weakens the evidence that experience internalization consistently provides a substantial benefit beyond the underlying refinement procedure.
Questions for authors
Q1. How are sample-level EEG–semantic correspondences constructed for EGSA? Section 3.2 states that each EEG sample is associated with EEG-observable neurophysiological descriptions based on its characteristics. Could the authors specify the exact procedure for constructing these positive EEG–text pairs? In particular, are the descriptions derived from explicitly computed EEG features, class labels, expert annotations, or another mechanism? Although Appendix H provides examples of descriptor sets, it does not clarify the sample-level association procedure. Could the authors also confirm how training, validation, and test data are separated during semantic alignment?
Q2. Could the authors clarify the statistical testing procedure used to obtain p < 0.001? Section 4.2 states that a two-sided Wilcoxon signed-rank test was conducted on paired results across nine datasets. If each dataset contributes one independent paired observation, an exact two-sided p-value below 0.001 is not attainable. Could the authors specify the actual number of paired observations, how different metrics and random seeds were aggregated, which baselines were compared, and whether corrections for multiple comparisons were applied? Providing the individual paired differences and corresponding p-values would help verify the reported significance claims.
Q3. How is validation-set overfitting controlled during iterative semantic refinement? SemLoop repeatedly evaluates and selects semantic configurations using validation feedback. Could the authors clarify whether descriptor-level utility estimation, candidate selection, and successful experience construction all rely on the same validation partition? More importantly, do semantic configurations selected using one validation split retain their advantages under alternative subject-level splits or independently constructed validation partitions? Such evidence would help establish the robustness of the proposed feedback-driven refinement procedure.
Q4. What is the isolated contribution of LLM-based semantic reasoning and experience internalization? Although the authors evaluate utility filtering, utility ranking, and several refinement variants, could they provide stronger comparisons against budget-matched non-LLM search strategies and size-matched random or semantically unrelated descriptor banks? For EISP, the existing ablation and leave-one-task-out results are encouraging, but several improvements are small. Could the authors further isolate the contribution of gain-weighted planner fine-tuning, for example by comparing it with unweighted experience fine-tuning, and provide stronger paired statistical evidence for its benefits on held-out tasks?
7 comments
Response to Reviewer dvDp (2/5)
Q2 & W2: Statistical Testing
Response:
Overall and Per-dataset Statistical Testing. In our original analysis, we treated the nine datasets and three evaluation metrics as 27 paired observations when comparing SemLoop with the best-performing baseline (underlined in Table 1) on each dataset. We acknowledge that this aggregation does not account for the dependence among metrics computed on the same dataset. In the revised analysis, we increase the number of random seeds from five to ten and conduct statistical comparisons against the baselines. Specifically, we use Balanced Accuracy as the primary metric. For each dataset, we conduct a two-sided exact Wilcoxon signed-rank test on the ten paired results obtained from the ten random seeds. For overall comparisons, we average the results over ten seeds for each dataset and perform two-sided exact Wilcoxon signed-rank tests on the nine paired dataset-level differences. Holm–Bonferroni correction is applied jointly to the nine per-dataset tests and the overall test within each baseline comparison. As shown in Table R4, the results demonstrate statistically significant improvements over most baselines on most individual datasets, as well as significant overall improvements across the nine datasets after correction.
Table R4:Per-dataset and overall statistical comparison of SemLoop against the representative baselines across nine EEG datasets.
| Dataset | EEG-CLIP | USEA | BLPM | CBraMod | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gain (Bal.Acc.) | p | Adj. p | Gain (Bal.Acc.) | p | Adj. p | Gain (Bal.Acc.) | p | Adj. p | Gain (Bal.Acc.) | p | Adj. p | |
| BCIC-2A | +5.16 | 0.001953 | 0.019531 | +2.35 | 0.001953 | 0.019531 | +2.63 | 0.003906 | 0.031250 | +2.70 | 0.001953 | 0.019531 |
| BCIC-2B | +1.68 | 0.001953 | 0.019531 | +2.01 | 0.003906 | 0.019531 | +1.45 | 0.005859 | 0.035156 | +1.19 | 0.005859 | 0.031250 |
| SHU-MI | +1.56 | 0.001953 | 0.019531 | +1.77 | 0.019531 | 0.039062 | +1.17 | 0.009766 | 0.039062 | +1.53 | 0.003906 | 0.031250 |
| FACED | +4.59 | 0.001953 | 0.019531 | +4.82 | 0.001953 | 0.019531 | +1.43 | 0.001953 | 0.019531 | +0.97 | 0.001953 | 0.019531 |
| SEED-V | +0.73 | 0.001953 | 0.019531 | +0.93 | 0.027344 | 0.039062 | +0.79 | 0.019531 | 0.039062 | +0.77 | 0.027344 | 0.031250 |
| Mumtaz2016 | +1.24 | 0.001953 | 0.019531 | +0.65 | 0.001953 | 0.019531 | +0.57 | 0.013672 | 0.039062 | +0.61 | 0.009766 | 0.031250 |
| MentalArithmetic | +5.20 | 0.009766 | 0.019531 | +7.54 | 0.001953 | 0.019531 | +6.01 | 0.005859 | 0.035156 | +5.96 | 0.009766 | 0.031250 |
| BCIC2020-3 | +3.70 | 0.001953 | 0.019531 | +2.96 | 0.011719 | 0.035156 | +1.04 | 0.011719 | 0.039062 | +0.81 | 0.003906 | 0.031250 |
| TUEV | +2.42 | 0.003906 | 0.019531 | +12.16 | 0.001953 | 0.019531 | +2.52 | 0.001953 | 0.019531 | +2.39 | 0.003906 | 0.031250 |
| Overall | +2.92 | 0.003906 | 0.019531 | +3.91 | 0.003906 | 0.019531 | +1.96 | 0.003906 | 0.031250 | +1.88 | 0.003906 | 0.031250 |
| Dataset | EEGPT | BrainOmni | LaBraM | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Gain (Bal.Acc.) | p | Adj. p | Gain (Bal.Acc.) | p | Adj. p | Gain (Bal.Acc.) | p | Adj. p | |
| BCIC-2A | +13.45 | 0.001953 | 0.019531 | +10.75 | 0.001953 | 0.019531 | +4.41 | 0.003906 | 0.019531 |
| BCIC-2B | +4.16 | 0.001953 | 0.019531 | +5.00 | 0.001953 | 0.019531 | +3.07 | 0.001953 | 0.019531 |
| SHU-MI | +9.40 | 0.001953 | 0.019531 | +1.69 | 0.048828 | 0.048828 | +1.25 | 0.001953 | 0.019531 |
| FACED | +23.52 | 0.001953 | 0.019531 | +35.04 | 0.001953 | 0.019531 | +2.24 | 0.001953 | 0.019531 |
| SEED-V | +15.58 | 0.001953 | 0.019531 | +15.39 | 0.001953 | 0.019531 | +0.49 | 0.001953 | 0.019531 |
| Mumtaz2016 | -0.37 | 0.695312 | 0.695312 | +8.12 | 0.001953 | 0.019531 | -0.18 | 0.679688 | 0.679688 |
| MentalArithmetic | +12.60 | 0.001953 | 0.019531 | +4.23 | 0.009766 | 0.019531 | +2.82 | 0.001953 | 0.019531 |
| BCIC2020-3 | +11.16 | 0.001953 | 0.019531 | +5.78 | 0.001953 | 0.019531 | +7.20 | 0.001953 | 0.019531 |
| TUEV | +1.56 | 0.001953 | 0.019531 | +33.32 | 0.001953 | 0.019531 | +3.84 | 0.001953 | 0.019531 |
| Overall | +10.12 | 0.007812 | 0.019531 | +13.26 | 0.003906 | 0.019531 | +2.79 | 0.007812 | 0.019531 |
Response to Reviewer dvDp (3/5)
Q3 & W2: Validation-set Overfitting
Response:
Alternative Subject-level Validation Splits. To assess the sensitivity of validation-driven refinement to validation-subject composition, we divide the original validation set into three subject-disjoint subsets and independently perform MVSR using each subset for feedback and candidate selection. The methods are evaluated on the same held-out test set. As shown in Table R3, we compare the performance across the three alternative validation splits with that obtained using the original validation set. The results show limited performance variations across splits for MVSR, demonstrating the robustness of validation-driven semantic refinement to different validation-subject compositions.
Table R3. Robustness of MVSR to alternative subject-level validation splits.
| Method | Validation Setting | SHU-MI | FACED | TUEV | |||
|---|---|---|---|---|---|---|---|
| Bal.Acc. | AUROC | Bal.Acc. | Kappa | Bal.Acc. | Kappa | ||
| MVSR | Original | 61.23 ± 2.05 | 67.24 ± 2.34 | 57.79 ± 0.72 | 52.14 ± 0.82 | 61.98 ± 1.96 | 60.25 ± 2.68 |
| Split 1 | 61.14 ± 1.91 | 67.19 ± 2.16 | 57.68 ± 0.67 | 51.98 ± 0.79 | 62.12 ± 1.87 | 60.31 ± 2.53 | |
| Split 2 | 61.27 ± 2.02 | 67.26 ± 2.31 | 57.85 ± 0.76 | 52.19 ± 0.84 | 61.84 ± 1.98 | 60.07 ± 2.72 | |
| Split 3 | 61.21 ± 1.98 | 67.23 ± 2.29 | 57.74 ± 0.62 | 52.12 ± 0.71 | 62.01 ± 1.94 | 60.29 ± 2.61 |
Q4 & W1 & W2: LLM-based Semantic Reasoning and Experience Internalization (1/3)
Response:
We thank the reviewer for this insightful suggestion.
1. Contribution of LLM-based semantic reasoning.
(1) Budget-matched Non-LLM Search Comparison. We compare MVSR with Evolutionary Search under the same search budget. Evolutionary Search generates candidates through descriptor mutation and selects them based on validation performance, without LLM-based semantic reasoning. As shown in Table R1, MVSR achieves better performance, demonstrating the benefit of LLM-based semantic reasoning beyond conventional search.
Table R1. Performance comparison of MVSR with budget-matched non-LLM search.
| Method | SHU-MI | FACED | TUEV | |||
|---|---|---|---|---|---|---|
| Bal.Acc. | AUROC | Bal.Acc. | Kappa | Bal.Acc. | Kappa | |
| Evolutionary Search | 60.57 ± 1.50 | 66.38 ± 1.93 | 56.84 ± 0.65 | 51.19 ± 0.74 | 61.37 ± 1.85 | 59.35 ± 1.43 |
| MVSR | 61.23 ± 2.05 | 67.24 ± 2.34 | 57.79 ± 0.72 | 52.14 ± 0.82 | 61.98 ± 1.96 | 60.25 ± 2.68 |
(2) Size-matched Random/Unrelated Descriptor Banks. We compare two additional baselines: Random EEG Search and Random Unrelated Search. Random EEG Search iteratively replaces descriptors with randomly sampled EEG-related descriptors, whereas Random Unrelated Search iteratively replaces descriptors with randomly sampled task-unrelated semantic descriptors from a predefined pool. Both baselines follow the same protocol as MVSR, with matched descriptor-bank sizes. As shown in Table R2, MVSR achieves the best performance, followed by Random EEG Search and Random Unrelated Search, demonstrating the importance of EEG-related semantic knowledge and task-aware semantic reasoning.
Table R2. Performance comparison of MVSR with size-matched random EEG-related and task-unrelated descriptor search.
| Method | SHU-MI | FACED | TUEV | |||
|---|---|---|---|---|---|---|
| Bal.Acc. | AUROC | Bal.Acc. | Kappa | Bal.Acc. | Kappa | |
| Random Unrelated Search | 60.35 ± 1.64 | 65.89 ± 1.81 | 56.62 ± 0.76 | 50.93 ± 0.80 | 61.05 ± 2.12 | 58.92 ± 1.24 |
| Random EEG Search | 60.53 ± 1.59 | 66.10 ± 1.68 | 56.75 ± 0.69 | 51.08 ± 0.84 | 61.23 ± 1.93 | 59.14 ± 1.36 |
| MVSR | 61.23 ± 2.05 | 67.24 ± 2.34 | 57.79 ± 0.72 | 52.14 ± 0.82 | 61.98 ± 1.96 | 60.25 ± 2.68 |
Response to Reviewer dvDp (1/5)
We sincerely thank the reviewer for their helpful suggestions.
W1: Methodological Novelty
Response:
Methodological novelty. Our contribution lies not in introducing entirely new individual components, but in establishing a new paradigm for semantic adaptation of EEG foundation models. Existing semantic-guided approaches primarily follow a one-way knowledge injection paradigm, incorporating predefined task-relevant semantics into EEG models without explicitly validating whether these semantics are useful to the target model. As illustrated in Fig. 1, this leaves a gap between semantic relevance and model utility. SemLoop addresses this gap through a closed-loop Ground–Validate–Refine–Internalize paradigm, in which model feedback is used to evaluate and iteratively refine task-relevant semantic knowledge, while successful refinement experiences are internalized for subsequent planning. This offers a new perspective on semantic-guided EEG foundation model adaptation.
Q1 & W1: Sample-level Descriptions
Response:
Training-time construction of Si. For each training EEG sample Xi, we construct its associated semantic set Si using features extracted directly from the EEG signal, without using class labels or external annotations. Specifically, we extract 73 EEG features, including band power, spatial asymmetry, and spectral entropy. Each feature is standardized using its mean and standard deviation computed exclusively from the training set. For each sample, we retain features with absolute z-scores of at least 0.35 and select up to eight with the largest absolute z-scores. If no feature meets this threshold, we retain the highest-ranked feature. Predefined feature-to-text templates convert the selected features into EEG-observable descriptions using the format
[increased/decreased] [EEG feature], forming Si. Each sample Xi is paired with all descriptions in its associated Si, yielding positive EEG–text pairs for training the Semantic Aligner gϕ with a bidirectional multi-positive contrastive objective. Importantly, class labels are not used in feature selection, standardization, descriptor generation, or positive-pair construction.Test-time usage. During training, the sample-level semantic sets Si serve as auxiliary supervision for learning the alignment between EEG representations and EEG-derived neurophysiological descriptions. Once this alignment is learned, constructing sample-level semantic sets is no longer necessary at test time. Instead, the trained Semantic Aligner maps the task-level descriptor set T, which is shared across samples within a target task, into a grounded semantic bank B for downstream prediction. Importantly, T contains no ground-truth class labels or label-derived information, and no test-sample labels are required throughout this process.
For completeness, the training protocol is summarized in our response to the Area Chair's Q4, clarifying the parameter updates across different stages of SemLoop.