- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 3(+1)
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the possible shortcomings of traditional category-supervised CLS and proposes a LaS approach that uses frozen text encoders, learnable neural/text projections, and bidirectional InfoNCE to align neural trials with category descriptions and classify by candidate text prototypes. The paper evaluates seven encoder architectures across motor imagery, SSVEP, and overt speech decoding, and explores candidate-subset reuse, adaptation, and conditional unseen-frequency decoding. It also analyzes repeated class descriptions in contrastive training and visualizes neural and word contributions to the same matching score.
Critical strengths
- I appreciated seeing the same class-description interface tested on motor imagery, SSVEP, and speech decoding, with both EEG and intracranial recordings and several encoder families. I also found the adaptation and candidate-set experiments informative when considering its possible uses beyond the main classification tasks.
- I think the task-dependent results and cases without improvement were informative, and I appreciated that the authors discuss them alongside the positive findings. This gives the work value as an investigation of language-guided decoding and helps readers understand its scope.
Critical weaknesses
- I think the methodological advance and the contribution of text semantics need clearer support. NICE++ [1] already uses a similar language-guided neural alignment based on infoNCE loss on image classification tasks, where I think the novelty is not prominent enough due to seemingly high method overlap. In addition, EmotionCLIP and EEG-PRIME mentioned in related work also use class-text matching. A comparison with an applicable text-prototype approach would help establish LaS's contribution. More critically, the paper lacks random or learned class prototypes under the same readout and loss. The nonsignificant closed-set P0/P1 differences make these controls particularly relevant.
- I cannot yet separate the contribution of language supervision from the accompanying decoder changes. For example, MI EEGNet changes its readout from 48 to 400 dimensions, CNN-BiGRU changes how it reads the bidirectional sequence, and other models change pooling, dropout, or trial aggregation. And parameter counts also change substantially in either direction. These choices can affect information retention and regularization. I would find the language-supervision claim more convincing with matched readouts, training settings, and effective parameter budgets. [1] Song, Yonghao, et al. "Recognizing natural images from eeg with language-guided contrastive learning." IEEE Transactions on Neural Networks and Learning Systems 36.9 (2025): 15896-15910.
Questions for authors
- Have authors also evaluated LaS on a Du-IN MAE-pretrained encoder, alongside the corresponding pretrained CLS baseline? This would help establish whether the benefit carries over to the strongest available Du-IN setting.
- Could authors provide experimental results on multiple random seed pairs for CLS/LaS? It would help to see the variation in gains across runs.
- In Figure 3, could authors also restrict the predictions of an existing four-class CLS checkpoint to the same candidate subsets? The current comparison uses subset-trained CLS models. This additional result would clarify how much of the reuse benefit comes from the text interface versus training on the larger class set.
8 comments
Response to g8g8 (2/6): W1: Fixed-random and unseen-frequency results
Contribution of text semantics and fixed random-vector controls
More critically, the paper lacks random or learned class prototypes under the same readout and loss.
We understand your concern: the role of class representations needs to be tested directly under the same architecture and training objective.
The differences in closed-set mean accuracy between fixed random vectors and text are below 1 pp in all three paradigms.
For each class, we independently generate a standard Gaussian vector with the same dimensionality as the language features, apply L2 normalization, and then fix it: . We replace only the precomputed language features and retain the trainable projection ; the rest of the neural model, bidirectional InfoNCE loss, temperature, data splits, and training settings remain the same as in the language control. The vectors before projection are fixed, while the projected prototypes are updated during training.
| Task and encoder | Condition | Real text (%) | Fixed random vectors (%) | Text−random (pp) |
|---|---|---|---|---|
| MI, EEG-Conformer | Four classes, 1.0 s | 41.94 | 41.39 | +0.55 |
| SSVEP, MTSNet | 40 classes, 1.0 s, RW→IW | 66.79 | 67.36 | −0.57 |
| Speech, Du-IN | 61 classes, 3.0 s | 60.58 | 59.79 | +0.79 |
For each paradigm, the fixed-random control reports the mean accuracy of experiments with three independent sets of random class vectors. MI uses the newly established five-fold protocol with held-out validation; 1.0 s is the model input window length, and trial predictions are obtained by averaging the window scores within each trial. SSVEP uses the corresponding conditions in Table 1 and aggregates results with equal weight for each of the 35 subjects.
Both text vectors and random vectors can provide reference coordinates for class alignment. The pairwise inner products of independently normalized Gaussian vectors have zero mean and an expected squared value of , so these vectors are usually nearly orthogonal in high dimensions. This gives different classes distinct fixed references.
The trainable projection further allows the model to adjust these references. Let contain the random vectors; when , has full row rank almost surely. For any target prototype matrix whose rows are unit vectors, an unconstrained linear projection can be chosen as
\quad\Longrightarrow\quad RW_t^{(r)}=Q.$$ Under the full-row-rank condition above, the trainable projection can map fixed random vectors to given seen-class prototypes, providing a geometric explanation for the similar closed-set results. The compensation for seen-class permutations analyzed in Appendix C.1 also uses this expressive capacity of the projection. **With the model fixed, introducing frequency adjacency only for unseen classes increases ZSL accuracy from 17.92% to 94.52%.** We compare independent random class vectors, real text, and a frequency-adjacency construction using MTSNet with 1-second EEG inputs. The three conditions use the same eight class splits and evaluation trials, with predictions among five unseen frequency classes in each split and a chance level of 20%. The 40 frequencies range from 8.0 to 15.8 Hz in increments of 0.2 Hz; the held-out frequencies in split j are $8.0+0.2(j-1)+1.6k$ Hz, where $j=1,\ldots,8$ and $k=0,\ldots,4$. The unseen frequencies in each split have adjacent seen frequencies available for the construction. Within each class split, the two random-representation conditions reuse the same trained checkpoint, keeping both the neural branch and the projection fixed and changing only the input vectors for unseen classes at inference. The adjacency-based construction is performed before projection: for an interior unseen frequency, we average the initial unit random vectors of its neighboring seen classes on either side and normalize the result again. For example, 8.2 Hz uses the vectors for 8.0 and 8.4 Hz. | Class representation | 1.0 s ZSL accuracy (%) | | -------------------------- | -------------------: | | Independent random class vectors | 17.92 ± 8.66 | | Real text | 90.89 ± 1.43 | | Random vectors constructed using frequency adjacency | 94.52 ± 0.83 | The results are the mean accuracy and sample SD across the eight class splits, covering 35 subjects. The endpoints 8.0 and 15.8 Hz use the vectors of the nearest seen frequencies, 8.2 and 15.6 Hz, respectively.Response to g8g8 (3/6): W1: Frequency relationships and stimulus control
Let , and let and be the initial unit random vectors of adjacent seen classes. For an interior unseen class , under a linear projection and provided that the vectors being normalized are nonzero,
t_u=\nu(r_uW_t) =\nu(r_-W_t+r_+W_t).$$ The adjacency construction connects the projected direction of an unseen class to the trained directions of adjacent classes. Independent random vectors provide distinct class references; the adjacency construction further encodes local frequency relationships in the class representations. When neural representations vary smoothly with frequency locally and are compatible with this interpolation relationship, the model can use the matching relationships learned from seen classes to recognize unseen frequencies. Section 1 notes that one-hot targets identify classes without explicitly encoding class attributes or semantic relationships. Independently sampled fixed random inputs likewise do not encode these relationships in advance. The frequency-adjacency control adds relationships to the random inputs by construction, improving unseen-frequency accuracy by 76.61 pp with the same random-vector model. Text achieves 90.89% through frequency descriptions; together with the P0/P1 permutation results, these findings support the role of class relationships in this prediction task. Frequencies have an explicit numerical order, making adjacency rules straightforward to specify; more abstract class relationships may lack an equally direct vector construction. Natural language can describe class attributes and relationships directly, and a pretrained text encoder converts these descriptions into representations, allowing classes in MI, SSVEP, and Speech to use the same descriptive approach. With the neural input held fixed, word attribution allows researchers to inspect each word’s contribution to the matching score for a specified class. **Comparison of text and stimulus representations** We also compared supervision that directly encodes stimulus frequency and phase. The stimulus-encoding and text conditions use the same MTSNet neural encoder, 1-second EEG inputs, eight class splits, and evaluation trials. The stimulus branch uses ModernTCN ([Luo and Wang, 2024](https://openreview.net/forum?id=vpJMJerXHU)) to encode a 1-second sinusoidal reference and is trained jointly with MTSNet. | Method | 1.0 s accuracy (%) | Relative to FBCCA (pp) | | ------------------------- | ---------------: | --------------: | | FBCCA | 88.83 ± 1.22 | — | | MTSNet + ModernTCN stimulus encoding | 82.57 ± 3.52 | −6.26 | | MTSNet + text representations (LaS) | 90.89 ± 1.43 | +2.06 | The results are the mean accuracy and sample SD across the eight class splits. Text achieves higher unseen-frequency accuracy in this comparison. Text features can be precomputed with the frozen text encoder, and training updates only the linear projection on the text side; the stimulus approach requires additional trainable parameters for ModernTCN and joint training of the stimulus encoder.
Response to g8g8 (1/6): W1: Methodological contribution and prior methods
Thank you for taking the time to review our paper and for recognizing the value of evaluating the same class-description interface across paradigms, recording modalities, and encoders.
The additional results show that LaS achieves MI four-class accuracy close to LEAF-FT with approximately 1/55 as many trainable parameters, and that all four Speech encoders retain positive mean gains across five runs against the updated CLS controls. The random-vector and frequency-adjacency experiments further address how class representations support closed-set fitting and extension to unseen frequencies. Below, we address your concerns and questions point by point using these additional experiments and analyses. The MI class labels are: 1=thumb, 2=index finger, 3=middle finger, and 4=little finger.
Weakness 1. Methodological contribution, existing methods, and the role of class representations
Text provides general-purpose semantic representations for classes across neural decoding paradigms, allowing class attributes and relationships to inform supervision and prediction. LaS generates class representations with a pretrained text encoder and uses the same descriptive approach in independently trained models for MI, SSVEP, and Speech.
Relationship to existing methods and direct comparison
Thank you for stressing the importance of a direct comparison with existing text-prototype methods; we have added fine-tuning results for LEAF (Jiang et al., 2026) using the same target training set and validation and evaluation splits.
LaS replaces the classification head with neural projection and text-prototype matching, using the same form of language supervision with classical encoders across multiple paradigms. NICE++ (Song et al., 2025) uses language to improve image decoding, EmotionCLIP (Yan et al., 2026) targets EEG emotion recognition, and EEG-PRIME (Zhang et al., 2026) uses pretraining and multi-level conditioning. We examine this approach through paired evaluations of CLS and LaS and further analyze how class descriptions affect learning and prediction.
We distinguish the class-discrimination role of N2T from the within-class matching role of T2N along text-prototype directions and analyze the loss terms arising from repeated descriptions. With the input on the other side held fixed, dual-sided attribution relates the contributions of neural channels, spatial locations, time intervals, and prompt words to the same specified matching score, allowing researchers to inspect the basis of matching from both input sides.
The new direct comparison shows that LaS achieves MI four-class accuracy close to LEAF-FT with approximately 1/55 as many trainable parameters. LEAF-FT starts from the official instruction-trained weights and fine-tunes the EEG encoder and Q-Former on the same target training set as LaS, with fixed text prototypes and the same validation and evaluation splits. Both use the five-fold protocol with held-out validation and trial-level evaluation adopted here, and LEAF-FT follows the official EEG encoder + Q-Former fine-tuning procedure.
Accuracy is reported in %, as the mean across 21 subjects and the between-subject sample SD. Parameter counts include the parameters being fine-tuned or trained and exclude the frozen text encoder.
The neural encoder in LaS is trained from scratch, with its four-class mean differing from LEAF-FT by only 0.05 pp and its means on the four subtasks higher by 0.38–1.98 pp.
When each class has m identical text entries in a balanced batch, N2T decomposes into class cross-entropy and the constant logm; ablations of direction, loss objective, and batch further examine how these training effects relate to decoding performance.