acceptodds
Under review as a conference paper at ICLR 2027

AGraSS: Understanding and Shaping Gradient Spectra for EEG Learning

Abstract

EEG signals exhibit substantial complexity, inter-subject heterogeneity, and are susceptible to noise, making it challenging to learn meaningful task-relevant representations. While the mainstream methods use neural networks trained through gradient-based optimization, the information carried by different gradient directions during EEG learning remains insufficiently understood. To this end, we investigate the spectral structure of task gradients in EEG models and propose a new optimizer tailored to EEG learning. Our analysis reveals pronounced spectral imbalance of gradients during training and finds a strong association between the leading spectral head and subject-related variation, as well as a low estimated gradient signal-to-noise ratio in the spectral tail. Theoretical analyses with a latent subject-variation model explain such gradient structure of the spectral head. Motivated by this analysis, we propose AGraSS (daptive dient pectral caling), an optimizer that combines automatic spectral-head detection with soft spectral shrinkage. AGraSS adaptively rescales the spectral head with additional attenuation on the pre-detected subject-related directions, moderating dominant update directions. Experiments on six EEG datasets with both expert models and pretrained EEG foundation models demonstrate the superiority of AGraSS, outperforming mainstream optimizers such as AdamW and Muon with improved performance. These findings highlight EEG-aware optimizer design as an important direction for improving EEG learning.

Primary area:
applications to neuroscience & cognitive science
Keywords:
EEG, gradient spectral analysis

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_VVhTOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper characterizes the singular value spectra of task gradients in EEG models. It reports that gradient energy concentrates in a few leading modes throughout training, that these leading modes carry most of the subject related gradient signal, which the authors formalize with a theorem under a latent additive subject variation model, and that tail modes have low gradient signal to noise ratio. Based on this analysis the paper proposes AGraSS, a Muon style optimizer that selects the gradient spectral head ratio once before training, rescales the selected head using the residual spectral scale, and applies additional attenuation to leading modes exhibiting consistent subject-related conflict. Across six public EEG datasets, four expert models, and three EEG foundation models, with cross-subject splits and five seeds, AGraSS outperforms AdamW, SGD, and Muon in the large majority of settings, and ablations in two settings support adaptive head selection and subject-aware shrinkage.

Critical strengths
  1. The pipeline from empirical gradient spectral analysis through a stylized theory to a concrete optimizer is novel and coherent. The idea of an EEG aware optimizer is new and connects naturally to current research on matrix preconditioners such as Muon.
  2. The evaluation is broad and disciplined for an optimizer paper. It spans 28 model and dataset combinations over seven architectures with cross-subject splits, a shared hyperparameter search, and validation based model selection, and AGraSS is best or tied best in nearly all of them according to Tables 1 and 2.
Critical weaknesses

The superiority claim is statistically under-supported. Gains over Muon are often within one standard deviation, for example 0.702±0.022 versus 0.701±0.035 on CBraMod with TUSL. Several settings favor the baselines, for example LaBraM on MentalArithmetic with 0.595 against 0.614 for AdamW. No significance tests are reported, and in several settings the fixed test splits contain only two to five subjects, for example two subjects for BCIC-IV-2a and four for Mental Arithmetic in Appendix A.2, so the small margins may not be robust.

Questions for authors
  1. Please report paired significance tests across seeds, and ideally across repeated subject splits, for AGraSS against Muon and against AdamW. In how many of the 28 settings is the improvement significant?
  2. In Table 3, the row with head shrinkage set to None and subject-aware shrinkage disabled reproduces the AdamW entries of Tables 1 and 2 digit for digit, including the standard deviations. Did this row simply reuse the AdamW runs, and which update rule does the ablation build on? For attribution, please add a variant with momentum and RMS normalization but no spectral shaping, and a Muon-style variant combined with subject-aware shrinkage, so that the contribution of each component can be isolated.
  3. How sensitive is AGraSS to the one-time calibration? How do the detected head ratio, the head size K_c, and the attenuation coefficients evolve during training, and how much does performance vary when calibration is run at different epochs or seeds? What is the number of power iterations used per step? Relatedly, what microbatch size and how many microbatches are used for the SNR analysis in Section 2.3? Since the estimated fluctuation across microbatches scales inversely with the batch size, the SNR profile in Fig. 3, and therefore the noise sensitivity of the tail, depends on this choice.
  4. Do encoders trained with AGraSS actually become more subject invariant, for example in terms of subject decoding accuracy from frozen features, than under Muon or AdamW? This would directly validate the mechanism.
  5. The related work cites objective and data level methods that encourage patient invariant representations, such as patient-adversarial learning (Zhang et al., 2024) and distributionally robust decoding (Duan et al., 2023), and states that optimizer level design is potentially complementary to these strategies. Can you substantiate this with an experiment, for example AGraSS combined with a patient-adversarial objective versus each alone, and can you include a within-subject condition to show that the gains of AGraSS are specific to the cross-subject setting that motivates the method?
  6. In Fig. 2 the largest absolute projections reach only about 0.08 to 0.30 after normalization by the Frobenius norm of the surrogate. Can you provide a null baseline for this concentration, for example the projection profile onto random orthogonal directions of matched scale, onto the singular basis of gradients computed with shuffled subject labels, or between two independent estimates of the task gradient itself? This would show whether the concentration of subject related signal in the spectral head exceeds what high-dimensional random overlap would produce, and would substantiate the strong association claim.
2 comments
AuthorsOpenReview ↗

Response to Reviewer VVhT (1/2)

We sincerely thank the reviewer for appreciating our novel idea and broad evaluation, and providing insightful suggestions. We carefully address your concerns as follows.

  1. Statistical reliability.

To evaluate the statistical evidence of broad performance benefits, we perform paired statistical significance analysis on the mean balanced accuracy across all 28 settings, and the results show that AGraSS > AdamW is statistically significant (p<0.001), AGraSS > Muon is statistically significant (p<0.001), while the difference between AdamW and Muon is not significant (p=0.84). So broadly, AGraSS show robust performance gains over other optimizers. All of our experiments for data/subject split follow the common setting in the EEG literature, and the limited number of subjects is the intrinsic property of EEG datasets. Therefore, it is not common to expect a statistical significance for every setting, and instead, we consider the broad evaluation covering all 28 settings to demonstrate the overall statistical significance of the advantage of our method over AdamW and Muon. The results indicate that the gains are relatively robust in a broad range.

  1. Clarification of the ablation baseline and additional controls.

We thank the reviewer for identifying this ambiguity. Yes, the first row of the original Table 3 reused the AdamW runs reported in the main comparisons. We additionally add a variant that retains momentum update and RMS normalization but removes all spectral shaping. We have updated the results in the revised version and the updated results are reported below.

Table R1. Component ablations on REVE with Mental Arithmetic and EEGNet with BCIC-IV-2a.

Head shrinkageSubject-aware shrinkageREVE + Mental Arithmetic ACC-BAUROCAUC-PREEGNet + BCIC-IV-2a ACC-BKappaF1-W
AdamW0.741±0.0010.741 \pm 0.0010.835±0.0170.835 \pm 0.0170.644±0.0370.644 \pm 0.0370.579±0.0180.579 \pm 0.0180.438±0.0230.438 \pm 0.0230.570±0.0190.570 \pm 0.019
None✗0.702±0.0400.702 \pm 0.0400.851±0.0270.851 \pm 0.0270.707±0.0640.707 \pm 0.0640.576±0.0110.576 \pm 0.0110.434±0.0150.434 \pm 0.0150.563±0.0090.563 \pm 0.009
Fixed 10%✗0.695±0.0700.695 \pm 0.0700.850±0.0340.850 \pm 0.0340.728±0.0560.728 \pm 0.0560.582±0.0180.582 \pm 0.0180.443±0.0230.443 \pm 0.0230.571±0.0180.571 \pm 0.018
Adaptive✗0.770±0.0500.770 \pm 0.0500.873±0.0320.873 \pm 0.0320.739±0.0660.739 \pm 0.0660.584±0.0220.584 \pm 0.0220.445±0.0300.445 \pm 0.0300.575±0.0220.575 \pm 0.022
Adaptive✓0.788±0.004\mathbf{0.788 \pm 0.004}0.901±0.013\mathbf{0.901 \pm 0.013}0.790±0.041\mathbf{0.790 \pm 0.041}0.588±0.021\mathbf{0.588 \pm 0.021}0.450±0.027\mathbf{0.450 \pm 0.027}0.577±0.024\mathbf{0.577 \pm 0.024}

Regarding the Muon-style variant, as standard Muon aims to flatten their singular-value spectrum, it contradicts subject-aware shrinkage.

  1. Sensitivity of AGraSS to one-time calibration

We supplement how the KcK_c, head ratio and attenuation coefficient a1a_1 evolve during training every 5 epoch in Table R2. For Conformer on Mental Arithmetic, KcK_c remains close to the calibration estimate of 1, head ratio is close to the calibration estimate of 0.063, recalculated attenuation coefficient a1a_1 during training and the calibration estimate 0.557 is at the similar level. This suggests that one-time calibration can remain relatively stable.

Table R2. Re-evaluated KcK_c during training for Conformer on Mental Arithmetic. The calibration estimate is Kc=1K_c=1.

Training epochRe-evaluated KcK_cRe-evaluated head ratioRe-evaluated a1a_1
010.0630.557
500.0671.000
1010.0640.677
1510.0830.677
2010.0780.684
2520.1320.637
3010.0880.673
3510.0670.643
4010.0660.676
4510.0880.762
5010.0890.737

We further supplement the performance difference when calibration is run at different epochs, as shown in Table R3. It shows broadly comparable performance across different calibration durations. We have supplemented the results in Table 11 in the revised version.

Table R3. Effect of calibration duration on AGraSS using Conformer on Mental Arithmetic. Results are test performance at the validation-selected checkpoint.

Calibration epochs123 (default)45678910AdamW
AUROC0.7850.7860.7990.7880.7790.7880.7930.7990.7970.7890.756

For some details of hyperparameters, we use one cached power iteration per optimization step (T=1T=1). For the SNR analysis in Section 2.3, models use a large batch of 512, specifically, 32 microbatches of size 16.

AuthorsOpenReview ↗

Response to Reviewer VVhT (2/2)

  1. Subject identity decodability

We supplement the evaluation of the linear decodability of subject identity from the learned representations. We froze the encoders trained with AdamW, Muon, and AGraSS, and fitted a separate linear subject classifier to each encoder’s representations on conformer. The resulting mean test subject-classification accuracies across seeds are:

DatasetAdamWMuonAGraSS
Mental Arithmetic84.71%75.68%67.65%
FACED90.80%84.56%80.60%

These results show that subject identity is less linearly decodable from representations learned with AGraSS, validating the mechanism. We have supplemented the results in Table 4 in the revised version.

  1. Combination with objective level method

We thank the reviewer for suggesting this experiment. We evaluated AGraSS in combination with patient-adversarial learning (PANN) [1], and conducted experiments on the CHB-MIT dataset [2] using 4-second EEG clips and a DCRNN backbone [3] following PANN. It shows that optimizer level design of AGraSS is complementary to other strategies. We have supplemented the results in Table 13 in the revised version.

Model (Optimizer)AUROCF1 score
DCRNN (Adam)0.931 ± 0.0170.668 ± 0.052
DCRNN (AGraSS)0.946 ± 0.0040.725 ± 0.018
DCRNN+PANN (Adam)0.940 ± 0.0130.689 ± 0.029
DCRNN+PANN (AGraSS)0.954 ± 0.0110.739 ± 0.041

We also provide the result under the within-subject condition. We split each subject into training, validation, and test sets, targeting a 70%/15%/15% split. We then use the training data of all subjects to train the DCRNN model. It can be seen AGraSS can also improve the performance under the within-subject condition. We’d like to clarify that our objective is not to remove subject-related information entirely or enforce complete subject invariance, which is different from PANN. Instead, AGraSS applies additional soft shrinkage to leading subject-specific gradient spectra so that subject-specific features will not dominate gradients. Thus AGraSS can encourage learning more generalizable features without sacrificing the capacity to learn useful patient-specific information under within-subject settings.

Model (Optimizer)AUROCF1 score
DCRNN (Adam)0.933 ± 0.0010.742 ± 0.001
DCRNN (AGraSS)0.963 ± 0.0020.807 ± 0.008

[1] Cross-patient automatic epileptic seizure detection using patient-adversarial neural networks with spatio-temporal eeg augmentation. Biomedical Signal Processing and Control 2024.

[2] Application of machine learning to epileptic seizure detection, in: Proceedings of the 27th International Conference on Machine Learning (ICML-10), 2010, pp. 975–982.

[3] Self-supervised graph neural networks for improved electroencephalographic seizure analysis, in: International Conference on Learning Representations, 2021.

  1. Null baseline for gradient spectral-head.

Thank you for this suggestion. We added a random-direction null baseline for the settings examined in Fig. 2. As shown in the table below, the observed projections of leading direction are approximately 5.5–223 times the corresponding null means across the four settings. Thus, these projections substantially exceed the average overlap with matched random directions. This comparison provides additional evidence for subject-related alignment with the leading task-gradient direction in the examined settings. We have supplemented the results in Table 7 in the revised version.

Model / DatasetObserved normalized absolute projectionRandom-directionObserved / Null
CTNet / Mumtaz20160.0850.0155.7×
EEGNet / FACED0.2350.0435.5×
Conformer / Mental Arithmetic0.8800.01848.9×
REVE / Mental Arithmetic0.2900.0013223.1×
Reviewer_H9FWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the singular-value spectra of gradients during EEG model training. It observes that gradient energy is concentrated in a few leading modes, that gradients from subject-identity classification have large projections onto the leading task-gradient modes, and that some spectral-tail modes exhibit low estimated signal-to-noise ratios. Under an additive input model comprising class-dependent components, subject-dependent components, and noise, the authors provide an asymptotic explanation for the concentration of these projections in the spectral head when subject-related variation is sufficiently strong. Motivated by these findings, they propose the AGraSS optimizer.

Critical strengths
  1. The authors translate their observations of spectral-head concentration, subject-related gradient projections, and noise sensitivity in the spectral tail into a concrete update rule. The paper therefore goes beyond reporting accuracy improvements from replacing an optimizer. Head rescaling, additional conflict-based shrinkage, and residual preservation have distinct and interpretable roles. This provides a useful research direction for EEG representation learning and matrix-based optimization.
  2. The evaluation covers both task-specific networks and pretrained foundation models, with explicitly disjoint training, validation, and test subjects. The range of evaluated models and the promising gains in some configurations support the potential usefulness of the approach.
Critical weaknesses
  1. The algorithm derives shared coefficients from the signs of gradient projections at corresponding singular-value indices across matrices during calibration, then applies these coefficients to evolving singular modes of the momentum matrices during training. However, singular-value ordering does not assign a fixed subject-related interpretation to an index, and corresponding indices across different matrices need not represent comparable directions. Appendix A.6 also states that calibration uses a disposable model copy, making it necessary to establish how the calibration state relates to the subsequent training states. The paper does not report the actual values of KcK_c, the coefficient distributions, the frequency of Kc=0K_c=0, or whether the modes selected for additional shrinkage continue to exhibit conflict during training. The 95% cross-matrix consensus requirement and the contiguous-prefix rule could also leave the additional subject-aware shrinkage inactive, but its activation frequency is unclear.
  2. The current experiments do not sufficiently establish the robustness of the overall performance advantage. For example, the BCIC-IV-2a test set contains only two subjects. Improvements are small in several configurations: Conformer’s ACC-B on BCIC-IV-2a increases from 0.500 with Muon to 0.502, and CBraMod’s ACC-B on TUSL increases from 0.701 to 0.702. There are also configurations in which AGraSS does not outperform the baselines, such as LaBraM on Mental Arithmetic, where its ACC-B is lower than that of AdamW. Means and standard deviations alone do not establish whether these differences are statistically significant. However, without paired comparisons and subject-level uncertainty analyses, the evidence supporting broad performance benefits remains limited.
Questions for authors
  1. To what extent is the subject-aware shrinkage actually activated? Please report the selected spectral-head ratios, KcK_c, and the distributions of aka_k for the evaluated model–dataset configurations, together with the frequency of Kc=0K_c=0 across seeds. Please also re-estimate the projections at several training checkpoints to assess whether the indices initially selected for additional shrinkage continue to predominantly exhibit negative projections. How are the initialization and parameter states of the calibration copy related to those of the model used for subsequent training?
  2. Is subject-identity information necessary for the observed gains? Under otherwise identical settings in representative configurations, does calibration using true subject identities consistently outperform calibration using subject labels randomly shuffled across samples or windows, as well as a control that applies comparable additional shrinkage without using subject-identity information?
  3. Which gains are reliable across unseen subjects? Please provide paired performance differences for each seed in the main comparisons, together with results for individual test subjects or an appropriate subject-level uncertainty analysis.
3 comments
AuthorsOpenReview ↗

Response to Reviewer H9FW (1/2)

We sincerely thank the reviewer for appreciating our research direction and broad evaluation, and providing insightful suggestions. We carefully address your concerns as follows.

  1. More details and analyses about subject-aware shrinkage and our method

The reason why we derive shared coefficients across matrices is to conservatively suppress these directions only when most matrices have the consistent trend. Our analysis experiments show that top singular indices are directions with more subject-related information, and in practice, we leverage the consistency among matrices to identify the potential reliability of applying suppression. This is a conservative approach which may not be optimal, and a more fine-grained design can be potential future work.

For the actual values of selected head numbers, we have supplemented Table 6 in the revised version (i.e. Table R1 below), which reports the selected subject-specific spectral head number KcK_c values for all 28 model–dataset combinations. In the reported configurations, Kc>0K_c>0 in 18 of 28 settings, which may depend on the model and dataset. The frequency of (Kc>0K_c>0=0) across seeds are relatively stable.

Table R1. Automatically selected subject-specific spectral head size (KcK_c). dmind_{min} refer to the minimal dimension of the matrices, representing the singular number that KcK_c is selected from.

Expert models

ModelBCIC-IV-2aMental ArithmeticFACEDMumtaz
EEGNet (dmin=16d_{min}=16)1001
Conformer (dmin=32d_{min}=32)1111
ShallowConvNet (dmin=40d_{min}=40)1172
CTNet (dmin=22/20/32/19d_{min}=22/20/32/19)1010

EEG foundation models

ModelADFTDTUSLBCIC-IV-2aMental Arithmetic
REVE (dmin=512d_{min}=512)42026
LaBraM (dmin=200d_{min}=200)0000
CBraMod (dmin=100d_{min}=100)4011

For the distribution of scaling coefficient aka_k, it can be calculated from median bm,kb_{m,k} through the definition. We have supplemented the spectral analyses (bm,kb_{m,k}) of all 28 model–dataset combinations in Figures 4-6 in the revised version, so here we supplement the scaling coefficient in several representative model–dataset settings below as Table R2. (supplemented in Table 9 in the revised version)

Table R2. Additional scaling coefficients aka_k across singular indices. Ranges report the minimum and maximum within each index group.

Model–dataset settingSingular index kkScaling coefficient aka_k
Conformer + FACED10.642222277
Conformer + Mental Arithmetic10.557334461
REVE + ADFTD10.674855820
REVE + ADFTD20.947755705
REVE + ADFTD30.979950701
REVE + ADFTD40.987120800
REVE + ADFTD5–420.993596351–0.999985726

To assess whether the modes selected for additional shrinkage may change during training, we re-evaluate how the KcK_c evolve during training every 5 epoch, as shown in Table R3 (i.e. Table 10 in the revised version). For Conformer on Mental Arithmetic, KcK_c remains close to the calibration estimate of 1, representing the same first index for suppression. This suggests that the extent of the conflict-selected leading prefix remains relatively stable.

Table R3. Re-evaluated KcK_c during training for Conformer on Mental Arithmetic. The calibration estimate is Kc=1K_c=1.

Training epoch05101520253035404550
Re-evaluated KcK_c10111211111

We have also supplemented Table 5 in the revised version (i.e. Table R4 below) that reports spectral-head ratios for all 28 model–dataset combinations.

Table R4. Automatically selected spectral-head ratios across model–dataset combinations. Each ratio determines the proportion of leading singular modes subject to spectral-head rescaling.

Expert models

ModelBCIC-IV-2aMental ArithmeticFACEDMumtaz
EEGNet0.2650.1200.1350.195
Conformer0.0250.0630.0650.025
ShallowConvNet0.2650.2600.2300.280
CTNet0.1000.0300.0300.025

EEG foundation models

ModelADFTDTUSLBCIC-IV-2aMental Arithmetic
REVE0.0900.0650.0900.050
LaBraM0.0450.0550.0350.040
CBraMod0.0550.0800.0650.045
AuthorsOpenReview ↗

Response to Reviewer H9FW (2/2)

  1. Statistical reliability.

To evaluate the statistical evidence of broad performance benefits, we perform paired statistical significance analysis on the mean balanced accuracy across all 28 settings, and the results show that AGraSS > AdamW is statistically significant (p<0.001), AGraSS > Muon is statistically significant (p<0.001), while the difference between AdamW and Muon is not significant (p=0.84). So broadly, AGraSS show robust performance gains over other optimizers.

  1. Verification of the subject-identity information for the observed gains

We supplement a test of the effect of subject-identity information. We compare calibration using true subject labels, using randomly shuffled subject labels across samples, and removing this additional shrinkage component (using no subject labels) in Table R5 below for REVE on Mental Arithmetic and EEGNet on BCIC-IV-2a (results are supplemented in Table 12 in the revised version). It shows that calibration using true subject identities consistently outperform calibration using subject labels randomly shuffled across samples. Meanwhile, ablation of subject-aware shrinkage also demonstrates its effectiveness for the observed gains.

Table R5. Effect of subject-identity information on calibration. We compare true subject labels, randomly shuffled subject labels across samples, and no subject-aware shrinkage.

REVE + Mental Arithmetic

SettingACC-BAUROCAUC-PR
Calibration using true subject labels0.788 ± 0.0040.901 ± 0.0130.790 ± 0.041
Calibration using randomly shuffled subject labels0.686 ± 0.0430.738 ± 0.0490.455 ± 0.056
No subject-aware shrinkage0.770 ± 0.0500.873 ± 0.0320.739 ± 0.066

EEGNet + BCIC-IV-2a

SettingACC-BKappaF1-W
Calibration using true subject labels0.589 ± 0.0160.452 ± 0.0210.579 ± 0.021
Calibration using randomly shuffled subject labels0.581 ± 0.0230.441 ± 0.0310.574 ± 0.023
No subject-aware shrinkage0.584 ± 0.0220.445 ± 0.0300.575 ± 0.022
  1. Results for individual test subjects

To evaluate the gains across subjects, we supplemented subject-level evaluation in Table 14 in the revised version. We provide balanced accuracy for each of the test subjects on a representative setting (Conformer on MentalArithmetic) trained with AGraSS and AdamW, shown in Table R6 below. All subjects have better performance under AGraSS.

Table R6: Per-subject test balanced accuracy of Conformer on MentalArithmetic.

OptimizerSubject33Subject34Subject35Subject36
AdamW0.508 ±\pm 0.0190.603 ±\pm 0.1260.508 ±\pm 0.0190.550 ±\pm 0.068
AGraSS (Ours)0.536 ±\pm 0.0120.808 ±\pm 0.0450.575 ±\pm 0.0350.914 ±\pm 0.035

Dear Reviewer H9FW,

Thank you again for your careful evaluation and constructive comments. As the discussion period draws to a close, we would appreciate your feedback on whether our responses have addressed your concerns. We would be happy to clarify any remaining points or provide additional information. We appreciate your time and effort, and hope our responses help you reassess the submision.

Sincerely,

The Authors

Reviewer_GQcHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the spectral structure of gradients in EEG learning and proposes AGraSS (Adaptive Gradient Spectral Scaling), an EEG-oriented optimizer. The authors identify three characteristics of EEG training gradients: pronounced spectral concentration in leading singular directions, an association between these directions and subject-related variation, and relatively low signal-to-noise ratios in the spectral tail. Motivated by these observations, AGraSS combines automatic spectral-head detection with subject-aware soft shrinkage to moderate dominant gradient directions while preserving residual components. Experiments on six EEG datasets, involving four task-specific architectures and three pretrained EEG foundation models, show generally improved performance compared with AdamW, SGD, and Muon.

Critical strengths
  1. An interesting gradient-spectral perspective on EEG optimization. The paper investigates EEG learning from the perspective of gradient singular-value spectra, rather than focusing primarily on model architectures or representation-learning objectives. The analyses of spectral concentration, subject-related gradient projections, and tail reliability provide useful empirical insights and motivate an EEG-oriented optimizer design. The accompanying theoretical analysis further strengthens the motivation, although it relies on simplified assumptions.
  2. Broad empirical evaluation across EEG models and tasks. The experiments cover six EEG datasets and seven model architectures, including both specialized EEG networks and pretrained EEG foundation models. Comparisons with AdamW, SGD, and Muon demonstrate generally favorable performance, while the ablation studies provide evidence that adaptive head selection and subject-aware shrinkage both contribute to the improvements. This breadth supports the practical relevance of the proposed optimization approach.
Critical weaknesses
  1. Limited evidence for the generalizability of the identified gradient characteristics. The paper identifies three gradient spectral characteristics in EEG learning and uses them to motivate AGraSS. However, the analyses in Figures 1–3 are conducted on limited model–dataset combinations, without systematically disentangling task-dependent, dataset-dependent, and architecture-dependent effects. Moreover, some of these characteristics, such as spectral concentration and low-SNR gradient directions, may also occur in general neural network optimization. Therefore, it remains unclear whether these observations represent general properties of EEG learning, specific model–dataset behaviors, or broader optimization phenomena. This limits the justification for the proposed EEG-specific optimization strategy.
  2. Insufficient validation of the subject-aware gradient suppression mechanism. A central motivation of AGraSS is that dominant gradient directions contain subject-related variations that may interfere with task-relevant learning. However, subject-related information is not necessarily detrimental across all EEG tasks and may carry useful discriminative information, particularly in clinical classification settings. Although the gradient projection analyses and ablation experiments support the proposed approach, they do not directly demonstrate that AGraSS suppresses harmful subject-specific information while preserving task-relevant features. Consequently, it remains unclear whether the observed improvements arise from effective subject-related gradient suppression or more general spectral regularization.
  3. Insufficient analysis of optimization efficiency and performance robustness. AGraSS introduces additional spectral calibration and low-rank decomposition operations, but the paper mainly provides theoretical complexity analysis without comprehensive empirical comparisons of training time, peak memory consumption, and convergence efficiency. Furthermore, while the method achieves generally favorable results, improvements are relatively small in several settings, and some metrics remain inferior to those of baseline optimizers. The lack of statistical significance analysis makes it difficult to assess whether the improvements consistently justify the additional computational overhead.
Questions for authors
  1. Generalizability of Gradient Spectral Characteristics Are the three identified gradient spectral characteristics consistent across different EEG tasks, datasets, and model architectures? Since Figures 1–3 mainly examine different model–dataset combinations, how can the authors distinguish task-dependent effects from architecture-dependent effects? Furthermore, what evidence supports the claim that these characteristics are specific to EEG learning rather than general neural network optimization?
  2. Effectiveness and Task Dependence of Subject-Aware Shrinkage How does AGraSS distinguish harmful subject-related gradient components from subject-specific information that may be beneficial for downstream prediction? Can the authors provide direct evidence, such as changes in subject-identification accuracy, representation separability, or cross-subject generalization performance, to demonstrate that subject-aware shrinkage suppresses irrelevant information rather than merely acting as generic spectral regularization?
  3. Computational Efficiency and Reliability of Improvements Can the authors provide empirical comparisons of wall-clock training time, peak memory usage, and convergence efficiency against AdamW and Muon, including calibration overhead? Additionally, given the relatively small improvements or occasional underperformance in certain settings, can statistical significance tests establish the robustness of the reported gains?
3 comments
AuthorsOpenReview ↗

Response to Reviewer GQcH (1/2)

We sincerely thank the reviewer for appreciating our analysis perspective and broad evaluation, and providing insightful suggestions. We carefully address your concerns as follows.

  1. Evidence for the generalizability of the identified gradient characteristics.

We have expanded the spectral analyses to all 28 model–dataset combinations as shown in Appendix Figures 4-6 in the revised version. The additional figures report initial and final gradient spectra, gradient SNR across singular modes, and normalized subject-gradient projections under different settings. These analyses show consistent gradient characteristics across architectures and dataset, including: spectral head is several orders of magnitude higher than the smooth tail; subject-related gradient projections tend to concentrate at leading indices; and tail directions often exhibit lower SNR.

Considering these characteristics, first, the pattern that the subject-related gradient projections tend to concentrate at the head, is specific to the EEG domain due to subject-related heterogeneity in EEG data. For other properties such as spectral concentration, they may also appear in other domains and we do not claim that these properties alone are unique to EEG. However, when combined with the unique property of subject-related head spectra in EEG, these properties turn to unique problems in EEG as subject information (in head) can dominate the gradients, while in other domains these properties alone do not lead to such problem. Therefore, it is the combination of these properties that motivate the proposed EEG-specific optimization strategy. These phenomenon and problems have not been analyzed in EEG learning before.

  1. Validation of the subject-aware gradient shrinkage mechanism

Firstly, we have validated the effectiveness of subject-aware gradient shrinkage in the ablation study in Table 3. We further supplement the effectiveness of subject-aware shrinkage on the clinical dataset ADFTD.

Table R1. Effectiveness of subject-aware shrinkage using REVE on ADFTD.

Head shrinkageSubject-aware shrinkageACC-BKappaF1-W
Adaptive✗0.508 ± 0.0670.305 ± 0.0860.522 ± 0.063
Adaptive✓0.576 ± 0.0360.403 ± 0.0360.596 ± 0.029

Actually, we do not suppose subject-related information is always detrimental across all EEG tasks, and our method is to suppress subject-related gradient to encourage learning more generalizable features rather than removing all of them. So even in clinical classification settings where subject information may carry useful discriminative information, our method can have better results arising from subject-related gradient suppression in addition to general spectral regularization.

To further verify that our method has suppressed such information, we complement the linear decodability of subject identity from the learned representations (supplemented in Table 4 in the revised version). We froze the encoders trained with AdamW, Muon and AGraSS, and fitted a separate linear subject classifier to each encoder’s representations on conformer. The resulting test subject-classification accuracies are:

Table R2.Mean subject-classification accuracy across seeds using linear probes on frozen Conformer representations.

DatasetAdamWMuonAGraSS
Mental Arithmetic84.71%75.68%67.65%
FACED90.80%84.56%80.60%

These results indicate that subject identity is less linearly decodable from representations learned with AGraSS, and subject-classification accuracy remains above chance. These results suggest AGraSS can suppresses subject-specific information while also retaining some that has not been completely removed, as they may carry useful discriminative information in some tasks.

AuthorsOpenReview ↗

Response to Reviewer GQcH (2/2)

  1. Analysis of optimization efficiency and performance robustness

We have added wall-clock training time, peak memory usage, and convergence comparisons (supplemented in Table 8 and Figure 7 in the revised version). The following table reports calibration time separately from the main training stage, together with their combined cost.

SettingMeasurementAdamWMuonAGraSS
ShallowConvNet + MumtazCalibration time——11.66 s
Training time (50 epochs)179.12 s195.84 s195.70 s
Total time, including calibration179.12 s195.84 s207.36 s
Peak memory70.95 MiB70.83 MiB70.86 MiB
REVE + BCIC-IV-2aCalibration time——18.60 s
Training time (30 epochs)153.70 s160.33 s160.00 s
Total time, including calibration153.70 s160.33 s178.60 s
Peak memory7181.62 MiB7423.37 MiB7149.89 MiB

The one-time, three-epoch calibration stage takes approximately 6.0% to 11.6% of AGraSS’s main training time in these two settings. AGraSS has similar main-training time to Muon and AdamW. Peak memory usage remains comparable to both baselines. These measurements show that AGraSS achieves performance improvements with only modest increases in computational cost and runtime.

Statistical reliability. Our results report the mean and standard deviation of the results over five runs, so the results are generally reliable. Across all 28 settings, AGraSS outperforms AdamW in 27 settings considering the mean of balanced accuracy, and outperforms Muon in 28 settings, while AdamW and Muon each outperform the other in 14 settings. We further perform paired statistical significance analysis on the mean balanced accuracy across all settings, and the results show that AGraSS > AdamW is statistically significant (p<0.001), AGraSS > Muon is statistically significant (p<0.001), while the difference between AdamW and Muon is not significant (p=0.84). These results indicate that the reported gains are relatively robust considering the broad setting.

Dear Reviewer GQcH,

Thank you again for your careful evaluation and constructive comments. As the discussion period draws to a close, we would appreciate your feedback on whether our responses have addressed your concerns. We would be happy to clarify any remaining points or provide additional information. We appreciate your time and effort, and hope our responses help you reassess the submision.

Sincerely,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Responses and Revisions

Dear Area Chair,

We sincerely thank you for handling our submission and thank the reviewers for their careful evaluation and valuable comments. As the discussion period draws to a close, while we do not receive feedback from reviewers, we would like to briefly summarize our responses and the revisions we have made in the manuscript. We hope it useful for your final recommendation.

All three reviewers recognized the novel perspective of our method and the broad empirical evaluation across EEG models and tasks, describing it as “novel and coherent” (VVhT), “interesting gradient-spectral perspective on EEG optimization” (GQcH), “useful empirical insights” (GQcH), “provides a useful research direction”(H9FW), and “The evaluation is broad and disciplined” (VVhT). The reviewers also recognized the performance advantage of our method: “AGraSS is best or tied best in nearly all of them” (VVhT), “Comparisons demonstrate generally favorable performance”(GQcH), and “the promising gains” (H9FW). We appreciate the reviewers’ recognition of our effort to connect the characteristics of EEG gradients to a principled EEG optimizer design.

The reviewers' remaining concerns were about more evidence and further analyses, and we believe each of them is now addressed with our responses and new results in the revised manuscript. We summarize the responses and revisions (changes marked in blue in the revised manuscript) below.

Reviewer GQcH (Generalizability, subject-aware shrinkage, computational efficiency, and performance robustness): We expand the spectral analyses to all 28 model–dataset combinations (Figures 4–6) to demonstrate the generalizability. Frozen-encoder probes show lower subject-decoding accuracy under AGraSS than under AdamW or Muon (Table 4), validating the subject-aware shrinkage mechanism. Real training time, memory, and convergence comparisons are explicitly added (Table 8 and Figure 7), showing that our method only introduce a small increase in the total runtime. We add paired significance test across settings, favoring AGraSS over both AdamW and Muon p<0.001p<0.001 for each, while AdamW vs Muon is not statistically significant p=0.84p=0.84 (Appendix A.10), indicating the robustness of gains.

Reviewer H9FW (subject-aware shrinkage, reliability across subjects, and statistical significance). We report head ratios and KcK_c across all configurations, representative attenuation coefficients, and re-estimation of calibration statistics during training (Tables 5–6 and 9–10). We add comparisons using true subject labels, shuffled labels, and no subject-aware shrinkage (Table 12), verifying the subject-identity information for the observed gains. Individual test-subject results show higher performance under AGraSS than AdamW on all test subjects (Table 14), demonstrating that gains are reliable across subjects. We add paired significance test across settings, favoring AGraSS over both AdamW and Muon p<0.001p<0.001 for each, while AdamW vs Muon is not statistically significant p=0.84p=0.84 (Appendix A.10).

Reviewer VVhT (statistical significance, component attribution, calibration sensitivity, null comparisons, and complementarity with objective-level methods). We add paired significance test across settings, favoring AGraSS over both AdamW and Muon p<0.001p<0.001 for each, while AdamW vs Muon is not statistically significant p=0.84p=0.84 (Appendix A.10). We add the requested momentum-and-RMS variant without spectral shaping in ablation study (Table 3). We evaluate the performance under different calibration durations to demonstrate the sensitivity (Table 11), clarify the power-iteration and SNR microbatch settings, and provide a null baseline for gradient spectral-head to verify our results (Tables 7). We also supplement experiments to show that our method is complementary to objective-level methods such as PANN (Table 13).

We believe our responses and revisions have further strengthened the contribution of our paper, and properly addressed all questions from reviewers. We hope that the revised manuscript and our detailed responses assist your assessment of the submission.

Thank you again for your time and effort for handling our paper!

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.