AGraSS: Understanding and Shaping Gradient Spectra for EEG Learning

EEG signals exhibit substantial complexity, inter-subject heterogeneity, and are susceptible to noise, making it challenging to learn meaningful task-relevant representations. While the mainstream methods use neural networks trained through gradient-based optimization, the information carried by different gradient directions during EEG learning remains insufficiently understood. To this end, we investigate the spectral structure of task gradients in EEG models and propose a new optimizer tailored to EEG learning. Our analysis reveals pronounced spectral imbalance of gradients during training and finds a strong association between the leading spectral head and subject-related variation, as well as a low estimated gradient signal-to-noise ratio in the spectral tail. Theoretical analyses with a latent subject-variation model explain such gradient structure of the spectral head. Motivated by this analysis, we propose AGraSS ($\text{A}$daptive $\text{Gra}$dient $\text{S}$pectral $\text{S}$caling), an optimizer that combines automatic spectral-head detection with soft spectral shrinkage. AGraSS adaptively rescales the spectral head with additional attenuation on the pre-detected subject-related directions, moderating dominant update directions. Experiments on six EEG datasets with both expert models and pretrained EEG foundation models demonstrate the superiority of AGraSS, outperforming mainstream optimizers such as AdamW and Muon with improved performance. These findings highlight EEG-aware optimizer design as an important direction for improving EEG learning.