SPD-MetaFormer Is What You Need for Small-Data Brain Decoding

Brain signal decoding is challenging because neural recordings are noisy and vary across individuals, while labeled data are often limited. Recent attention-based models on the symmetric positive definite (SPD) manifold have nevertheless achieved strong performance using covariance and connectivity representations, yet the contribution of learned token weighting remains unclear. We examine two representative architectures, MAtt (based on log-Euclidean geometry) and GBWAtt (based on generalized Bures–Wasserstein geometry), and find that their learned attention weights remain close to uniform after training. We trace this behavior to bounded similarity parameterizations that, under the original softmax scaling, limit attention-weight contrast. Moreover, replacing learned weights with uniform weights, either throughout training or post hoc in fully trained models without further optimization, has little effect on mean predictive performance while preserving each model’s original aggregation geometry. Motivated by these findings, we introduce SPD-MetaFormer, an attention-free architecture built on uniformly weighted Fréchet aggregation under log-Euclidean geometry. Its backbone combines geodesic residual updates and spectral feedforward transformations to refine a summary token, while retaining memory tokens for a learned weighted readout. Token states remain SPD-valued until tangent-space classification. Across three EEG benchmarks, SPD-MetaFormer matches or improves upon the performance of strong Euclidean and manifold baselines, including the original and uniform-weight variants of MAtt and GBWAtt. These results suggest that, in the short-sequence and limited-data regimes studied, carefully designed SPD architectures can provide a simpler and effective alternative to adaptive manifold attention.