- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper compares three EEG front ends, raw patches, Morlet log power and complex Morlet coefficients, feeding the same small transformer trained from scratch. It covers P300 and N170 ERP tasks and one-second sleep segments at five training subject counts from 20 to 240. Raw beats log power on ERPs by 0.09 to 0.12 balanced accuracy, complex coefficients recover much of that gap, and log power helps on sleep only at small subject counts.
Critical strengths
- The study is carefully controlled and unusually honest. The transformer body, splits and schedule are fixed, and the authors add the controls a skeptical reader would ask for, including normalization scope in both directions, token and compute matching for sleep, a six-channel ERP control and a capacity-matched convolution. Failed hypotheses, a falsified mechanism and corrected analyses are all reported rather than hidden, which I appreciate.
- The results give practical guidance. On both ERP tasks raw patches beat fixed log power at every subject count, and keeping complex coefficients recovers a median of 72 percent of that gap. Rotating phase per trial and frequency costs 0.12 to 0.20 balanced accuracy. Together with the sleep result, this supports the main message that the best input depends on task and data budget.
Critical weaknesses
- The scope is too narrow for a framing aimed at foundation model designers. Everything uses a 0.8M-parameter, four-layer transformer trained from scratch, with a two-point learning rate grid and three seeds. The pretraining experiments end without a conclusion, and sleep uses one-second windows rather than standard 30-second staging. It is unclear whether any of the rankings would hold for large pretrained models or standard benchmarks.
- The main mechanistic result is hard to interpret. The per-trial phase rotation changes event alignment and cross-frequency relationships at once, and its angle is a hash of the epoch content, with label independence checked on only three people. That breaking phase locking hurts ERP decoding is also expected from ERP physiology. Separate interventions that change only alignment, such as a time shift, or only cross-frequency structure would make this result far more informative.
- The paper is very hard to read. The main text leans on claim codes, registered readings and appendix pointers, and many key numbers sit only in the 37-page appendix. The preregistration is internal, several readings were amended after results or after an earlier review, and the code and evidence archive are private, so reviewers cannot check the provenance the paper relies on. A shorter main text built around three or four findings would help greatly.
Questions for authors
see weaknesses.