- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
NeuroSketch presents a practical neural-decoding recipe that combines two-dimensional convolutions, gradual feature expansion, early downsampling, and grouped convolutions across multiple recording modalities and speech, visual, and auditory tasks.
Critical strengths
The study is unusually comprehensive, covering eight tasks, six datasets, multiple recording modalities, and numerous architectural alternatives. The appendices provide substantial implementation details, including model sizes, preprocessing procedures, training settings, baseline configurations, and supplementary analyses. The resulting Base and Large models are compact, practical, and frequently competitive across heterogeneous neural-decoding scenarios.
Critical weaknesses
Most headline results rely on random within-subject trial splits, primarily demonstrating subject-specific decoding rather than generalization to unseen participants. Repeated stimuli, shared sessions, or temporally adjacent trials may lead to overly favorable splits and inflated performance estimates. Furthermore, the architectural recipe is developed and evaluated on the same eight tasks, without an independent benchmark held out from architecture selection. This raises concerns about recipe-level overfitting. Cross-subject evaluation is limited to two tasks and three models, with improvements over the strongest baseline of only 0.8 and 1.2 percentage points. Performance on SEED-DV also remains near chance level, weakening claims of robust generalization to unseen participants.
Questions for authors
-
Can the authors provide block-wise, session-wise, or temporally separated evaluations to assess whether random trial splits lead to overly optimistic performance estimates?
-
Given that the recipe was developed and evaluated on the same eight tasks, how do the authors address the risk of recipe-level overfitting?
Please Engage in the Author–Reviewer Discussion
Dear Reviewers,
As the author–reviewer discussion period is approaching its deadline (November 18, AoE), I would like to kindly remind you to review the authors' rebuttals and actively engage in the discussion as soon as possible.
Please address any outstanding questions or concerns and update your assessments and ratings if appropriate.
Thank you very much for your time and efforts!
Best regards, Area Chair