- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates whether high-frequency scalp EEG (>50 Hz) supports retrieval of the speech segment heard by a listener and contributes information complementary to low-frequency EEG. It trains separate low- and high-frequency matching branches and combines their scores using a weight selected on the validation set. On 17 unseen listeners, high-frequency EEG achieves 5.62% full-pool Top-1 accuracy among approximately 1,099 five-second candidates per listener, while fusion improves low-frequency accuracy from 8.65% to 15.39%. Comparisons among waveform, continuous phase, one-bit phase, and power representations suggest that fine temporal structure carries useful retrieval information under this protocol. The paper also evaluates frozen models on two public datasets and examines timing sensitivity, frequency bands, and electrode subsets.
Critical strengths
-
The evaluation addresses a useful empirical question with several sound design choices. Training, validation, and test participants are disjoint; fusion weights are selected using source validation data; and statistical comparisons use participants as independent units with multiple-comparison correction. The improvement from combining the two branches is substantial on the main dataset and is also observed in frozen evaluation on two public datasets.
-
The representation analyses are informative and the presentation is careful. Comparing waveform, continuous phase, one-bit phase, and power under a common retrieval protocol helps characterize the information accessible to the models. The authors explicitly acknowledge that one-bit phase is equivalent to the sign of the high-pass waveform and does not establish a unique neural phase-coding mechanism. They also acknowledge the limitations of the self-collected cohort comparison. The appendix provides implementation and statistical details, and the submission includes an anonymous code repository link.
Critical weaknesses
- The contribution of residual non-neural signals to the high-frequency results remains insufficiently assessed.
The experiments establish that high-frequency recordings contain information useful for identifying the synchronized speech segment. Given the motivation in perceived-speech decoding and brain–computer interfaces, a central remaining question is whether residual non-neural signals materially contribute to the reported gains.
The original SparrKULee acquisition protocol used electromagnetically shielded ER-3A insert earphones (Accou et al., acquisition protocol, Section 2.4), and the manuscript describes artifact interpolation and multichannel Wiener filtering. These safeguards are relevant, but the paper does not directly assess residual stimulus-synchronous electrical contamination or muscle activity in the high-frequency features. Such stimulus artifacts have been documented in continuous-speech EEG experiments (Maddox and Lee, eNeuro 2018, “Stimulus artifact removal”). Timing sensitivity and successful sign-based retrieval are compatible with several signal sources. Cross-dataset transfer strengthens the empirical finding, but does not resolve this attribution question. The authors appropriately avoid claiming a unique neural generator or phase-coding mechanism. Calibrated temporal-response analysis and targeted artifact-sensitivity checks would help establish the robustness of the gains to plausible non-neural contributions.
- The scope of generalization and the stability across training runs need clearer characterization.
The participant-disjoint split supports evaluation on unseen listeners, but the manuscript does not clearly report how speech recordings or stories overlap across training, validation, and test participants. Shared stimuli can be valid for the stated unseen-listener task, while leaving generalization to unseen speech content less clearly established. The frozen public-dataset evaluations provide useful complementary evidence. Reporting stimulus overlap and, where feasible, performance on held-out speech content would clarify the scope of the main benchmark result.
Appendix B.4 specifies a single prespecified random seed. Participant-level confidence intervals quantify variability across listeners conditional on the trained models; they do not measure variability due to training. Repeated runs for a small set of central comparisons, particularly LF, HF, fusion, and the main representation comparisons, would clarify the robustness of the conclusions.
The self-collected comparison provides additional observational evidence, but each cohort contains only six different participants and actual acoustic timing was not measured. The authors appropriately acknowledge these limitations in Section 3.5, so this comparison should receive limited weight as evidence for the effect of trigger precision.
Questions for authors
-
What evidence indicates that the high-frequency retrieval gains are robust to residual stimulus-synchronous electrical contamination and muscle activity? Please summarize the relevant acquisition safeguards across datasets. Could existing data support a temporal-response analysis referenced to actual acoustic input, including negative lags and accounting for preprocessing effects, or another targeted control addressing these alternative sources? A nonzero peak alone would need to be interpreted in light of acquisition delays and possible muscle responses.
-
How much do speech recordings, stories, and exact audio segments overlap across the participant splits? Can you report performance on a subset of held-out speech content, if such a subset is available, while retaining the unseen-listener evaluation? Please also clarify how the external datasets inform this aspect of generalization.
-
Are repeated-run results available for LF, HF, fusion, and the principal representation comparisons? A small number of additional seeds for these central comparisons would help establish whether their ordering and gains are stable, alongside the existing participant-level statistics.
-
In Figures 4C and S3, the temporal group has the lowest HF two-bin-phase full-pool Top-1 accuracy (0.12%, although significantly above chance with adjusted q = 0.017), while it appears strongest for LF. How should this contrast be interpreted given the common-average reference computed from all 64 electrodes and the different inputs and encoders used by the two branches? Could the authors discuss what, if anything, this pattern indicates about neural and non-neural contributions, including possible muscle contamination?
Thank you for your feedback on the experimental design and representation analysis. The general response covers the shared results. We clarify the scope and interpretation below.
W1. Residual non-neural signals in the HF results
Performance with the earphones out of the ears was close to chance (C3, Table G3). This provides additional evidence that performance varies with the listening condition. Whether residual non-neural signals materially contribute to the reported gains remains unresolved.
W2. Generalization, training stability, and the self-collected datasets
C4 reports results on unseen content and stimulus overlap. C1 reports variation across training runs.
Five-way accuracy was 26.04% for hardware-EEG and 19.09% for Matlab-EEG (20% chance). These datasets are small, and we plan to expand them to further substantiate our interpretation.
Q1. Acquisition safeguards, acoustic timing, and artifact controls
SparrKULee used shielded ER-3A earphones, artifact-segment interpolation, and multichannel Wiener filtering. For Etard and Weissbart, we used the audio onset provided with the public data. Our self-collected datasets used hardware triggers and MATLAB playback events, respectively. These references do not measure the actual sound arrival time at the ear. We have yet to perform positive- and negative-lag analyses against an acoustic reference or matched comparisons before and after additional artifact processing. The contribution of residual electrical crosstalk or EMG therefore remains unquantified. See C3 and Table G3 for the earphone control.
Q2. Stimulus overlap and unseen speech content
Listener splits are disjoint. Seven of 13 test stimulus IDs also occur in training or validation. Matching five-second windows by stimulus file and nominal start time, training and test share 183 windows, and validation and test share 1,310. The six held-out podcast episodes yield 6,768 queries from 14 unseen listeners. Fusion outperformed either branch on all three metrics (C4, Table G4). The full-pool candidates still include seen content.
For Etard and Weissbart, we trained source models on the 56 shared electrodes described in the paper and evaluated them frozen. The target datasets were excluded from training and fusion-weight selection (C4, Table G5). These tests provide evidence of transfer across separately collected corpora, while the held-out episodes test unseen content within the source dataset.
Q3. Repeated training of LF, HF, and fusion
C1 reports the three-seed results for the core waveform models and their fusion. Fusion ranked first, followed by LF and HF, in all three runs. We kept the 54/14/17 listener split and trained on 468 recordings after excluding the two earphone-control recordings. All three seeds used the same 18,680 test queries and candidate sets from 17 listeners, and each fusion paired LF and HF from the same seed. The other main HF representations have yet to be run with all three seeds. The historical seed-0 results are separate from the three-seed statistics.
Q4. Temporal-group differences between HF and LF
The in-ear and out-of-ear comparison in C3 leaves a possible contribution from EMG. Prior work reports that scalp EMG contamination from temporalis activity tends to be stronger at temporal and nearby peripheral sites [2]. In our electrode-group analysis, the temporal group had the lowest accuracy for the HF two-bin-phase model, while the parietal group performed better. This decoding pattern differs from the typical spatial distribution of temporalis EMG amplitude. Our analysis is descriptive, and the underlying neural encoding remains unclear.
[2] Goncharova, Irina I., et al. "EMG contamination of EEG: spectral and topographical characteristics." Clinical neurophysiology 114.9 (2003): 1580-1593.