acceptodds
Under review as a conference paper at ICLR 2027

Decoding Perceived Speech from High-Frequency EEG Phase Patterns

Abstract

Retrieving segments of continuous speech from scalp electroencephalography (EEG) identifies what a listener heard among candidate audio segments. Existing methods mainly exploit low-frequency EEG related to slower changes in speech. High-frequency EEG (>50 Hz) contains timing cues associated with the speech fundamental frequency and rapid envelope fluctuations, but its low signal-to-noise ratio has limited its use for this task. We ask whether these signals independently support multi-candidate retrieval, complement low-frequency decoding, and retain segment-discriminative information in their phase structure. We train separate matching branches for low- and high-frequency EEG with speech features at corresponding time scales, fuse their scores, and compare waveform, phase, two-bin phase, and power representations of the same high-frequency signal. For 17 unseen listeners, the high-frequency branch attains 5.62% Top-1 accuracy in five-second retrieval from an average of more than 1,000 candidates per listener. Score fusion raises full-pool Top-1 accuracy from 8.65% for the low-frequency branch to 15.39%. Continuous phase and even a one-bit phase representation retain substantial retrieval performance after discarding amplitude scale. These results establish the complementary value of high-frequency EEG and the segment-level utility of its rapid phase-related timing structure. Code is available at https://anonymous.4open.science/r/HFEEG-decoding-2203/.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Scalp EEG, perceived speech decoding, speech segment retrieval, high-frequency phase patterns, cross-participant generalization, score fusion

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_93iQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether high-frequency scalp EEG (>50 Hz) supports retrieval of the speech segment heard by a listener and contributes information complementary to low-frequency EEG. It trains separate low- and high-frequency matching branches and combines their scores using a weight selected on the validation set. On 17 unseen listeners, high-frequency EEG achieves 5.62% full-pool Top-1 accuracy among approximately 1,099 five-second candidates per listener, while fusion improves low-frequency accuracy from 8.65% to 15.39%. Comparisons among waveform, continuous phase, one-bit phase, and power representations suggest that fine temporal structure carries useful retrieval information under this protocol. The paper also evaluates frozen models on two public datasets and examines timing sensitivity, frequency bands, and electrode subsets.

Critical strengths
  1. The evaluation addresses a useful empirical question with several sound design choices. Training, validation, and test participants are disjoint; fusion weights are selected using source validation data; and statistical comparisons use participants as independent units with multiple-comparison correction. The improvement from combining the two branches is substantial on the main dataset and is also observed in frozen evaluation on two public datasets.

  2. The representation analyses are informative and the presentation is careful. Comparing waveform, continuous phase, one-bit phase, and power under a common retrieval protocol helps characterize the information accessible to the models. The authors explicitly acknowledge that one-bit phase is equivalent to the sign of the high-pass waveform and does not establish a unique neural phase-coding mechanism. They also acknowledge the limitations of the self-collected cohort comparison. The appendix provides implementation and statistical details, and the submission includes an anonymous code repository link.

Critical weaknesses
  1. The contribution of residual non-neural signals to the high-frequency results remains insufficiently assessed.

The experiments establish that high-frequency recordings contain information useful for identifying the synchronized speech segment. Given the motivation in perceived-speech decoding and brain–computer interfaces, a central remaining question is whether residual non-neural signals materially contribute to the reported gains.

The original SparrKULee acquisition protocol used electromagnetically shielded ER-3A insert earphones (Accou et al., acquisition protocol, Section 2.4), and the manuscript describes artifact interpolation and multichannel Wiener filtering. These safeguards are relevant, but the paper does not directly assess residual stimulus-synchronous electrical contamination or muscle activity in the high-frequency features. Such stimulus artifacts have been documented in continuous-speech EEG experiments (Maddox and Lee, eNeuro 2018, “Stimulus artifact removal”). Timing sensitivity and successful sign-based retrieval are compatible with several signal sources. Cross-dataset transfer strengthens the empirical finding, but does not resolve this attribution question. The authors appropriately avoid claiming a unique neural generator or phase-coding mechanism. Calibrated temporal-response analysis and targeted artifact-sensitivity checks would help establish the robustness of the gains to plausible non-neural contributions.

  1. The scope of generalization and the stability across training runs need clearer characterization.

The participant-disjoint split supports evaluation on unseen listeners, but the manuscript does not clearly report how speech recordings or stories overlap across training, validation, and test participants. Shared stimuli can be valid for the stated unseen-listener task, while leaving generalization to unseen speech content less clearly established. The frozen public-dataset evaluations provide useful complementary evidence. Reporting stimulus overlap and, where feasible, performance on held-out speech content would clarify the scope of the main benchmark result.

Appendix B.4 specifies a single prespecified random seed. Participant-level confidence intervals quantify variability across listeners conditional on the trained models; they do not measure variability due to training. Repeated runs for a small set of central comparisons, particularly LF, HF, fusion, and the main representation comparisons, would clarify the robustness of the conclusions.

The self-collected comparison provides additional observational evidence, but each cohort contains only six different participants and actual acoustic timing was not measured. The authors appropriately acknowledge these limitations in Section 3.5, so this comparison should receive limited weight as evidence for the effect of trigger precision.

Questions for authors
  1. What evidence indicates that the high-frequency retrieval gains are robust to residual stimulus-synchronous electrical contamination and muscle activity? Please summarize the relevant acquisition safeguards across datasets. Could existing data support a temporal-response analysis referenced to actual acoustic input, including negative lags and accounting for preprocessing effects, or another targeted control addressing these alternative sources? A nonzero peak alone would need to be interpreted in light of acquisition delays and possible muscle responses.

  2. How much do speech recordings, stories, and exact audio segments overlap across the participant splits? Can you report performance on a subset of held-out speech content, if such a subset is available, while retaining the unseen-listener evaluation? Please also clarify how the external datasets inform this aspect of generalization.

  3. Are repeated-run results available for LF, HF, fusion, and the principal representation comparisons? A small number of additional seeds for these central comparisons would help establish whether their ordering and gains are stable, alongside the existing participant-level statistics.

  4. In Figures 4C and S3, the temporal group has the lowest HF two-bin-phase full-pool Top-1 accuracy (0.12%, although significantly above chance with adjusted q = 0.017), while it appears strongest for LF. How should this contrast be interpreted given the common-average reference computed from all 64 electrodes and the different inputs and encoders used by the two branches? Could the authors discuss what, if anything, this pattern indicates about neural and non-neural contributions, including possible muscle contamination?

1 comment

Thank you for your feedback on the experimental design and representation analysis. The general response covers the shared results. We clarify the scope and interpretation below.

W1. Residual non-neural signals in the HF results

Performance with the earphones out of the ears was close to chance (C3, Table G3). This provides additional evidence that performance varies with the listening condition. Whether residual non-neural signals materially contribute to the reported gains remains unresolved.

W2. Generalization, training stability, and the self-collected datasets

C4 reports results on unseen content and stimulus overlap. C1 reports variation across training runs.

Five-way accuracy was 26.04% for hardware-EEG and 19.09% for Matlab-EEG (20% chance). These datasets are small, and we plan to expand them to further substantiate our interpretation.

Q1. Acquisition safeguards, acoustic timing, and artifact controls

SparrKULee used shielded ER-3A earphones, artifact-segment interpolation, and multichannel Wiener filtering. For Etard and Weissbart, we used the audio onset provided with the public data. Our self-collected datasets used hardware triggers and MATLAB playback events, respectively. These references do not measure the actual sound arrival time at the ear. We have yet to perform positive- and negative-lag analyses against an acoustic reference or matched comparisons before and after additional artifact processing. The contribution of residual electrical crosstalk or EMG therefore remains unquantified. See C3 and Table G3 for the earphone control.

Q2. Stimulus overlap and unseen speech content

Listener splits are disjoint. Seven of 13 test stimulus IDs also occur in training or validation. Matching five-second windows by stimulus file and nominal start time, training and test share 183 windows, and validation and test share 1,310. The six held-out podcast episodes yield 6,768 queries from 14 unseen listeners. Fusion outperformed either branch on all three metrics (C4, Table G4). The full-pool candidates still include seen content.

For Etard and Weissbart, we trained source models on the 56 shared electrodes described in the paper and evaluated them frozen. The target datasets were excluded from training and fusion-weight selection (C4, Table G5). These tests provide evidence of transfer across separately collected corpora, while the held-out episodes test unseen content within the source dataset.

Q3. Repeated training of LF, HF, and fusion

C1 reports the three-seed results for the core waveform models and their fusion. Fusion ranked first, followed by LF and HF, in all three runs. We kept the 54/14/17 listener split and trained on 468 recordings after excluding the two earphone-control recordings. All three seeds used the same 18,680 test queries and candidate sets from 17 listeners, and each fusion paired LF and HF from the same seed. The other main HF representations have yet to be run with all three seeds. The historical seed-0 results are separate from the three-seed statistics.

Q4. Temporal-group differences between HF and LF

The in-ear and out-of-ear comparison in C3 leaves a possible contribution from EMG. Prior work reports that scalp EMG contamination from temporalis activity tends to be stronger at temporal and nearby peripheral sites [2]. In our electrode-group analysis, the temporal group had the lowest accuracy for the HF two-bin-phase model, while the parietal group performed better. This decoding pattern differs from the typical spatial distribution of temporalis EMG amplitude. Our analysis is descriptive, and the underlying neural encoding remains unclear.

[2] Goncharova, Irina I., et al. "EMG contamination of EEG: spectral and topographical characteristics." Clinical neurophysiology 114.9 (2003): 1580-1593.

Reviewer_FrNAOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper investigates whether single five-second windows of high-frequency EEG support speech-segment retrieval and complement low-frequency decoding. It independently trains LF and HF EEG–audio matching pipelines, then fuses them by combining their scores. It also compares waveform, continuous-phase, two-bin-phase, and log-power inputs. It validates with participant-disjoint retrieval on SparrKULee, frozen evaluation on external datasets, and timing, frequency-band, and sensor-subset analyses. The paper demonstrates improved retrieval through LF/HF fusion and substantial retention of HF retrieval performance using phase-derived inputs.

Critical strengths
  1. Demonstrated performance improvement from combining LF and HF scores. Full-pool Top-1 accuracy increases from 8.65% to 15.39% on 17 unseen participants. This shows that the two trained branches provide complementary information for retrieval.

  2. External validation strengthens the empirical result. Frozen evaluation on two public external datasets reproduces the improvement from fusion, providing evidence of cross-dataset transfer.

Critical weaknesses
  1. Unclear motivation, limited methodological novelty, and missing matched baselines. The paper acknowledges existing short-window HF matching but does not clearly connect the remaining challenge to the proposed design. The central strategy of combining LF and higher-frequency EEG–audio matching scores is already present in [r1], which evaluates five-second, five-candidate matching using LF and 35–150 Hz gamma-band decoders. The matched baselines only replace the LF EEG encoder. Without evaluating prior HF and combined-frequency methods under the same protocol, it remains unclear what the proposed design adds to the existing approaches.

  2. Insufficient controls and limited insight into the main findings. Design choices and fusion gain: Beyond matching temporal scales, the rationale for using different audio representations and encoder configurations in the LF and HF branches is insufficient. Adding the HF branch changes the EEG frequency range, audio representation, encoder configuration, and model count together. It is therefore unclear how much of the improvement specifically requires HF EEG rather than audio-feature diversity or ensembling. An LF-only ensemble and LF–wav2vec + LF–auditory-subband fusion would help isolate the contribution.

Phase-related finding: The main fusion experiment uses raw HF waveforms, while phase-derived inputs are evaluated separately. It is unclear whether phase-based fusion retains the improvement. The phase-versus-power comparison is also not repeated on the external datasets, leaving the generalizability concerns.

Additional analyses: The frequency-band and sensor-subset results in Sections 3.3.2 and 3.3.3 remain largely descriptive. The paper does not clearly relate the stronger 50–100 Hz retrieval or numerically poorer temporal-subset performance to established auditory responses, or explain how these interpretations can be distinguished from referencing and signal-quality effects. A supported neurophysiological interpretation would make these analyses more informative.

Questions for authors
  1. What limitation of existing short-window HF matching motivates the proposed design? What are the methodological and empirical advances over [r1], and could the closest HF and combined-frequency methods be evaluated as baseline?
  2. What motivates the different audio representations and encoder configurations for the LF and HF branches beyond matching temporal scales? Which of these choices are important to the reported improvement?
  3. How does independently trained LF + HF score fusion compare with an LF-only ensemble and LF–wav2vec + LF–auditory-subband fusion under comparable training budgets and appropriate temporal processing?
  4. Does replacing the raw HF waveform branch with continuous-phase or two-bin-phase input retain the improvement over LF-only retrieval? Is the phase-versus-power result also reproduced on the public external datasets under frozen evaluation?
  5. What neurophysiological interpretation is supported by the frequency-band and sensor-subset results? In particular, how do the stronger 50–100 Hz retrieval and numerically poorer temporal-subset performance relate to established auditory responses, and what evidence distinguishes this interpretation from effects of referencing or signal quality?

[r1] Thornton, M., Auernheimer, J., Jehn, C., Mandic, D., & Reichenbach, T. (2024). Detecting gamma-band responses to the speech envelope for the ICASSP 2024 Auditory EEG Decoding Signal Processing Grand Challenge. In 2024 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) (pp. 55–56). IEEE. DOI: 10.1109/ICASSPW62465.2024.10626244.

1 comment

Thank you for highlighting the fusion gains and external validation. We refer to the general response for shared experiments and address your specific points below.

W1. Motivation and closest baseline

Thornton et al. [1] studied LF/gamma score fusion on five-second windows. We added a Thornton-style baseline using the same splits and candidate sets. See C1 for the three-seed five-way results. Full-pool Top-1 and Top-10 were 15.72 ± 0.75% and 38.01 ± 0.54% for our method, and 11.16 ± 0.36% and 31.26 ± 0.35% for the baseline (mean ± sample SD).

W2. Fusion controls and interpretation

W2-1. Same-band controls. Both cross-band pairs outperformed LF + LF and HF + HF at the same ensemble size (C1, Table G1). The LF and HF branches differ in EEG frequency band, audio representation, and encoder, so the gain cannot be attributed entirely to the frequency band.

W2-2. Phase fusion. Fusing either continuous phase or two-bin phase with LF improved test performance over LF alone (C2, Table G2). Table G2 uses existing frozen scores from seed 0 and 470 training recordings. The three-seed runs in C1 use 468 training recordings.

W2-3. Additional analyses. Section 3.3 reports descriptive observations. The physiological explanation in the manuscript remains incomplete, and the underlying causes remain unclear (see Q5).

Q1. Advances over existing short-window matching work

We examine which HF representations retain retrieval information and complement LF. We compare waveform, continuous phase, two-bin phase, and power, and evaluate retrieval among roughly 1,000 candidates and frozen transfer across datasets (C2, C4). On the source dataset, full-pool Top-1 was 4.89% for continuous phase, 4.34% for two-bin phase, and 0.27% for power. The phase inputs retain retrieval information, and its neural origin remains undetermined. See W1 for the closest baseline.

Q2. Choice of audio representations and encoders

The LF branch uses contextualized wav2vec features. The HF branch uses auditory subband features sampled at 1,024 Hz to retain rapid modulations. The current experiments do not quantify the individual contributions of the audio representation and encoder.

Q3. Two-model ensembles and fusion of two LF representations

Table G1 compares two-model fusions using same-band pairs across seeds 42 and 43 and cross-band pairs matched by seed. In a separate seed-0 comparison, we held the LF–wav2vec branch fixed and replaced HF–subband with LF–subband. Five-way Top-1 / full-pool Top-1 / full-pool Top-10 fell from 76.88% / 15.39% / 38.43% to 71.11% / 9.89% / 29.62%.

Q4. Phase fusion and representations on external datasets

We selected phase-fusion weights using only the source validation set. Fusing either phase representation with LF improved test performance over LF alone (Table G2). We evaluated frozen source models on the external datasets and excluded target data from training and weight selection. Continuous phase outperformed power on both public datasets (C4, Table G5).

Q5. Physiological interpretation of the band and sensor results

In the electrode-group analysis, the temporal group had the lowest accuracy for the HF two-bin-phase model, while the parietal group performed better. The frequency-following response is one possible interpretation. Our analyses remain descriptive and do not establish the underlying neural encoding mechanism.

Reference

[1] Thornton, M., et al. Detecting Gamma-Band Responses to the Speech Envelope for the ICASSP 2024 Auditory EEG Decoding Signal Processing Grand Challenge. ICASSPW, 2024, pp. 55–56.

Reviewer_fRUFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether high-frequency scalp EEG can be used for continuous speech segment retrieval and provide complementary information to low-frequency EEG. The authors train separate low-frequency and high-frequency EEG–speech matching branches, perform retrieval via score fusion, and compare input representations including high-frequency waveform, continuous phase, binary phase, and log power. On 17 unseen subjects, the high-frequency branch achieves 5.62% Top-1 accuracy with a large candidate pool; after fusion, accuracy improves from 8.65% (low-frequency branch) to 15.39%. Fusion gains are also observed on two public external datasets.

Critical strengths
  1. Valuable research question. Compared with existing binary matching tasks, this paper further examines whether high-frequency EEG can identify corresponding speech from a large candidate pool and assesses its complementary role to low-frequency systems. The experimental objective is clear.

  2. Main results are reasonably convincing. The authors adopt subject-disjoint splits, compare low-frequency, high-frequency, and fused results under identical queries and candidate sets, and conduct frozen-model tests on external datasets, supporting the transferability of the results.

  3. Comprehensive analyses with cautious interpretation boundaries. Experiments on phase, power, frequency bands, time offsets, and electrode subsets provide multiple perspectives. The authors also explicitly note that binary phase is equivalent to waveform sign and that electrode-subset results cannot be used for source localization; these qualifications are reasonable.

Critical weaknesses
  1. The effects of EEG frequency band versus speech representation are not sufficiently disentangled. The low-frequency branch uses wav2vec 2.0 features, whereas the high-frequency branch uses an auditory subband representation preserving rapid fluctuations; the two also differ in input temporal resolution and encoding design. Thus, the current results demonstrate that the two complete systems are complementary, but are insufficient to determine how much of the fusion gain originates from high-frequency EEG itself. Cross-pairing controls and fusion of two low-frequency models are recommended to rule out speech feature differences and general ensemble effects.

  2. The source of the high-frequency retrievable signal still lacks targeted validation. Although artifact processing is performed, existing experiments cannot fully exclude residual EMG, stimulus-related electrical or acoustic crosstalk, and other factors. Millisecond-level alignment sensitivity is consistent with rapid neural responses but may also arise from stimulus-locked non-neural signals. Additional time-structure-preserving mismatch controls, response latency analyses, and feasible artifact sensitivity checks are recommended to strengthen the assessment of the high-frequency EEG signal source.

  3. Interpretation of phase versus power comparisons remains constrained by representation construction. Computing Hilbert phase and power directly on broadband high-pass signals is not equivalent to extracting these quantities per frequency band. The poor performance of broadband log power does not sufficiently demonstrate that power information per se lacks retrieval value. The authors have limited their conclusions to the current modeling protocol, but adding per-subband power representations and phase-scrambling controls that preserve the power spectrum would more strongly support the core interpretation.

  4. Cross-subject generalization and cross-stimulus generalization need further distinction. The paper clearly specifies subject-disjoint splits but does not clearly state whether stories, speakers, or identical audio segments are shared across training, validation, and test sets. Shared stimuli do not negate the unseen-subject results but affect their scope of applicability. It is recommended to explicitly report stimulus overlap and, where possible, provide stimulus-disjoint evaluations.

  5. Training stability and reproducibility details could be strengthened. The appendix states that formal experiments use a single random seed. Subject-level statistical tests cannot reflect training randomness; multiple runs should be added for the main fusion results and representation comparisons. In addition, high-frequency filter parameters, line-noise handling, and the specific offset settings in training and test phases of the time-offset experiments should be clarified.

Questions for authors
  1. Do the different data splits share identical stories or audio segments? If so, are high-frequency retrieval capability and fusion gains still observed under stimulus-disjoint conditions?

  2. When low-frequency and high-frequency EEG use the same type of speech representation, does the high-frequency branch still provide significant gains? How much improvement does fusion of two low-frequency branches yield?

  3. Are there response latency or other control experiments that help distinguish neural responses from stimulus-related crosstalk? How much do existing artifact processing procedures affect the main results?

1 comment

Thank you for your feedback. The general response (C1–C4, Tables G1–G5) covers the shared results. We address your specific points below.

Weaknesses

W1. EEG frequency band and speech representation

Cross-band fusion outperformed two-model same-band fusion (C1, Table G1), supporting the complementarity of the LF and HF branches.

W2. Sources of the HF signal and residual artifacts

With the earphones out of the ears, the frozen models performed close to the 20% five-way chance level (C3, Table G3). The two recordings came from the original training set and used the same participant and story. We evaluated 56 non-overlapping five-second windows spanning 10 to 290 s in each recording.

This result provides additional evidence that performance varies with the listening condition. Residual EMG, stimulus-related electrical or acoustic crosstalk, and other factors remain possible contributors and warrant further study.

W3. Broadband phase and power construction

Computing phase and power from a broadband signal is a limitation. During the rebuttal period, we could not evaluate per-subband power or a phase-scrambling control that preserves the power spectrum. We leave these analyses for future work.

W4. Generalization across listeners and stimuli

The training, validation, and test listeners are disjoint. Of 13 test stimulus IDs, seven also appear in training or validation and six podcast episodes are absent from both. This subset includes 6,768 queries from 14 unseen listeners. Fusion outperformed either branch on all three metrics (C4, Table G4).

W5. Training stability and implementation details

We kept the 54/14/17 listener split and trained LF and HF independently with seeds 42, 43, and 44. We tested each model using its best validation checkpoint. Fusion improved performance in all three runs. See C1 for means and sample standard deviations.

HF preprocessing uses a symmetric 1,029-tap Kaiser FIR high-pass filter with a nominal cutoff of approximately 50 Hz and a design cutoff of 52.5 Hz. We compensate for the 514-sample group delay and trim 10 s from each end. For each offset condition, we draw a fixed EEG offset per recording in the training, validation, and test sets and retrain a separate model. The audio stays fixed.

Questions

Q1. Shared stimuli and results on unseen content

Seven of the 13 test stimulus IDs also occur in training or validation. C4 and Table G4 report retrieval for the six episodes absent from both splits.

Q2. Matched speech representations and fusion of two LF models

Table G1 compares two-model fusions using same-band pairs across seeds 42 and 43 and cross-band pairs matched by seed. In a separate seed-0 comparison, we held the LF–wav2vec branch fixed and replaced HF–subband with LF–subband. Five-way Top-1 / full-pool Top-1 / full-pool Top-10 fell from 76.88% / 15.39% / 38.43% to 71.11% / 9.89% / 29.62%.

Q3. Response latency, crosstalk, and artifact processing

See C3 for the timing-sensitivity and earphone controls. We lack an independent measure of acoustic arrival at the ear and direct EMG measurements. We have yet to establish what drives the latency effects or run matched comparisons before and after additional artifact processing. The contribution of residual artifacts therefore remains unquantified.

Comments on the paper

2 comments
Area_Chair_uXAXOpenReview ↗

Please Engage in the Author–Reviewer Discussion

Dear Reviewers,

As the author–reviewer discussion period is approaching its deadline (November 18, AoE), I would like to kindly remind you to review the authors' rebuttals and actively engage in the discussion as soon as possible.

Please address any outstanding questions or concerns and update your assessments and ratings if appropriate.

Thank you very much for your time and efforts!

Best regards, Area Chair

AuthorsOpenReview ↗

Rebuttal: General Response

Thank you to all three reviewers for their feedback. We address common points below. Unless otherwise specified, results are from seed 0.

C1. Fusion gains and training stability

We trained LF and HF models independently with seeds 42, 43, and 44, then froze each model at its best validation checkpoint for testing. Fusion weights were selected using only the source validation set. Five-way Top-1 (mean ± sample SD) was 68.28 ± 0.47% for LF, 56.34 ± 0.40% for HF, and 76.73 ± 0.66% for fusion. Fusion outperformed LF in all three runs and exceeded the adapted Thornton baseline (72.15 ± 0.45%) evaluated with the same splits and candidates.

The same-band and cross-band comparisons in Table G1 support the complementarity of the LF and HF branches.

Table G1. Same-band and cross-band fusion on the same test set (%)

ConditionFive-way Top-1Full-pool Top-1Full-pool Top-10
LF (42) + LF (43)70.099.4828.28
HF (42) + HF (43)57.726.2618.18
LF (42) + HF (42)76.3715.0537.56
LF (43) + HF (43)77.4916.5338.61

C2. Phase representations

We evaluated what happens when continuous phase or two-bin phase replaces the HF waveform. Both phase inputs retain retrieval information under the current protocol, and fusing either with LF improves on LF alone.

Table G2. Phase fusion using existing frozen models (%)

Condition (seed 0)Five-way Top-1Five-way gain over LF (pp)Full-pool Top-1Full-pool Top-10
LF alone69.55—8.6527.43
LF + HF waveform76.88+7.3315.3938.43
LF + HF continuous phase76.56+7.0114.4537.11
LF + HF two-bin phase75.21+5.6613.3435.40

C3. Artifact controls and temporal alignment

We evaluated the trained models (seed 0) on two recordings from the original training set for the same participant and story, one with the earphones inserted and one with the earphones out of the ears. Five-way accuracy was 14.29%–21.43% with the earphones out, close to the 20% chance level even on this training-set recording. The models performed substantially better on the in-ear recording. This control shows that performance differs by listening condition. It leaves the contribution of residual crosstalk and EMG to the main results unquantified.

Table G3. Five-way Top-1 in the training-set earphone control (%)

ModelEarphones insertedEarphones out of the ears
LF94.6417.86
HF92.8614.29
LF + HF10021.43

On the source dataset, adding random recording-level offsets of ±10 ms and retraining reduced two-bin-phase full-pool Top-1 from 4.34% to 1.39%, showing sensitivity to relative EEG–audio alignment. The training-set earphone control cannot distinguish neural activity, device coupling, and EMG contributions.

C4. Unseen content and cross-dataset generalization

The training, validation, and test sets contain disjoint listeners. Of 13 test stimulus IDs, seven also occur in training or validation. The other six are podcast episodes absent from both. This subset comprises 6,768 queries from 14 unseen listeners, with positive targets drawn from the six held-out episodes. Full-pool retrieval uses the original candidate pools for comparability.

Table G4. Retrieval of targets from unseen podcast episodes (%)

ModelFive-way Top-1Full-pool Top-1Full-pool Top-10
LF70.669.4029.18
HF54.052.5712.07
LF + HF77.1915.1338.27

Table G5. Frozen cross-dataset evaluation (%), reported as five-way Top-1 / full-pool Top-1 / full-pool Top-10.

HF representationEtardWeissbart
Waveform (W)36.01 / 3.68 / 16.9736.82 / 1.58 / 9.24
Continuous phase (Φ)35.72 / 3.22 / 16.3137.86 / 1.98 / 10.16
Two-bin phase (Q)33.30 / 2.33 / 14.9837.29 / 1.89 / 10.01
Log power (P)24.68 / 1.04 / 8.4824.40 / 0.51 / 4.02

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.