acceptodds
Under review as a conference paper at ICLR 2027

BiGMNet: Bidirectional Representations with Predictive Supervision for EEG-Guided Target Speaker Extraction

Abstract

Brain-assisted target speaker extraction aims to recover the attended target speech from competing speech, using electroencephalography (EEG) as the sole auxiliary cue. However, architectures relying on local post-fusion mappings and waveform supervision lack explicit coupling between temporal organization and direct supervision of target-speech dynamics, while uniform elementwise updates overlook the distinct roles of internal matrices and other parameters. To address these issues, this paper proposes Bidirectional Griffin Adapter with Multi-Frame Prediction Network (BiGMNet), an EEG-conditioned encoder–mask–decoder framework for target speaker extraction. The method introduces three coordinated components: the Bidirectional Griffin Adapter (BiGriffinAdapter) for bidirectional temporal refinement, direction-aware multi-frame prediction (DA-MFP) for supervising directional intermediate states, and Hybrid Muon–AdamW optimization for parameter-role-aware updates. BiGriffinAdapter combines bidirectional gated recurrence and local attention to refine fused representations for mask estimation, and training-only DA-MFP supervises its pre-concatenation forward and backward states through sequential prediction of future and past continuous clean-speech latents, respectively. Hybrid Muon–AdamW approximately orthogonalizes updates to selected internal matrices and applies elementwise adaptive updates to the remaining parameters according to their computational roles. This paper evaluates subject-specific models on MM-AAD and AVED with ten participants per dataset through EEG-only baseline comparisons and component ablations. BiGMNet achieves 9.76 and 10.90 dB SI-SDR, exceeding IFENet's reported scores by 1.57 and 2.14 dB, respectively, with 12.31 G MACs, 15.5% fewer than IFENet. These results support directly supervising EEG-conditioned temporal states to improve target speaker extraction, with the optimization rule chosen according to parameter roles.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Target Speaker Extraction, Speech Separation, Electroencephalography (EEG), Auditory Attention, Brain–Computer Interfaces, Multimodal Learning, Representation Learning.

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_CjXLOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes BiGMNet: Bi-directional Griffin Adapter with Multi-Frame Prediction Network to improve target speaker extraction in multi-speaker audio using EEG as an auxiliary cue. The proposed model includes three architectural modifications compared to prior methods (1) the Bi-directional Griffin Adapter: in order to perform bi-directional temporal refinement of fused audio-EEG embeddings, (2) Direction-aware multi-frame prediction (DA-MFP): module that provides an auxiliary loss by predicting future/prior latent states from the bi-directional embeddings, which provides additional target-speech context to fused embeddings, and (3) Hybrid Muon-AdamW optimization: which combines Muon and AdamW optimizers for specific parameter groups in order to account for heterogeneous optimization behaviors. Improvements with the proposed architecture are shown on two datasets (1) MM-AAD and (2) AVED datasets with models trained per-subject. Reported metrics include, SDR and SI-SDR for correlation with target speaker speech and STOI, PESQ and ESTOI for speech quality. Results indicate that the proposed method shows marginal improvements over prior methods.

Critical strengths

The general idea of using a related biological (neural) signal as an auxiliary cue to process another signal (audio) is interesting. But I find the critical technical strengths of this work to be limited.

Critical weaknesses

These are the main weaknesses: What is the motivation for proposed architectural changes? The authors state “bi-directional temporal modeling” for proposed Bi-direction Griffin Adapter. However, prior works such as TIDENet (https://ojs.aaai.org/index.php/AAAI/article/view/40514) contain “bi-directional temporal modeling. TIDENet is composed of a cross-attention multi-layer module that fuses both EEG and speech embeddings. Following this, the DPRNN block is composed of bi-directional LSTM blocks which performs “bi-directional temporal modeling”. Authors should clarify how the proposed architecture differs.

Cross-subject/Cross-trial generalization: The evaluation setting of the proposed method does not test cross-subject or cross-trial generalization. Given that a given trial is split between both train and test, it is plausible that models learn trial specific information and do not generalize across unseen-trials. This would severely limit the applicability of proposed models, and any stated performance improvements.

Questions for authors

Architectural details and Dataset size: Authors should provide information regarding the model size, number of samples in the training set of data. Given that a specific trial information is both in training and testing set, authors should provide information regarding over-fitting of model and sensitivity of model to learning trial-specific information. As stated in KULueven dataset page (while this disclaimer specifically mentions classification tasks, it would be possible for similar issues to extend to target speaker extraction), splitting within-trial is not preferred method for model evaluation: https://zenodo.org/records/4004271 Avoid splitting data from the same experimental trial into training and testing segments. Classifiers can detect whether a test segment belongs to a specific trial if other, non-overlapping segments from that trial are in the training set.

Baseline Performance: Manuscript mentions that baseline numbers are taken from literature. Could the authors clarify whether they re-produce MM-AAD results for TIDENet, IFENet as the original papers do not have results for that dataset?

Audio Samples: Could the authors provide extracted audio samples in order for the reviewers to be able to assess performance of proposed method and baselines?

1 comment
AuthorsOpenReview ↗

Architecture, evaluation protocol, and baseline provenance

Thank you for the careful review.

1. Motivation and distinction from TIDENet (W1).

TIDENet already applies bidirectional DPRNN processing after audio-EEG fusion. BiGMNet couples a Griffin-based residual refiner with explicit supervision of its directional states. Its independently parameterized forward/backward stacks combine gated recurrence and local attention, then align, concatenate, normalize, and residually project their outputs before mask estimation. DA-MFP supervises the states before concatenation: forward states predict future clean-speech latents, while backward states predict past latents in reverse temporal order. The motivation is to supplement waveform-level supervision with direction-specific intermediate targets on EEG-conditioned representations. Hybrid Muon-AdamW is a supporting training strategy.

Under AdamW without DA-MFP, replacing AVConv with the complete adapter increases SI-SDR from 8.94 to 9.19 dB on MM-AAD and from 8.84 to 8.96 dB on AVED. With the adapter fixed, DA-MFP adds 0.41/0.05 dB under AdamW on MM-AAD/AVED. These comparisons support the complete adapter and auxiliary objective within our backbone.

2. Evaluation scope, data volume, and model size (W2; Q1).

We evaluate ten participants per dataset using subject-specific models. AVED contains 16 trials of 152 s per participant, and MM-AAD contains 20 trials per participant. Each trial is split chronologically into 80%/10%/10% training/validation/test portions before aligned 2-s segmentation. Checkpoints are selected by validation SI-SDR.

The partitions contain different time segments but share trial identity, speaker identities, and broader story context. The reported results therefore concern within-trial extraction. Sensitivity to trial-specific information and generalization to unseen trials or participants require separate evaluation. The manuscript specifies recording-level data volume and computational cost; parameter counts and exact split-wise segment totals are not tabulated.

3. Baseline provenance and the size of the gains (Q2).

TIDENet was reproduced on both MM-AAD and AVED using the same participants, splits, and EEG/audio preprocessing as BiGMNet, with each method retaining its own training configuration. IFENet and the other baseline scores are literature-reported values.

BiGMNet reaches 9.76/10.90 dB SI-SDR versus TIDENet's 9.71/10.74 dB on MM-AAD/AVED, respectively. The comparison combines numerically close extraction performance with lower reported computation: 12.31 versus 57.67 G MACs, approximately 78.7% fewer operations. These MAC counts describe computation, separately from parameter count and measured runtime.

4. Audio examples (Q3).

Audio examples would complement the quantitative assessment using SDR, SI-SDR, PESQ, STOI, and ESTOI. Extracted-audio examples are not provided with this response.

We will clarify the related-work discussion and contribution statements to make the architectural distinction from TIDENet explicit.

Reviewer_skpDOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This study proposes BiGMNet, an EEG-guided target speaker extraction framework designed to improve the temporal modeling of fused audio–EEG representations. The approach combines three components: (1) BiGriffinAdapter, introducing bidirectional recurrent and local-attention modeling following audio–EEG fusion; (2) Direction-Aware Multi-Frame Prediction (DA-MFP), an auxiliary objective supervising forward and backward intermediate representations by predicting future and past clean-speech latents; and (3) a hybrid Muon–AdamW optimization strategy. The method is evaluated on MM-AAD and AVED datasets, using subject-specific models for ten participants per dataset. BiGMNet is slightly outperforming the strongest reported baseline while requiring fewer multiply-accumulate operations (MACs).

Critical strengths
  • EEG-guided target speaker extraction addresses an interesting problem at the intersection of neural decoding and speech separation, possibly expanding the applications of EEG to hearing assistance.

  • The introduction of an auxiliary prediction objective targeting intermediate representations is an interesting design choice that complements the final waveform reconstruction loss. The use of clean-speech latents as supervision provides an additional learning signal without increasing inference-time computational requirements.

  • Efficiency. The proposed architecture achieves competitive extraction performance with comparatively low computational requirements.

Critical weaknesses
  • The proposed framework largely combines existing techniques: a Griffin-based bidirectional architecture, an auxiliary latent prediction objective inspired by multi-token prediction, and existing optimization algorithms. While their combination is reasonable, the methodological novelty appears limited, and the work does not sufficiently establish what fundamentally distinguishes its proposed mechanisms from existing approaches. Overall, the contribution seems relatively incremental for a standalone ICLR paper.

  • The improvements over the strongest baseline, TIDENet, are only 0.05 dB and 0.16 dB SI-SDR on MM-AAD and AVED respectively. These differences are small relative to the reported standard deviations, and their statistical significance is not established. While the model is considerably more computationally efficient than TIDENet, its computational requirements are relatively close to IFENet, against which it demonstrates larger performance gains. It is also unclear whether all reported baselines use identical participants, data splits, and evaluation protocols.

  • The evaluation relies exclusively on quantitative signal-level and perceptual metrics, without providing qualitative examples or directly assessing whether the reconstructed speech preserves the linguistic content of the attended speaker. Although the provided metrics are informative, they do not fully establish whether the extracted speech is intelligible or correctly captures the target utterance. For example, passing the extracted audio through a pretrained ASR model like Whisper and comparing the resulting transcriptions with the attended speaker's ground-truth speech could provide a useful complementary assessment of linguistic intelligibility.

  • The paper does not establish how strongly the model actually relies on EEG-derived attention cues. An audio-only baseline or experiments involving zeroed or mismatched EEG would help demonstrate whether the model meaningfully exploits neural attention information rather than primarily relying on acoustic features.

  • The manuscript is difficult to follow in several places, particularly in its explanation of the proposed architecture and auxiliary prediction mechanism. The presentation is dense and there's limited intuition accompanying the methodological details. The practical significance of the problem and the proposed improvements could be articulated more clearly.

Questions for authors
  • How does BiGMNet perform with zeroed or randomly mismatched EEG? Could the authors include an audio-only baseline to demonstrate the contribution of neural attention cues?
  • Could the authors provide qualitative examples of extracted speech or evaluate the outputs using a pretrained ASR model and report transcription error rates against the target utterances?
  • Are the improvements over TIDENet and IFENet statistically significant?
3 comments
AuthorsOpenReview ↗

Temporal supervision and EEG conditioning

Thank you for the careful review.

1. Technical contribution and intuition (W1, W5)

The application goal is to recover the listener's attended speech from competing talkers, using EEG as the auxiliary attention cue. BiGMNet couples bidirectional refinement of the fused audio–EEG representation with direction-specific supervision of its intermediate states. The extraction path is: mixture/EEG encoding → aligned fusion → BiGriffinAdapter → mask estimation → waveform decoding. Fusion brings together acoustic content and the EEG conditioning cue; the adapter then models the temporal evolution of this conditioned representation before mask estimation.

The technical distinction lies in the location of refinement and its coupling to predictive supervision. Independently parameterized forward and backward stacks combine gated recurrence with local attention. Their temporally aligned states are concatenated, normalized, and projected through a residual connection. DA-MFP supervises these directional states before concatenation, assigning future clean-speech latent targets to the forward state and past targets to the backward state.

For intuition, at frame tt, the first forward prediction step combines the forward state with clean latent qtq_t to predict qt+1q_{t+1}; the backward step combines its state with qtq_t to predict qt−1q_{t-1}. Deeper steps combine the preceding prediction state with the next time-shifted clean latent in that direction. This teacher-forced chain provides direction-specific latent supervision to the EEG-conditioned states, complementing waveform-level reconstruction. The clean-target path and auxiliary predictors are removed at inference. Hybrid Muon–AdamW serves as a supporting training strategy.

The existing ablations provide the following evidence:

  • Under AdamW without DA-MFP, replacing AVConv with BiGriffinAdapter raises SI-SDR from 8.94 to 9.19 dB on MM-AAD and from 8.84 to 8.96 dB on AVED.
  • With K=3K=3, DA-MFP raises SI-SDR by 0.41/0.05 dB on MM-AAD/AVED under AdamW and by 0.31/1.38 dB under Hybrid Muon–AdamW.
  • In the appendix's five-participant MM-AAD comparison under Hybrid Muon–AdamW, increasing KK from 1 to 3 without AVConv raises SI-SDR from 9.37 to 9.86 dB.

These comparisons evaluate the complete adapter, auxiliary objective, and prediction-depth settings; they do not separately isolate bidirectionality or the direction–target assignment.

2. Contribution of EEG (W4; Q1)

EEG is the sole auxiliary conditioning modality, entering fusion before temporal refinement and mask estimation; the training target is the attended speech. Models are subject-specific, with chronological 80%/10%/10% splits within each trial before aligned 2-second segmentation.

On AVED, controls with an early BiGMNet version used the same participants and splits as the main evaluation. Both a separately retrained audio-only model and a model trained with correctly matched EEG but tested with EEG randomly mismatched across participants performed worse than matched EEG–audio extraction. These qualitative observations concern the early model and support the utility of matched EEG in that setting. Zeroed-EEG results and quantitative EEG-control results for the submitted model are not reported.

We will clarify the manuscript's opening computation path, add a short time-index example for DA-MFP, and organize the contribution around temporal refinement and directional supervision, with the hybrid optimizer as a supporting strategy.

AuthorsOpenReview ↗

Baseline comparison and intelligibility

3. Baseline protocol, performance, and significance (W2; Q3)

TIDENet was reproduced locally using the same participants, data splits, and EEG/audio preprocessing as BiGMNet; each method retained its own training configuration. BASEN, MSFNet, GALENet, M3ANet, and IFENet are quoted from the literature and retain their reported protocols. The shared-protocol comparison therefore applies to our TIDENet reproduction.

DatasetBiGMNet SI-SDRTIDENet SI-SDRIFENet SI-SDR (literature)
MM-AAD9.76 dB9.71 dB8.19 dB
AVED10.90 dB10.74 dB8.76 dB

BiGMNet's SI-SDR differences relative to TIDENet are +0.05/+0.16 dB on MM-AAD/AVED; the SDR differences are +0.25/+0.47 dB. Reported MACs are 12.31 G for BiGMNet versus 57.67 G for TIDENet, approximately 78.7% fewer operations. The empirical benefit is numerically close extraction performance with lower reported computation.

The IFENet comparison addresses the lower-cost baseline directly: BiGMNet's SI-SDR is higher by 1.57/2.14 dB on MM-AAD/AVED, with 12.31 G versus 14.57 G MACs, approximately 15.5% fewer reported operations. DA-MFP is training-only, so its auxiliary prediction computation is absent at inference. MACs describe operation counts rather than measured hardware speed.

The reported SD for our results summarizes variation across ten participants under one training seed (0). Statistical significance against TIDENet or IFENet has not been established by the reported analyses. In particular, the IFENet entries are literature aggregates rather than paired participant-level results from our evaluation.

4. Intelligibility and linguistic content (W3; Q2)

The evaluation includes STOI and ESTOI as objective intelligibility-related measures, alongside SDR, SI-SDR, and PESQ. The relevant scores are:

DatasetBiGMNet STOI / ESTOITIDENet STOI / ESTOI
MM-AAD0.91 / 0.810.90 / 0.81
AVED0.88 / 0.800.88 / 0.79

Listening examples and ASR transcription errors against the attended speaker's reference text would complement these signal-based measures with an assessment of the preservation of the target utterance's content. The current evaluation includes no listening examples or ASR results measured against reference transcripts.

We will make the provenance of each baseline explicit in the experimental description and distinguish reproduced results from literature-reported values.

Thank you for the responses. I appreciate the clarification around the baseline comparisons, particularly that TIDENet was reproduced using the same protocol as BiGMNet. However, my main concerns remain largely unresolved:

EEG dependence: The authors mention preliminary experiments on an earlier version of the model, but provide no quantitative evidence demonstrating the contribution of EEG conditioning to the presented architecture which seems to be the central premise of the work. As training and testing are performed on different portions of the same trials, there is a possibility that recurring speaker identities and acoustic conditions (amplitude, surroundings) may allow the model to exploit audio-related cues without necessarily relying on EEG signal

Significance: The improvements over TIDENet remain marginal (+0.05/+0.16 dB SI-SDR) and in the reply authors acknowledge that statistical significance hasn't been established. The results also come from just one seed.

Intelligibility/qualitative evaluation: While STOI and ESTOI provide useful estimates of speech intelligibility they don't directly assess whether the reconstructed speech matches the attended speaker's linguistic content. I would still consider qualitative audio examples and an ASR-based evaluation (the proposed passing of the extracted speech through Whisper and comparing the resulting transcriptions against the original target utterances) important complementary evidence. Without such evaluation it remains difficult to assess the practical quality of the reconstructed speech.

The explanation of the architecture in the response is helpful, but my concern regarding limited methodological novelty remains. The main contribution appears to be direction-specific predictive supervision added to an otherwise largely existing architecture, and the ablations do not establish the necessity of this particular design.

Finally, the proposed manuscript revisions do not appear to have been uploaded yet as the PDF I can download is identical to the original submission.

Overall, while the rebuttal clarifies several points, it does not provide new evidence to change my original assessment.

Reviewer_wkZKOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes BiGMNet for EEG-guided target speaker extraction. After fusing audio and EEG features, a bidirectional Griffin adapter is introduced, and direction-aware multi-frame prediction (DA-MFP) is used to supervise both forward and backward hidden states, together with a hybrid Muon–AdamW optimization strategy. The method is evaluated on MM-AAD and AVED, showing competitive extraction performance with low inference cost.

Critical strengths
  1. The motivation is clear. Combining temporal modeling after fusion with intermediate feature prediction supervision offers a reasonable way to improve EEG-conditioned speech representations.

  2. The DA-MFP branch is used only during training, providing auxiliary supervision without increasing inference cost, which is of practical value.

  3. Experiments cover two datasets and include ablations on the adapter, prediction supervision, and optimizer, as well as appendix analyses of module placement and prediction depth, giving a relatively complete component evaluation.

Critical weaknesses

1.The necessity of the direction-aware prediction design is not sufficiently demonstrated. Existing ablations mainly compare whether DA-MFP is used and cannot distinguish whether gains come from ordinary clean-speech feature supervision, multi-step prediction, or the directional correspondence of forward prediction of the future and backward prediction of the past. Controls such as current-frame reconstruction, unidirectional prediction, and direction swapping should be added to clarify the independent contribution of the core design.

2.The auxiliary prediction may rely too heavily on clean speech input. Eq. (10) uses the clean target latent as input at every prediction depth. Since adjacent speech frames are strongly correlated, the predictor may complete prediction mainly using this input rather than sufficiently constraining the EEG-conditioned hidden states. Experiments removing or shuffling directional hidden states and removing the clean latent input should be added to verify that the prediction task indeed drives the backbone to learn target-relevant temporal information.

3.The fairness of baseline comparison needs clarification. Section 4 states that external baseline results come from the literature, whereas Table 1 says they come from the literature or author reproduction. Please indicate the source of each result and explain whether participants, data splits, preprocessing, and training settings are consistent. Compared with the strongest baseline TIDENet, the SI-SDR improvement is only 0.05 dB on MM-AAD and 0.16 dB on AVED, so multi-seed results and appropriate statistical analysis are needed to judge whether the advantage is stable.

4.The contribution and reproducibility of the optimization strategy still need strengthening. On AVED, DA-MFP improves by only 0.05 dB under AdamW but by 1.38 dB under the hybrid optimizer, indicating that its effect clearly depends on the optimization configuration. Please explain this difference and add a well-tuned AdamW comparison, as well as details on Muon parameter grouping, momentum, and weight decay. In addition, MACs cannot directly reflect runtime speed; please report parameter count, actual inference time, and the context latency required for bidirectional processing.

Questions for authors

1.Clarification on baseline comparison: Which results in Table 1 come from the literature and which were reproduced by the authors? Do these methods use the same participants, data splits, and preprocessing? Compared with TIDENet, the SI-SDR improvements on the two datasets are 0.05 and 0.16 dB, respectively; are these differences stable across random seeds or subjects?

2.Clarification on the effectiveness of prediction supervision: Eq. (10) uses the clean target speech latent as predictor input. How do the authors confirm that the predictor relies on EEG-conditioned hidden states rather than mainly using adjacent clean speech frames to perform prediction? Are there control results that distinguish direction-aware prediction from ordinary latent reconstruction supervision?

3.Clarification on data splitting and target selection: Do the training, validation, and test portions within the same trial contain the same speakers and continuous speech content? Are there currently results without EEG, with shuffled EEG, or with replaced attended-target EEG that can confirm the model indeed selects the target speaker based on EEG?

4.Clarification on optimizer dependence: On AVED, DA-MFP brings only a 0.05 dB SI-SDR improvement under AdamW but a 1.38 dB improvement under the hybrid optimizer. Has this difference been verified through repeated experiments? Did AdamW receive a tuning budget comparable to that of the hybrid optimizer?

2 comments
AuthorsOpenReview ↗

Response to Reviewer (1/2): Bidirectional refinement and directional supervision

Thank you for the careful review.

1. Post-fusion refinement and direction-aware supervision (W1; Q2)

BiGMNet couples bidirectional refinement of the fused audio–EEG representation with direct supervision of its directional states. BiGriffinAdapter processes the fused sequence through independently parameterized forward and backward stacks, combining gated recurrence and local attention. Their aligned states are concatenated, normalized, and projected through a residual connection before mask estimation. Thus, temporal refinement acts on representations that already contain EEG conditioning.

The structural contribution is supported by the adapter ablation: under AdamW without DA-MFP, replacing AVConv with BiGriffinAdapter improves SI-SDR from 8.94 to 9.19 dB on MM-AAD and from 8.84 to 8.96 dB on AVED.

DA-MFP adds a distinct state–target correspondence before concatenation: the forward state predicts future clean-speech latents, and the backward state predicts past latents in reversed temporal order. Direction awareness is specified by both the supervised state and its temporal prediction target. This supplies direction-specific intermediate supervision in addition to waveform-level SI-SDR.

With K=3K=3 fixed, enabling DA-MFP improves SI-SDR by 0.41/0.05 dB on MM-AAD/AVED under AdamW and by 0.31/1.38 dB under Hybrid Muon–AdamW. The appendix also reports a five-participant MM-AAD depth comparison under the hybrid optimizer: increasing KK from 1 to 3 raises SI-SDR from 9.37 to 9.86 dB without AVConv. These results support the implemented auxiliary objective and its depth setting. Current-frame, unidirectional, and direction-swapped comparisons would further distinguish the independent contributions of reconstruction supervision, prediction depth, and directional correspondence.

2. Clean-latent inputs and the supervision pathway (W2; Q2)

Eq. (10) defines a teacher-forced prediction chain. Each chain is initialized from the corresponding EEG-conditioned directional state; subsequent depths combine the preceding prediction state with a time-shifted clean latent to predict the next latent in that direction. The clean latent therefore provides training-time guidance within a state-conditioned prediction task.

The auxiliary loss is attached to the pre-concatenation states, creating a gradient path to the directional branches and their upstream fusion and encoding modules. Clean latent targets are detached. At inference, the clean-target path and prediction modules are removed, and extraction uses mixture speech and synchronized EEG only.

These properties specify the implemented learning pathway and inference interface. The remaining question is the relative influence of the clean input and directional states: detaching targets and removing the branch at inference do not quantify that dependence.

3. EEG conditioning and evaluation protocol (Q3)

EEG is the sole auxiliary conditioning modality. Its encoded representation is aligned and fused with acoustic features before temporal refinement and mask estimation; the attended speech is the training target.

Models are subject-specific. Each trial is partitioned chronologically into 80%/10%/10% portions before aligned 2-second segmentation. The partitions share speaker identities and broader story context, so the results concern a within-trial protocol. Direct EEG interventions would quantify how target selection depends on the conditioning cue.

We will supplement the existing evidence with targeted directional and clean-input/hidden-state controls, together with EEG removal/shuffling tests and alternate-attention EEG where suitable paired recordings are available.

AuthorsOpenReview ↗

Response to Reviewer (2/2): Comparison protocol, optimization, and inference cost

4. Baseline provenance and the performance–cost comparison (W3; Q1)

BASEN, MSFNet, GALENet, M3ANet, and IFENet are quoted from the literature. TIDENet was reproduced using the same participants, data splits, and EEG/audio preprocessing as BiGMNet; training follows each method's own configuration. Section 4's blanket attribution to literature results is therefore incomplete. The shared-protocol clarification applies to the TIDENet reproduction, while the other baselines retain their original reported protocols.

The comparison with TIDENet is:

DatasetTIDENet SI-SDRBiGMNet SI-SDRSI-SDR gainSDR gain
MM-AAD9.719.76+0.05 dB+0.25 dB
AVED10.7410.90+0.16 dB+0.47 dB

The practical comparison includes computational cost: Table 1 reports 12.31 G MACs for BiGMNet versus 57.67 G for TIDENet, approximately 78.7% fewer reported operations. Our interpretation is numerically close extraction performance with substantially lower reported computation. The empirical benefit is this performance–computation balance, alongside the component gains in Table 2.

All our training runs use seed 0, and the reported SD describes variation across participants. These results do not establish training-seed stability or statistical significance of the small method differences.

5. Role of the hybrid optimizer (W4; Q4)

The main methodological contribution is post-fusion bidirectional refinement coupled with directional intermediate supervision. Hybrid Muon–AdamW serves as a supporting training strategy: Muon updates selected internal feature-mixing matrices, while AdamW updates the remaining groups, including convolutional layers, normalization parameters, decoder components, and DA-MFP modules.

The existing ablation already compares DA-MFP on/off under both optimizers, with K=3K=3 for both prediction-enabled configurations:

DatasetDA-MFP gain under AdamWDA-MFP gain under Hybrid
MM-AAD+0.41 dB+0.31 dB
AVED+0.05 dB+1.38 dB

DA-MFP has positive observed SI-SDR gains under both optimizers on both datasets. Its gain magnitude depends on the training configuration, particularly on AVED. This supports the utility of the complete training configuration while leaving the cause of the larger AVED interaction unresolved. The MM-AAD result also shows that hybrid optimization does not uniformly amplify the auxiliary objective.

6. Inference computation and latency (W4)

DA-MFP is training-only by construction. Removing its prediction chains and clean-target path leaves the extraction backbone unchanged and adds no auxiliary prediction computation at inference. This property is already part of the proposed method.

The reported 12.31 G MACs are also approximately 15.5% below IFENet's 14.57 G. These counts support the computational-cost comparison; actual execution time requires hardware measurements. Bidirectional processing uses future context within the input segment, so context acquisition must be reported separately from execution time. Evaluation on 2-second segments alone does not establish a streaming-latency guarantee.

We will strengthen these comparisons with multi-seed paired analysis, a validation-tuned AdamW comparison with a comparable tuning budget, complete baseline/optimizer documentation, and parameter-count, runtime, and context-latency reporting.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.