- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes BiGMNet: Bi-directional Griffin Adapter with Multi-Frame Prediction Network to improve target speaker extraction in multi-speaker audio using EEG as an auxiliary cue. The proposed model includes three architectural modifications compared to prior methods (1) the Bi-directional Griffin Adapter: in order to perform bi-directional temporal refinement of fused audio-EEG embeddings, (2) Direction-aware multi-frame prediction (DA-MFP): module that provides an auxiliary loss by predicting future/prior latent states from the bi-directional embeddings, which provides additional target-speech context to fused embeddings, and (3) Hybrid Muon-AdamW optimization: which combines Muon and AdamW optimizers for specific parameter groups in order to account for heterogeneous optimization behaviors. Improvements with the proposed architecture are shown on two datasets (1) MM-AAD and (2) AVED datasets with models trained per-subject. Reported metrics include, SDR and SI-SDR for correlation with target speaker speech and STOI, PESQ and ESTOI for speech quality. Results indicate that the proposed method shows marginal improvements over prior methods.
Critical strengths
The general idea of using a related biological (neural) signal as an auxiliary cue to process another signal (audio) is interesting. But I find the critical technical strengths of this work to be limited.
Critical weaknesses
These are the main weaknesses: What is the motivation for proposed architectural changes? The authors state “bi-directional temporal modeling” for proposed Bi-direction Griffin Adapter. However, prior works such as TIDENet (https://ojs.aaai.org/index.php/AAAI/article/view/40514) contain “bi-directional temporal modeling. TIDENet is composed of a cross-attention multi-layer module that fuses both EEG and speech embeddings. Following this, the DPRNN block is composed of bi-directional LSTM blocks which performs “bi-directional temporal modeling”. Authors should clarify how the proposed architecture differs.
Cross-subject/Cross-trial generalization: The evaluation setting of the proposed method does not test cross-subject or cross-trial generalization. Given that a given trial is split between both train and test, it is plausible that models learn trial specific information and do not generalize across unseen-trials. This would severely limit the applicability of proposed models, and any stated performance improvements.
Questions for authors
Architectural details and Dataset size: Authors should provide information regarding the model size, number of samples in the training set of data. Given that a specific trial information is both in training and testing set, authors should provide information regarding over-fitting of model and sensitivity of model to learning trial-specific information. As stated in KULueven dataset page (while this disclaimer specifically mentions classification tasks, it would be possible for similar issues to extend to target speaker extraction), splitting within-trial is not preferred method for model evaluation: https://zenodo.org/records/4004271 Avoid splitting data from the same experimental trial into training and testing segments. Classifiers can detect whether a test segment belongs to a specific trial if other, non-overlapping segments from that trial are in the training set.
Baseline Performance: Manuscript mentions that baseline numbers are taken from literature. Could the authors clarify whether they re-produce MM-AAD results for TIDENet, IFENet as the original papers do not have results for that dataset?
Audio Samples: Could the authors provide extracted audio samples in order for the reviewers to be able to assess performance of proposed method and baselines?
Architecture, evaluation protocol, and baseline provenance
Thank you for the careful review.
1. Motivation and distinction from TIDENet (W1).
TIDENet already applies bidirectional DPRNN processing after audio-EEG fusion. BiGMNet couples a Griffin-based residual refiner with explicit supervision of its directional states. Its independently parameterized forward/backward stacks combine gated recurrence and local attention, then align, concatenate, normalize, and residually project their outputs before mask estimation. DA-MFP supervises the states before concatenation: forward states predict future clean-speech latents, while backward states predict past latents in reverse temporal order. The motivation is to supplement waveform-level supervision with direction-specific intermediate targets on EEG-conditioned representations. Hybrid Muon-AdamW is a supporting training strategy.
Under AdamW without DA-MFP, replacing AVConv with the complete adapter increases SI-SDR from 8.94 to 9.19 dB on MM-AAD and from 8.84 to 8.96 dB on AVED. With the adapter fixed, DA-MFP adds 0.41/0.05 dB under AdamW on MM-AAD/AVED. These comparisons support the complete adapter and auxiliary objective within our backbone.
2. Evaluation scope, data volume, and model size (W2; Q1).
We evaluate ten participants per dataset using subject-specific models. AVED contains 16 trials of 152 s per participant, and MM-AAD contains 20 trials per participant. Each trial is split chronologically into 80%/10%/10% training/validation/test portions before aligned 2-s segmentation. Checkpoints are selected by validation SI-SDR.
The partitions contain different time segments but share trial identity, speaker identities, and broader story context. The reported results therefore concern within-trial extraction. Sensitivity to trial-specific information and generalization to unseen trials or participants require separate evaluation. The manuscript specifies recording-level data volume and computational cost; parameter counts and exact split-wise segment totals are not tabulated.
3. Baseline provenance and the size of the gains (Q2).
TIDENet was reproduced on both MM-AAD and AVED using the same participants, splits, and EEG/audio preprocessing as BiGMNet, with each method retaining its own training configuration. IFENet and the other baseline scores are literature-reported values.
BiGMNet reaches 9.76/10.90 dB SI-SDR versus TIDENet's 9.71/10.74 dB on MM-AAD/AVED, respectively. The comparison combines numerically close extraction performance with lower reported computation: 12.31 versus 57.67 G MACs, approximately 78.7% fewer operations. These MAC counts describe computation, separately from parameter count and measured runtime.
4. Audio examples (Q3).
Audio examples would complement the quantitative assessment using SDR, SI-SDR, PESQ, STOI, and ESTOI. Extracted-audio examples are not provided with this response.
We will clarify the related-work discussion and contribution statements to make the architectural distinction from TIDENet explicit.