- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes RAMamba-Net for two-speaker auditory attention decoding (AAD). The architecture combines a Mamba-based band-aware EEG encoder, a spatio-temporal EOG encoder, bidirectional cross-attention, and a reliability-aware modulation (RAM) module that dynamically weights modalities based on feature descriptors and prediction agreement. Experiments and ablations are conducted on the AVGC and DTU datasets, including robustness tests under modality corruption.
Critical strengths
The evaluation includes useful internal controls. Concatenation, removal of RAM, eight combinations of RAM inputs, five initialization seeds, and modality-specific perturbations help assess the proposed fusion. Partitioning before windowing is also a sensible attempt to reduce overlap between evaluation and training data.
Critical weaknesses
-
Contradictory formulation of the unimodal supervision pathway. The claimed purpose of the unimodal loss is to preserve the discriminative ability of the modality encoders. However, Eq. (9) applies
stopgradto prior to mean pooling, Eq. (11) computes unimodal predictions from this detached descriptor, and Eq. (18) supervises those predictions. As formulated, gradients from Eq. (18) only update the linear classification heads and never backpropagate into the encoders (which only receive gradients via the multimodal and band losses). Adding to the confusion, Table A1 specifies the unimodal classifier input dimension as rather than the pooled , implying a flattened feature map is used instead. The mathematical formulation and architectural table do not match. -
Unconvincing verification of the RAM module's effectiveness. There are two issues regarding RAM. First, in the noise-perturbation experiment (Figure 3), the authors compare the full model against naive concatenation rather than against
RAMamba-Net w/o RAM(which is already available in Table 2). Because naive concatenation lacks bidirectional cross-attention, Figure 3 conflates the robustness of the cross-modal backbone with the contribution of RAM. Second, given the hyperparameters , , and , the modulation multiplier is strictly bounded within . It is hard to see how a scaling factor can meaningfully suppress a heavily corrupted modality. -
Inconsistent metric reporting and statistical tests. In Table 2, the reported Macro-F1 exceeds Accuracy for binary classification tasks (e.g., AVGC EEG-only: 74.23% Acc vs. 77.07% Macro-F1; DTU EEG-only: 87.03% Acc vs. 87.82% Macro-F1). Mathematically, for any single confusion matrix, Macro-F1 cannot exceed Accuracy. While averaging metrics separately across imbalanced trials or subjects could theoretically cause this artifact, the exact evaluation and aggregation protocol is left unspecified. This is particularly concerning given the chronological data split on AVGC, where the attended speaker switches midway through each trial. Furthermore, the main text claims significance is evaluated via paired t-tests, whereas Appendix C reports one-sided Wilcoxon signed-rank tests across 5 seeds.
Questions for authors
- Gradient path & dimensions: How is the unimodal supervision actually implemented in code? Please reconcile the
stopgradin Eq. (9)–(11) with the claim of training the encoders, and clarify why Table A1 uses an -dimensional input instead of . - RAM ablation under noise: How does
RAMamba-Net w/o RAMperform under the exact same noise corruption protocol shown in Figure 3? Also, given the tight bound on the modulation weights, could you show how the learned values actually shift as noise increases? - Evaluation protocol: How are Accuracy, BCA, and Macro-F1 aggregated across windows, trials, and subjects to produce Macro-F1 > Accuracy in Table 2? How does the chronological split handle the mid-trial attention switch in AVGC without creating severe class imbalance in the train/test splits?
7 comments
Q1—Unimodal supervision, gradient flow, and classifier dimensions
The stopgrad operation in Eq. (9) is intentional and serves to decouple the auxiliary reliability pathway from direct encoder optimization. Specifically, the mean-pooled descriptors used by the unimodal classifiers are detached from the EEG and EOG encoders. Consequently, the loss in Eq. (18) updates the unimodal classification heads but does not directly backpropagate into either encoder through this detached pathway. Both encoders nevertheless continue to be trained by the main multimodal classification loss, while the EEG encoder additionally receives band-wise supervision from . Thus, stopgrad does not freeze the encoders or remove discriminative supervision from the overall network. Rather, it separates how reliable auxiliary predictions are learned from how the shared representations are optimized.
This separation is important for RAM's prediction-consistency estimation. The supervised unimodal heads produce label-discriminative predictions, which make the agreement measure in Eq. (12) informative for the decoding task. Without such supervision, two uninformative predictions could agree without indicating that either modality is reliable. We have therefore clarified that primarily trains the auxiliary predictors that supply reliability cues, rather than directly preserving the encoder representations through its own gradient path.
We also correct the dimensional inconsistency in Appendix Table A1. Eqs. (9)–(11) operate on mean-pooled -dimensional modality descriptors, so the correct unimodal-head mapping is , not . This was a notation error in the architectural table and does not alter the model implementation or experimental results.
Q2—Corrected metrics and subject-level statistical evidence
We identified an error in batch-size handling during metric aggregation, which led to inconsistent results in the original Table 2. We corrected the aggregation and updated the affected ACC, BCA, Macro-F1, and Cohen's Kappa values, without changing the model architecture or the experimental split. For each subject and seed, the four metrics are computed from the same pooled test-window predictions, with class-wise precision, recall, and F1 calculated before taking the macro average. We then average the subject-level metrics within each seed, followed by aggregation across the five seeds, with the reported standard deviations calculated over the five seed-wise subject means. The corrected Table 2 no longer exhibits the originally noted Macro-F1 discrepancies. Importantly, the corrected performance ordering remains consistent with our main conclusion that RAMamba-Net benefits from EEG–EOG fusion and reliability-aware modulation.
We also distinguish the original five-seed statistical analysis from a new analysis that treats subjects as the independent statistical units, directly addressing the reviewer's concern. Specifically, we first average each subject's ACC over the five matched random seeds (41–45), then test paired subject-wise differences using two-sided paired -tests with Holm adjustment across four prespecified comparisons. We additionally performed a two-sided Wilcoxon signed-rank sensitivity analysis. The subject-level results are summarized below.
| Dataset | Comparison | Mean ACC | 95% CI | Holm-adjusted | Improved subjects |
|---|---|---|---|---|---|
| AVGC | RAMamba-Net − Concat. | +5.247 | [2.193, 8.301] | 0.0091 | 11/12 |
| AVGC | RAMamba-Net − w/o RAM | +2.821 | [1.562, 4.081] | 0.0018 | 11/12 |
| DTU | RAMamba-Net − Concat. | +1.236 | [0.368, 2.105] | 0.0160 | 14/18 |
| DTU | RAMamba-Net − w/o RAM | +0.520 | [0.098, 0.943] | 0.0188 | 13/18 |
All four subject-level comparisons remain statistically significant after Holm correction. The complementary Wilcoxon analysis gives adjusted values of 0.0420, 0.0039, 0.0361, and 0.0420, respectively, in the table order. Notably, RAMamba-Net improves over w/o RAM in 11/12 AVGC subjects and 13/18 DTU subjects, with positive mean gains of 2.821 and 0.520 percentage points. These findings show that RAM's contribution is supported by paired effects across subjects, rather than by seed-level variation alone. We also resolve the reporting inconsistency in the original statistical description. The reference to a one-sided Wilcoxon test in Appendix C was a methodological labeling error. The original five-seed significance analysis used paired -tests, as stated in Section 4.4, and we have aligned the revised appendix with the actual procedure. The new two-sided subject-level tests above are reported separately rather than conflated with that seed-level analysis.
Overall Response to Reviewer N5iV
We sincerely thank the reviewer for the careful evaluation and constructive comments. In response, we clarify the gradient pathway of unimodal supervision and correct the classifier dimension notation, revise the evaluation metrics affected by batch-size aggregation, and provide subject-level paired significance tests. We further clarify the within-trial partition around the AVGC attention switch and report an additional five-seed, three-fold cross-subject evaluation. Most importantly, we extend the robustness study by comparing RAMamba-Net with the identical backbone without RAM under matched noise conditions, adding fixed-weight and learned-gate controls, and reporting learned modality weights at every tested noise level. These additional analyses provide targeted evidence that RAM contributes beyond conventional fusion and that its reliability-aware calibration responds to changes in signal quality. We address the concerns in detail below.