acceptodds
Under review as a conference paper at ICLR 2027

RAMamba-Net: A Reliability-Aware and Mamba-Based Multimodal Fusion Network for Auditory Attention Detection

Abstract

Auditory attention decoding (AAD) identifies the attended speaker from physiological signals, supporting neuro-steered hearing devices and natural human-machine interaction. Electroencephalography (EEG) is the dominant modality for AAD but provides incomplete evidence in naturalistic audio-visual scenes, motivating EEG and electrooculography (EOG) fusion. Existing approaches remain limited by weak cross-modal interaction, inefficient temporal modeling, and low robustness to sample variations. To address the limitations, we propose RAMamba-Net, a reliability-aware Mamba-based multimodal fusion network for AAD. RAMamba-Net employs a Mamba-enhanced band-aware convolutional Transformer to capture band-specific EEG patterns and long-range temporal dynamics. A dual-branch temporal-spatial encoder models EOG temporal and inter-channel dependencies. Cross-modal attention enables explicit modality interaction. Then, a reliability-aware module is introduced to estimate sample-wise modality weights for feature and prediction consistency, thereby enhancing multimodal fusion. Experiments on two AAD benchmarks demonstrate that RAMamba-Net effectively exploits complementary EEG-EOG information, yielding accuracy gains of 5.76% over unimodal baselines, together with more robust decoding and discriminative representations. Further analyses show that explicit cross-modal interaction improves multimodal alignment, while the reliability-aware module suppresses unreliable modality evidence and is robust to signal perturbation and parameter variation.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Auditory Attention Decoding, Cross-Modal Fusion, Multimodal Learning, State Space Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_N5iVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RAMamba-Net for two-speaker auditory attention decoding (AAD). The architecture combines a Mamba-based band-aware EEG encoder, a spatio-temporal EOG encoder, bidirectional cross-attention, and a reliability-aware modulation (RAM) module that dynamically weights modalities based on feature descriptors and prediction agreement. Experiments and ablations are conducted on the AVGC and DTU datasets, including robustness tests under modality corruption.

Critical strengths

The evaluation includes useful internal controls. Concatenation, removal of RAM, eight combinations of RAM inputs, five initialization seeds, and modality-specific perturbations help assess the proposed fusion. Partitioning before windowing is also a sensible attempt to reduce overlap between evaluation and training data.

Critical weaknesses
  1. Contradictory formulation of the unimodal supervision pathway. The claimed purpose of the unimodal loss is to preserve the discriminative ability of the modality encoders. However, Eq. (9) applies stopgrad to ZmZ_m prior to mean pooling, Eq. (11) computes unimodal predictions from this detached descriptor, and Eq. (18) supervises those predictions. As formulated, gradients from Eq. (18) only update the linear classification heads and never backpropagate into the encoders (which only receive gradients via the multimodal and band losses). Adding to the confusion, Table A1 specifies the unimodal classifier input dimension as NmDN_m D rather than the pooled DD, implying a flattened feature map is used instead. The mathematical formulation and architectural table do not match.

  2. Unconvincing verification of the RAM module's effectiveness. There are two issues regarding RAM. First, in the noise-perturbation experiment (Figure 3), the authors compare the full model against naive concatenation rather than against RAMamba-Net w/o RAM (which is already available in Table 2). Because naive concatenation lacks bidirectional cross-attention, Figure 3 conflates the robustness of the cross-modal backbone with the contribution of RAM. Second, given the hyperparameters β=0.1\beta=0.1, ω=0.5\omega=0.5, and αm∈[0,1]\alpha_m \in [0, 1], the modulation multiplier is strictly bounded within [0.95,1.05][0.95, 1.05]. It is hard to see how a ±5%\pm 5\% scaling factor can meaningfully suppress a heavily corrupted modality.

  3. Inconsistent metric reporting and statistical tests. In Table 2, the reported Macro-F1 exceeds Accuracy for binary classification tasks (e.g., AVGC EEG-only: 74.23% Acc vs. 77.07% Macro-F1; DTU EEG-only: 87.03% Acc vs. 87.82% Macro-F1). Mathematically, for any single 2×22 \times 2 confusion matrix, Macro-F1 cannot exceed Accuracy. While averaging metrics separately across imbalanced trials or subjects could theoretically cause this artifact, the exact evaluation and aggregation protocol is left unspecified. This is particularly concerning given the chronological data split on AVGC, where the attended speaker switches midway through each trial. Furthermore, the main text claims significance is evaluated via paired t-tests, whereas Appendix C reports one-sided Wilcoxon signed-rank tests across 5 seeds.

Questions for authors
  1. Gradient path & dimensions: How is the unimodal supervision actually implemented in code? Please reconcile the stopgrad in Eq. (9)–(11) with the claim of training the encoders, and clarify why Table A1 uses an NmDN_m D-dimensional input instead of DD.
  2. RAM ablation under noise: How does RAMamba-Net w/o RAM perform under the exact same noise corruption protocol shown in Figure 3? Also, given the tight [0.95,1.05][0.95, 1.05] bound on the modulation weights, could you show how the learned αm\alpha_m values actually shift as noise increases?
  3. Evaluation protocol: How are Accuracy, BCA, and Macro-F1 aggregated across windows, trials, and subjects to produce Macro-F1 > Accuracy in Table 2? How does the chronological split handle the mid-trial attention switch in AVGC without creating severe class imbalance in the train/test splits?
7 comments
AuthorsOpenReview ↗

Overall Response to Reviewer N5iV

We sincerely thank the reviewer for the careful evaluation and constructive comments. In response, we clarify the gradient pathway of unimodal supervision and correct the classifier dimension notation, revise the evaluation metrics affected by batch-size aggregation, and provide subject-level paired significance tests. We further clarify the within-trial partition around the AVGC attention switch and report an additional five-seed, three-fold cross-subject evaluation. Most importantly, we extend the robustness study by comparing RAMamba-Net with the identical backbone without RAM under matched noise conditions, adding fixed-weight and learned-gate controls, and reporting learned modality weights at every tested noise level. These additional analyses provide targeted evidence that RAM contributes beyond conventional fusion and that its reliability-aware calibration responds to changes in signal quality. We address the concerns in detail below.

AuthorsOpenReview ↗

Q1—Unimodal supervision, gradient flow, and classifier dimensions

The stopgrad operation in Eq. (9) is intentional and serves to decouple the auxiliary reliability pathway from direct encoder optimization. Specifically, the mean-pooled descriptors used by the unimodal classifiers are detached from the EEG and EOG encoders. Consequently, the loss Luni\mathcal{L}_{\mathrm{uni}} in Eq. (18) updates the unimodal classification heads but does not directly backpropagate into either encoder through this detached pathway. Both encoders nevertheless continue to be trained by the main multimodal classification loss, while the EEG encoder additionally receives band-wise supervision from Lband\mathcal{L}_{\mathrm{band}}. Thus, stopgrad does not freeze the encoders or remove discriminative supervision from the overall network. Rather, it separates how reliable auxiliary predictions are learned from how the shared representations are optimized.

This separation is important for RAM's prediction-consistency estimation. The supervised unimodal heads produce label-discriminative predictions, which make the agreement measure in Eq. (12) informative for the decoding task. Without such supervision, two uninformative predictions could agree without indicating that either modality is reliable. We have therefore clarified that Luni\mathcal{L}_{\mathrm{uni}} primarily trains the auxiliary predictors that supply reliability cues, rather than directly preserving the encoder representations through its own gradient path.

We also correct the dimensional inconsistency in Appendix Table A1. Eqs. (9)–(11) operate on mean-pooled DD-dimensional modality descriptors, so the correct unimodal-head mapping is D→NcD\rightarrow N_c, not NmD→NcN_mD\rightarrow N_c. This was a notation error in the architectural table and does not alter the model implementation or experimental results.

AuthorsOpenReview ↗

Q2—Corrected metrics and subject-level statistical evidence

We identified an error in batch-size handling during metric aggregation, which led to inconsistent results in the original Table 2. We corrected the aggregation and updated the affected ACC, BCA, Macro-F1, and Cohen's Kappa values, without changing the model architecture or the experimental split. For each subject and seed, the four metrics are computed from the same pooled test-window predictions, with class-wise precision, recall, and F1 calculated before taking the macro average. We then average the subject-level metrics within each seed, followed by aggregation across the five seeds, with the reported standard deviations calculated over the five seed-wise subject means. The corrected Table 2 no longer exhibits the originally noted Macro-F1 discrepancies. Importantly, the corrected performance ordering remains consistent with our main conclusion that RAMamba-Net benefits from EEG–EOG fusion and reliability-aware modulation.

We also distinguish the original five-seed statistical analysis from a new analysis that treats subjects as the independent statistical units, directly addressing the reviewer's concern. Specifically, we first average each subject's ACC over the five matched random seeds (41–45), then test paired subject-wise differences using two-sided paired tt-tests with Holm adjustment across four prespecified comparisons. We additionally performed a two-sided Wilcoxon signed-rank sensitivity analysis. The subject-level results are summarized below.

DatasetComparisonMean Δ\DeltaACC95% CIHolm-adjusted ppImproved subjects
AVGCRAMamba-Net − Concat.+5.247[2.193, 8.301]0.009111/12
AVGCRAMamba-Net − w/o RAM+2.821[1.562, 4.081]0.001811/12
DTURAMamba-Net − Concat.+1.236[0.368, 2.105]0.016014/18
DTURAMamba-Net − w/o RAM+0.520[0.098, 0.943]0.018813/18

All four subject-level comparisons remain statistically significant after Holm correction. The complementary Wilcoxon analysis gives adjusted pp values of 0.0420, 0.0039, 0.0361, and 0.0420, respectively, in the table order. Notably, RAMamba-Net improves over w/o RAM in 11/12 AVGC subjects and 13/18 DTU subjects, with positive mean gains of 2.821 and 0.520 percentage points. These findings show that RAM's contribution is supported by paired effects across subjects, rather than by seed-level variation alone. We also resolve the reporting inconsistency in the original statistical description. The reference to a one-sided Wilcoxon test in Appendix C was a methodological labeling error. The original five-seed significance analysis used paired tt-tests, as stated in Section 4.4, and we have aligned the revised appendix with the actual procedure. The new two-sided subject-level tests above are reported separately rather than conflated with that seed-level analysis.

Reviewer_M1TyOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RAMamba-Net combines band-specific EEG encoding, an EOG encoder, cross-modal attention and sample-dependent fusion weights for auditory attention decoding. It is evaluated on AVGC and DTU using two-second windows. The reported accuracy improves over EEG-only and direct-concatenation baselines, with additional component ablations and noise tests.

Critical strengths
  1. The separate EEG and EOG encoders are a reasonable design, and the paper compares them with several unimodal alternatives. The AVGC improvement from 74.23% to 79.99% is large enough to merit attention.
  2. The RAM component ablations and modality-specific noise tests are useful attempts to examine the fusion mechanism beyond clean-set accuracy.
Critical weaknesses
  1. The evaluation protocol leaves uncertainty about independence between training and testing. Section 4.2 partitions each subject’s signals chronologically, but does not clearly state whether entire trials or sessions are held out. Segments from the same trial may share slow signal structure even without overlapping windows. Appendix C tests five random seeds rather than subject-level paired effects. Clear split boundaries and per-subject results are needed to assess the reliability of the gains.
  2. The evidence for reliability estimation is incomplete. With β = 0.1 and ω = 0.5, Equation 15 only rescales each branch within 0.95–1.05, after cross-modal information has already been mixed. Figure 3 compares against concatenation, so it does not isolate RAM’s contribution to robustness. Showing how the learned weights respond to corruption, and comparing with the same backbone without RAM or with fixed weights, would test the stated mechanism directly.
Questions for authors
  1. Clarification: Can one trial contribute to multiple splits? Is CSP fitted only after removing the validation subset from the training pool?
  2. Additional analysis: Under the same noise conditions, how do RAM weights and accuracy compare with the identical backbone using no RAM or fixed fusion weights?
9 comments
AuthorsOpenReview ↗

Overall Response to Reviewer M1Ty

We sincerely thank the reviewer for the careful assessment and constructive comments. We agree that a convincing evaluation of RAMamba-Net should distinguish the scope of the original within-trial protocol from generalization across participants, and should isolate the contribution of reliability-aware modulation from ordinary fusion. In response, we clarify the trial partition and CSP fitting procedure, correct the metric aggregation, and provide subject-level paired statistics with confidence intervals and per-participant results. We further report a five-seed, three-fold subject-disjoint evaluation on AVGC, together with new controls using the same cross-modal backbone without RAM, fixed modality weights, and a learned scalar gate. Finally, we compare RAMamba-Net and w/o RAM under matched EEG/EOG corruption and report the learned modality weights at every noise level. These complementary results show that RAMamba-Net achieves consistent decoding improvements and that its reliability-aware calibration contributes beyond conventional feature fusion and weighting. We address each concern below.

AuthorsOpenReview ↗

Q1—Trial partitioning, temporal dependence, and CSP fitting

Our main results follow a subject-dependent, within-trial evaluation protocol. For AVGC, each selected 10-minute trial contains two 5-minute portions with opposite attended directions. The evaluated subset contains two selected trials for 11 participants and one for S14. DTU includes 60 approximately 50-second two-speaker trials per participant. We use 2-second decision windows with a 1-second stride (50% overlap) and a chronological training–test division, with validation data subsequently drawn from the original training pool. This establishes a matched within-trial benchmark for all compared methods. Training and test windows are separated at the temporal boundary so that they do not share raw signal samples. We distinguish the original within-trial scores from the subject-disjoint evaluation reported in Q3, where no trial or participant is shared between the supervised training and test groups.

CSP is fitted using the original training pool, without accessing held-out test signals or labels. In the original preprocessing order, the validation subset is selected from this pool after CSP fitting, rather than being removed before spatial-filter estimation. We state this order explicitly to avoid conflating test-set exclusion with validation-set exclusion. The main comparisons use the same preprocessing and partitioning protocol across models, while the additional subject-disjoint experiment directly evaluates the method beyond the original within-trial setting.

AuthorsOpenReview ↗

Q2—Corrected metrics and subject-level statistical evidence

To address the concern that the original Appendix C relied on five random seeds rather than paired participant effects, we reanalyzed the corrected main results with subjects as the statistical units. We identified and corrected a batch-size handling error in the metric aggregation without changing the network architecture or the original split assignments. For each participant, we average ACC over the same five seeds (41–45) and then compare the paired subject-wise differences using two-sided paired t-tests, with Holm correction across four planned comparisons. This evaluates whether the observed gains persist across participants, rather than merely across random initializations.

DatasetComparisonΔACC95% CIHolm-adjusted pImproved subjects
AVGCRAMamba-Net − Concat.+5.247[2.193, 8.301]0.009111/12
AVGCRAMamba-Net − w/o RAM+2.821[1.562, 4.081]0.001811/12
DTURAMamba-Net − Concat.+1.236[0.368, 2.105]0.016014/18
DTURAMamba-Net − w/o RAM+0.520[0.098, 0.943]0.018813/18

All four comparisons remain significant after correction. In particular, RAMamba-Net outperforms the identical backbone without RAM for 11 of 12 AVGC participants and 13 of 18 DTU participants, yielding mean ACC improvements of 2.821 and 0.520 percentage points, respectively. A complementary two-sided Wilcoxon signed-rank sensitivity analysis also yields Holm-adjusted p values of 0.0420, 0.0039, 0.0361, and 0.0420 in the same row order. The consistency of these subject-level findings supports the contributions of multimodal interaction and reliability-aware modulation, including on DTU where the absolute gain is smaller. The original five-seed analysis and this new subject-level analysis address different sources of variation and are reported separately.

Reviewer_WGZvOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes RAMamba-Net for auditory attention decoding from EEG and EOG. The EEG branch is FAConformer with two Mamba blocks added per band. The EOG branch is DBConformer, with a temporal and a spatial Transformer. The two token streams exchange information through standard bidirectional cross-attention. A reliability module then predicts two modality weights. Its inputs are the pooled features, a cosine similarity, and a Jensen-Shannon agreement between two unimodal predictions. The weights rescale the fused tokens before a linear classifier. Training adds a unimodal loss and a band-wise loss. Experiments use AVGC and DTU. The split is subject-dependent and chronological, 90/10, with 2 second windows. The full model reaches 79.99% on AVGC and 88.42% on DTU, against 74.23% and 87.03% for EEG only. The paper adds t-SNE plots, a noise injection study, an ablation of the reliability terms, and a hyperparameter sweep.

Critical strengths

Two public datasets, five seeds, four metrics, and ablations. It is easy to see what was run. Table 1 benchmarks 13 models on EEG alone and on EOG alone. It is a useful reference. It also shows that EOG alone is well above chance.

Critical weaknesses

The central claim is not separated from a gaze shortcut. The task is left versus right attention. EOG tracks where the eyes point. The paper cites Rotaru et al., who show that decoders pick up gaze in this setting, and AVGC comes from that work. The paper still runs no control. It does not say whether AVGC gaze conditions are used or mixed. It mentions no ocular artifact removal for AVGC EEG, so EEG-only may already contain gaze information. If EOG helps because it reveals gaze, the model is a gaze tracker and not an attention decoder. The neuro-steered hearing motivation then does not follow. The protocol adds to this. Training and test data come from the same recordings of the same subject, split by time. Test size, class balance, and window overlap are not given. There is no cross-subject or cross-trial result. The conclusion lists this as future work. The reported numbers do not fit together, and the statistics are weak. For a binary task, macro-F1 cannot exceed accuracy. Table 2 reports it above accuracy in every row that uses EEG. AVGC EEG-only has 77.07 against 74.23. DTU full has 89.04 against 88.42. Kappa is also too high. At 74.23% accuracy, kappa is about 0.48 for balanced classes and cannot exceed about 0.52 in any case. Table 2 reports 0.58. DTU full has 0.80 where the bound is about 0.77. Accuracy, BCA, macro-F1, and kappa do not seem to come from one confusion matrix. This damages trust in every table. The Wilcoxon tests pair five seeds on one fixed test set. Seeds change only the initialisation. With five pairs the smallest one-sided p-value is 0.031. So the tests show the model won in each seed. They say nothing about new subjects. The pairing should be over subjects. Several gains are within noise. On DTU the full model beats the version without RAM by 0.52 points, with p of 0.062 for accuracy and 0.219 for F1. In Table 3 many single-term gains on AVGC are below one standard deviation, for example 78.54 against 78.40. The named contributions are not tested. Mamba. The EEG-only baseline is FAConformer without Mamba. Nothing compares the two. The efficiency argument is not measured. Windows are 2 seconds, so a few hundred samples. No parameters, FLOPs, or run time are reported. Baselines. The only fusion comparisons are concatenation and the authors’ own variant without RAM. Prior EEG and EOG work, such as Fan et al. 2025, which is cited, is not run. Late fusion, gated fusion, and a learned scalar gate are missing. RAM. After the softmax the two weights sum to one. With \beta=0.1 and \omega=0.5, the factor 1+\beta(\alpha-\omega) stays between 0.95 and 1.05. That is a 5% rescale ahead of a linear classifier. Fig. 4 shows flat accuracy for \beta from 0.025 to 0.8. This fits a module whose weights barely move. No figure shows the learned weights, on clean data or under noise. Nothing trains the weights to track reliability. Agreement between two predictions is also not reliability, since both can be wrong together. Noise study. It compares only with Concat, so cross-attention is mixed in. The noise model is not defined. The claim that RAM suppresses unreliable inputs is untested. Losses. Eq. 9 applies stop-gradient to the pooled features. So \mathcal{L}{\mathrm{uni}} cannot shape the encoders through them, yet the text says it preserves encoder discriminability. Table A1 also gives the unimodal classifier a flattened input, not a pooled one. Neither \mathcal{L}{\mathrm{uni}} nor \mathcal{L}_{\mathrm{band}} is ablated.

Questions for authors

How many trials, windows, and test windows per subject? What is the class balance of the test split, and do windows overlap? Which AVGC conditions are used? How were accuracy, BCA, macro-F1, and kappa computed? Please give the confusion matrices. Macro-F1 above accuracy is not possible for a binary task. What happens if ocular activity is regressed out of EEG? What happens if EOG is replaced by a gaze direction signal? Does the gain stay on data where gaze is controlled? What are the results per subject, with a paired test over subjects? What are the cross-trial and cross-subject results? How does Mamba-enhanced FAConformer compare with FAConformer alone in the EEG-only setting? Please report parameters, FLOPs, and run time. What do late fusion, a learned scalar gate, and fixed weights of 0.5 give? What do Fan et al. 2025 and Xu et al. 2026 give under your split? What is the distribution of \alpha on clean data and at each noise level? What is the result of the noise study for the variant without RAM? How is noise generated, and at which stage? How were \beta, \omega, and \lambda_{\mathrm{uni}} chosen, and on which split? How were the 13 baselines tuned? Was t-SNE drawn on training or test features? Does \mathcal{L}{\mathrm{uni}} update the encoders given the stop-gradient in Eq. 9? What do the results look like without \mathcal{L}{\mathrm{uni}} and without \mathcal{L}_{\mathrm{band}}?

9 comments
AuthorsOpenReview ↗

Overall Response to Reviewer WGZv

We sincerely thank the reviewer for the thorough evaluation and constructive suggestions. Your comments have helped us clarify the role of EOG in auditory attention decoding, strengthen the experimental controls, and provide a more comprehensive assessment of RAMamba-Net. In response, we have investigated additional gaze-controlled conditions, corrected the evaluation metrics, and conducted subject-level statistical tests and cross-subject experiments. We have also performed targeted ablations of the temporal modeling modules, introduced additional fusion baselines, and reported computational complexity and inference efficiency. Furthermore, we have clarified the behavior of reliability-aware modulation under different noise levels and the role of auxiliary supervision. These analyses provide stronger empirical evidence for the effectiveness of EEG–EOG multimodal fusion and reliability-aware modeling. We address the reviewer's concerns in detail below and indicate the corresponding results and revisions.

AuthorsOpenReview ↗

Q1— Multimodal objective and the gaze-shortcut concern

Our study focuses on EEG–EOG multimodal auditory attention decoding. EOG is intentionally incorporated to provide behavioral information complementary to EEG. In natural listening, people often orient their gaze toward the attended speaker [1]. Eye movements also track prioritized acoustic features of attended speech even in multi-speaker listening [2]. This relationship makes eye-movement information a meaningful auxiliary cue, rather than an unintended input. Ocular artifacts in EEG were also addressed during preprocessing.

To directly examine the gaze-shortcut concern, we added three AV-GC-AAD conditions in which gaze is no longer fixed to the attended speaker. NoVisuals requires central fixation. MovingVideo and MovingTargetNoise require following a horizontally moving visual target. We evaluated EEG-only, EOG-only, and RAMamba-Net using five random seeds and the original subject-dependent protocol. S1 and S3 have no MovingTargetNoise recording and were excluded only from that condition.

ConditionSubjectsEEG-only ACC (%)EOG-only ACC (%)RAMamba-Net ACC (%)Full − EEG
MovingVideo1268.6859.0368.88+0.20
MovingTargetNoise1063.4260.8362.92−0.50
NoVisuals1272.2660.7574.11+1.85
StaticVideo1274.2367.3079.99+5.76

RAMamba-Net achieved above-chance accuracy in all three conditions, with subject-level Holm-adjusted p values of 0.000527, 0.0144, and 0.000013. In particular, it retained 68.88% and 62.92% accuracy when visual tracking was spatially decoupled from auditory attention. These results directly show that the decoder's predictive ability extends beyond the fixed gaze–label relationship in FixedVideo. EOG-only remained near 60% on average, while the complete model performed substantially better. In NoVisuals, RAMamba-Net improved on EEG-only in 10 of 12 subjects, with a mean gain of 1.85 percentage points. The smaller fusion gains during moving-target tracking are consistent with EOG cues becoming less directly associated with the attended direction, alongside changes in visual task demands. The EEG-only results are based on the ocular-artifact-attenuated EEG used in our pipeline. A gaze-direction-only substitute would require calibrated gaze coordinates, which are not supplied as a model-input stream in these EEG/EOG recordings. Accordingly, the EOG-only baseline and the controlled-gaze conditions provide direct controls using the available signals, rather than replacing EOG with attention labels or an uncalibrated proxy.

References

[1] Ulrich Pomper and Maria Chait. The impact of visual gaze direction on auditory object tracking. Scientific Reports, 7(1):4640, 2017.

[2] Quirin Gehmacher, Juliane Schubert, Fabian Schmidt, Thomas Hartmann, Patrick Reisinger, Sebastian Rösch, Konrad Schwarz, Tzvetan Popov, Maria Chait, and Nathan Weisz. Eye movements track prioritized auditory features in selective attention to natural speech. Nature Communications, 15(1):3692, 2024.

AuthorsOpenReview ↗

Q2—Data protocol and correction of evaluation metrics

The main experiments use a subject-dependent chronological split with 2-second windows and 50% overlap. Training and test windows are separated at the temporal boundary so that they do not share raw signal samples. The class distribution is approximately balanced. CSP is fitted on the training pool without access to test recordings or test labels. For AVGC, the selected main-experiment subset contains two 10-minute trials for 11 participants and one for S14. Each trial contains two 5-minute portions with opposite labels. At 128 Hz, a 5-minute portion yields 299 candidate 2-second windows with a 1-second stride, and the original 90/10 partition reserves 29 test windows per portion. DTU contains 60 shorter two-speaker trials per participant. We obtain all four metrics from the pooled test-window predictions for each subject and seed, then aggregate subject and seed summaries after correcting the batch-size accounting.

We identified an error in batch-size handling during metric aggregation, which caused incorrect values in the previously reported results. We corrected the evaluation calculations and updated all affected entries in revised Table 2. ACC, BCA, Macro-F1, and Kappa now follow the same corrected evaluation protocol. The correction concerns the reporting calculation, rather than a change in the model architecture or the experimental partition. Importantly, the corrected results preserve the original performance trends and main conclusions. RAMamba-Net consistently outperforms EEG-only, Concat., and w/o RAM on both datasets, demonstrating the effectiveness and advantages of EEG–EOG multimodal fusion and reliability-aware feature modulation for auditory attention decoding.

Comments on the paper

1 comment
Area_Chair_oRH5OpenReview ↗

Please Engage in the Author–Reviewer Discussion

Dear Reviewers,

As the author–reviewer discussion period is approaching its deadline (November 18, AoE), I would like to kindly remind you to review the authors' rebuttals and actively engage in the discussion as soon as possible.

Please address any outstanding questions or concerns and update your assessments and ratings if appropriate.

Thank you very much for your time and efforts!

Best regards, Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.